Mailman 3 January 2008 - Python-ideas

Ideas towards GIL removal
by Greg Ewing July 18, 2008

July 18, 2008

I've been thinking about some ideas for reducing the amount of refcount adjustment that needs to be done, with a view to making GIL removal easier. 1) Permanent objects In a typical Python program there are many objects that are created at the beginning and exist for the life of the program -- classes, functions, literals, etc. Refcounting these is a waste of effort, since they're never going to go away. So perhaps there could be a way of marking such objects as "permanent" or "immortal". … [View More]

8 19

adopt an enum type for the standard library?
by Mark Summerfield April 12, 2008

April 12, 2008

AFAIK if you want enumerations you must either create your own, or use a third party module, or use namedtuple: Files = collections.namedtuple("Files", "minimum maximum")(1, 200) ... x = Files.minimum Using: MINIMUM, MAXIMUM = 1, 200 is often inconvenient, since you might have several different ones. Of course you could do: MIN_FILES = 1 MIN_DIRS = 0 Personally, I like enums and consider them to be a fundamental, but I don't like the above approaches. There is an … [View More]

10 14

non-Pre-PEP: Syntactic replacement of built-in types with user-defined callables (Take 2... fewer newlines!)
by Taro Jan. 31, 2008

Jan. 31, 2008

Hi all... again, Built-in types such as float, string, or list are first-class citizens in Python sourcefiles, having syntactic support: myfloat = 1.0 mystring = "my string" mylist = [1,2,4,8] mydict = {1:"a", 2:"b", 3:"c"} myset = {1,2,3} User-defined classes are second-class citizens, requiring data to be manually converted from a type: mydecimal = Decimal("1.00000000000000001") myrope = Rope("my rope") myblist = BList([1,2,4,8]) myordereddict = … [View More]OrderedDict((1,"a"), (2, "b"), (3, "c")) myfrozenset = frozenset([1,2,3]) If there's only one or two conversions needed in a file, then such conversion is not particularly burdensome, but if one wants to consistently use (say) decimals throughout a file then the ability to use a literal syntax makes for prettier source. Some languages have open types, allowing the addition of methods to built-in types. This is not considered desired behaviour for Python, since modifications made in one module can potentially affect code in other modules. A typedef is syntactic sugar to allow user-defined replacements to be treated as first-class citizens in code. They affect only the module in which they appear and do not modify the original type. To be typedeffable, something must be a builtin type, have a constant/syntactic representation, and be callable. Hence, typedeffable types would be limited to complex, dict, float, int, ?object?, list, slice, set, string, and tuple. No modification is made to __builtins__ or types, so conversion and/or reference to the original type is still possible. The syntax for a typedef is: from MODULE typedef ADAPTOR as TYPE OR typedef BUILTINADAPTOR as TYPE Syntactic constants of a given type are then wrapped with a call: ADAPTOR(SOURCELITERAL) where SOURCELITERAL is the string that appears in the sourcecode eg: from decimal typedef Decimal as float i = 1.000000000000000000000000000001 translates as: from decimal import Decimal as float i = Decimal("1.000000000000000000000000000001") Syntactic collections of a given type are always provided with a list of objects, eg: from decimal typedef Decimal as float from blist typedef BList as list b = [1.1, 4.2] translates as: from decimal import Decimal as float from blist import BList as list b = Blist([Decimal("1.1"), Decimal("4.2")]) and from collections typedef OrderedDict as dict d = {1:"a", 2:"b", 3:"c"} as: from collections import OrderedDict as dict d = OrderedDict([(1,"a"), (2,"b"), (3,"c")]) A typedef appears at the start of a module immediately after any __future__ imports. As no adaptors can be defined in a module before a typedef and typedefs are in no way a forward declaration, "typedef ADAPTOR as TYPE" only works for builtins, since to do otherwise would lead to one of two unpalatable options; either: a/ definition of adaptors would have to be allowed pre-typedef, which would allow them to be buried in code, making them far easier to miss; or b/ adaptors would be defined after the typedef, which means that you'd have to handle: typedef float as float def float(): pass or: typedef MyObject as object class MyObject(object): pass or: typedef myfloat as float x = 1.1 def myfloat(): pass It is true that if a valid typedef is made, the type can be redefined within the module; but the "consenting adults" rule applies -- it's possible to redefine str and float multiple times within a module as well, but that isn't recommended either (and the typedef at the top of the module at least indicates that non-standard behaviour is to be expected) It is a SyntaxError to typedef the same type more than once: from decimal typedef Decimal as float from types typedef FloatType as float #SyntaxError("Type 'float' already redefined.") Spelling: "typedef" is prettier than "pragma", and less likely to be in use than "use" but its behaviour is somewhat different from C's typedef, so perhaps another term may be preferred. Theoretical Performance Differences: Since a typedef is purely syntactic sugar, and all tranformational work would be done at compilation, running code should be no slower (there should be no new opcodes necessary) and by default no faster that performing manual conversion (though they may assist an optimisation). Unless optimisation magic is available, performance-critical code should be careful when typedeffing not to use typedeffed literals in inner loops. I know it's considered polite to provide code, but any implementation would have to be in C, so please accept this extremely fake dummy implementation which in no way resembles the way things really work as a poor substitute: typedefs = { FLOATTYPE: None, INTTYPE: None, LISTTYPE: None, DICTTYPE: None, ... } while True line = lines.next() type, module, adaptor = parsetypedef(line) if type is None: break if typedefs[type] is not None: raise SyntaxError("Typedef redef") typedefs[type] = adaptor if module is not None: emit_bytecode_for_import_from(type, module, adaptor) else: emit_bytecode_for_assignment(type, adaptor) parse([line] + lines) ... def emit_float(...): if typedefs[FLOATTYPE] is not None: emit_constant(typedefs[FLOATTYPE][0], stringliteral) else: ... # standard behaviour def emit_list(...): if typedefs[LISTTYPE] is not None: emit(LOADGLOBAL, typedefs[LISTTYPE]) # standard behaviour if typedefs[LISTTYPE] is not None: emit(CALL_FUNCTION) # standard behaviour All rights are assigned to the Python Software Foundation [View Less]

6 8

Re: [Python-ideas] non-Pre-PEP: Syntactic replacement of built-in types with user-defined callables (Take 2... fewer newlines!)
by Raymond Hettinger Jan. 31, 2008

Jan. 31, 2008

> If I understood your proposal correctly, it's just about > dynamically *replacing* the built-in types (with their > respective syntax) with some type of your own. FWIW, there is an interesting type replacement recipe on the bottom of the page at: http://docs.python.org/lib/module-tokenize.html Raymond

1 0

Re: [Python-ideas] non-Pre-PEP: Syntactic replacement of built-in pes with user-defined callables.
by Matt Chisholm Jan. 28, 2008

Jan. 28, 2008

I don't think I followed all of what you wrote. However, it would certainly be convenient shorthand to be able to say that, for an entire file, program, or module, the float literal syntax created Decimals instead. And I can imagine cases where an entire app or module was always using ordered dicts instead of dicts, or frozensets instead of sets. -matt

1 0

non-Pre-PEP: Syntactic replacement of built-in types with user-defined callables.
by Taro Jan. 28, 2008

Jan. 28, 2008

Hi all, Built-in types such as float, string, or list are first-class citizens in Python sourcefiles, having syntactic support: myfloat = 1.0 mystring = "my string" mylist = [1,2,4,8] mydict = {1:"a", 2:"b", 3:"c"} myset = {1,2,3} User-defined classes are second-class citizens, requiring data to be manually converted from a type: mydecimal = Decimal("1.00000000000000001") myrope = Rope("my rope") myblist = BList([1,2,4,8]) myordereddict = … [View More]OrderedDict((1,"a"), (2, "b"), (3, "c")) myfrozenset = frozenset([1,2,3]) If there's only one or two conversions needed in a file, then such conversion is not particularly burdensome, but if one wants to consistently use (say) decimals throughout a file then the ability to use a literal syntax makes for prettier source. Some languages have open types, allowing the addition of methods to built-in types. This is not considered desired behaviour for Python, since modifications made in one module can potentially affect code in other modules. A typedef is syntactic sugar to allow user-defined replacements to be treated as first-class citizens in code. They affect only the module in which they appear and do not modify the original type. To be typedeffable, something must be a builtin type, have a constant/syntactic representation, and be callable. Hence, typedeffable types would be limited to complex, dict, float, int, ?object?, list, slice, set, string, and tuple. No modification is made to __builtins__ or types, so conversion and/or reference to the original type is still possible. The syntax for a typedef is: from MODULE typedef ADAPTOR as TYPE OR typedef BUILTINADAPTOR as TYPE Syntactic constants of a given type are then wrapped with a call: ADAPTOR(SOURCELITERAL) where SOURCELITERAL is the string that appears in the sourcecode eg: from decimal typedef Decimal as float i = 1.000000000000000000000000000001 translates as: from decimal import Decimal as float i = Decimal("1.000000000000000000000000000001") Syntactic collections of a given type are always provided with a list of objects, eg: from decimal typedef Decimal as float from blist typedef BList as list i = 1.000000000000000000000000000001 b = [1.1, 4.2] translates as: from decimal import Decimal as float from blist import BList as list b = Blist([Decimal("1.1"), Decimal("4.2")]) and from collections typedef OrderedDict as dict d = {1:"a", 2:"b", 3:"c"} as: from collections import OrderedDict as dict d = OrderedDict([(1,"a"), (2,"b"), (3,"c")]) A typedef appears at the start of a module immediately after any __future__ imports. As no adaptors can be defined in a module before a typedef and typedefs are in no way a forward declaration, "typedef ADAPTOR as TYPE" only works for builtins, since to do otherwise would lead to one of two unpalatable options; either: a/ definition of adaptors would have to be allowed pre-typedef, which would allow them to be buried in code, making them far easier to miss; or b/ adaptors would be defined after the typedef, which means that you'd have to handle: typedef float as float def float(): pass or: typedef MyObject as object class MyObject(object): pass or: typedef myfloat as float x = 1.1 def myfloat(): pass It is true that if a valid typedef is made, the type can be redefined within the module; but the "consenting adults" rule applies -- it's possible to redefine str and float multiple times within a module as well, but that isn't recommended either (and the typedef at the top of the module at least indicates that non-standard behaviour is to be expected) It is a SyntaxError to typedef the same type more than once: from decimal typedef Decimal as float from types typedef FloatType as float #SyntaxError("Type 'float' already redefined.") Spelling: "typedef" is prettier than "pragma", and less likely to be in use than "use" but its behaviour is somewhat different from C's typedef, so perhaps another term may be preferred. Theoretical Performance Differences: Since a typedef is purely syntactic sugar, and all tranformational work would be done at compilation, running code should be no slower (there should be no new opcodes necessary) and by default no faster that performing manual conversion (though they may assist an optimisation). Unless optimisation magic is available, performance-critical code should be careful when typedeffing not to use typedeffed literals in inner loops. I know it's considered polite to provide code, but any implementation would have to be in C, so please accept this extremely fake dummy implementation which in no way resembles the way things really work as a poor substitute: typedefs = { FLOATTYPE: None, INTTYPE: None, LISTTYPE: None, DICTTYPE: None, ... } while True line = lines.next() type, module, adaptor = parsetypedef(line) if type is None: break if typedefs[type] is not None: raise SyntaxError("Typedef redef") typedefs[type] = adaptor if module is not None: emit_bytecode_for_import_from(type, module, adaptor) else: emit_bytecode_for_assignment(type, adaptor) parse([line] + lines) ... def emit_float(...): if typedefs[FLOATTYPE] is not None: emit_constant(typedefs[FLOATTYPE][0], stringliteral) else: ... # standard behaviour def emit_list(...): if typedefs[LISTTYPE] is not None: emit(LOADGLOBAL, typedefs[LISTTYPE]) # standard behaviour if typedefs[LISTTYPE] is not None: emit(CALL_FUNCTION) # standard behaviour All rights are assigned to the Python Software Foundation [View Less]

1 0

More on set literals
by VanL Jan. 26, 2008

Jan. 26, 2008

I like Raymond's suggestion (apparently adopted) that {1,2,3} is a frozenset literal. However, I also like the {{1,2,3}} syntax. So I would propose that {{}} become the literal for the *mutable* set, not the frozenset. So: frozenset() # empty frozenset {1,2,3} # frozenset literal {} # empty dict {1:2, 3:4} #dict literal {{}} (or set()) # empty mutable set {{1,2,3}} # set literal My rationale is as follows: 1. It visually distinguishes sets and frozensets, without making them take up a … [View More]

5 4

A number is a number
by Neil Toronto Jan. 26, 2008

Jan. 26, 2008

Isn't it? Python 3.0 integers are just integers. Now ceil and floor return ints from float arguments. It seems like things are moving closer to just having a single "number" type. For most applications, because of duck typing, you'd never know or care whether you're punting a float, int, or complex number around, and you'd usually not care. Why not just go whole-hog and unify them all? Neil

2 1

Re: [Python-ideas] An easier syntax for writing decorators(&similar things)?
by Steven Bethard Jan. 25, 2008

Jan. 25, 2008

On Jan 24, 2008 3:26 PM, Aaron Brady <castironpi(a)comcast.net> wrote: > > I don't consider that a use case, or real code. ;-) Yes, you can > > construct curry with it. But what do you want to use curry for? Show > > me some actual Python packages that use the curry function (or your > > prepartial function) and then we can talk. > > Original function: > 1. urlparse( urlstring[, default_scheme[, allow_fragments]]) > 2. urlretrieve( url[, … [View More]

2 1

Re: [Python-ideas] An easier syntax for writing decorators (& similar things)?
by Aaron Brady Jan. 24, 2008

Jan. 24, 2008

> -----Original Message----- > On 8 Oct 2007, at 10:57, Arnaud Delobelle wrote: > > > > On Mon, October 8, 2007 4:33 am, Adam Atlas wrote: > >> When writing decorators especially when it's one that needs arguments > >> other than the function to be wrapped, it often gets rather ugly... > [...] > > Whhy not create a (meta-)decorator to do this? Something like: > [...] Following up post from 10/8/07. > To follow up on my untested suggestion, … [View More]

4 16