Mailman 3 March 2008 - Python-ideas

Ideas towards GIL removal
by Greg Ewing July 18, 2008

July 18, 2008

I've been thinking about some ideas for reducing the amount of refcount adjustment that needs to be done, with a view to making GIL removal easier. 1) Permanent objects In a typical Python program there are many objects that are created at the beginning and exist for the life of the program -- classes, functions, literals, etc. Refcounting these is a waste of effort, since they're never going to go away. So perhaps there could be a way of marking such objects as "permanent" or "immortal". Any refcount operation on a permanent object would be a no-op, so no locking would be needed. This would also have the benefit of eliminating any need to write to the object's memory at all when it's only being read. 2) Objects owned by a thread Python code creates and destroys temporary objects at a high rate -- stack frames, argument tuples, intermediate results, etc. If the code is executed by a thread, those objects are rarely if ever seen outside of that thread. It would be beneficial if refcount operations on such objects could be carried out by the thread that created them without locking. To achieve this, two extra fields could be added to the object header: an "owning thread id" and a "local reference count". (The existing refcount field will be called the "global reference count" in what follows.) An object created by a thread has its owning thread id set to that thread. When adjusting an object's refcount, if the current thread is the object's owning thread, the local refcount is updated without locking. If the object has no owning thread, or belongs to a different thread, the object is locked and the global refcount is updated. The object is considered garbage only when both refcounts drop to zero. Thus, after a decref, both refcounts would need to be checked to see if they are zero. When decrementing the local refcount and it reaches zero, the global refcount can be checked without locking, since a zero will never be written to it until it truly has zero non-local references remaining. I suspect that these two strategies together would eliminate a very large proportion of refcount-related activities requiring locking, perhaps to the point where those remaining are infrequent enough to make GIL removal practical. -- Greg

8 19

adopt an enum type for the standard library?
by Mark Summerfield April 12, 2008

April 12, 2008

AFAIK if you want enumerations you must either create your own, or use a third party module, or use namedtuple: Files = collections.namedtuple("Files", "minimum maximum")(1, 200) ... x = Files.minimum Using: MINIMUM, MAXIMUM = 1, 200 is often inconvenient, since you might have several different ones. Of course you could do: MIN_FILES = 1 MIN_DIRS = 0 Personally, I like enums and consider them to be a fundamental, but I don't like the above approaches. There is an enum module in PyPI http://pypi.python.org/pypi/enum/ and there are several versions in the Python Cookbook. Wouldn't one of these be worth adopting for the standard library? -- Mark Summerfield, Qtrac Ltd., www.qtrac.eu

10 14

dictionary unpacking
by Mathias Panzenböck April 2, 2008

April 2, 2008

Maybe dictionary unpacking would be a nice thing? >>> d = {'foo': 42, 'egg': 23} >>> {'foo': bar, 'egg': spam} = d >>> print bar, spam 42 23 What do you think? Bad idea? Good idea? -panzi

7 9

Lambda again: Anonymous function definition
by Joel Bender March 31, 2008

March 31, 2008

Greg wrote: > What's needed is something very concise and unobtrusive, > such as > > x, y => x + y As inspired by Prolog: x, y :- x + y So this: f = lambda x, y: x ** 2 + y ** 2 Or this: def f(x, y): return x ** 2 + y ** 2 Becomes this: f = x, y :- x ** 2 + y ** 2 And would logically transpose into this: f(x, y) :- x ** 2 + y ** 2 Oooo...this rabbit hole is fun! Joel

3 2

Re: [Python-ideas] lambda
by Malte Helmert March 27, 2008

March 27, 2008

[follow up from py3k.devel list] Neil Toronto wrote: > Yep. In my seven years of CS instruction so far, I've only come across > this once, in a theory of programming languages course. "Lambda" simply > doesn't show up unless you do language theory or program in a Lisp... or > in Python. Since you mention Haskell below: > It's a little less terse than Haskell's "\->" it's worth pointing out that Haskell uses the backslash syntax because it is the nearest ASCII equivalent to the (lower-case) letter lambda. For example, see http://en.wikibooks.org/wiki/Haskell/More_on_functions or the Google results for "haskell lambda backslash" (without the quotes). Malte

2 1

Lambda again: unnamed arguments
by Giovanni Bajo March 27, 2008

March 27, 2008

Hello, inspired by Greg's post about ideas on making the lambda syntax more concise, like: x,y => x+y I was wondering if using unnamed arguments had already been debated. Something like: \(_1+_2) where basically you're declaring implicitally declaring that your lambda takes two arguments. You wouldn't be able to call them through keyword arguments, nor to accept a variable number of arguments (nor to accept more arguments than they are actually used), but wouldn't it cover most use cases and be really compact? Other examples: k.sort(key=\(_1.foo)) k.sort(key=\(_1[0])) -- Giovanni Bajo

2 1

Lib/lib-tk goes away, package tkinter joins in
by Guilherme Polo March 25, 2008

March 25, 2008

Hello, (this is an idea for Python 3) Is there any reason for keeping the directory lib-tk at Lib ? I believe renaming it to tkinter and making it a package would make more sense. Tkinter module's code could then reside into __init__.py maybe. Other change that could be done in this package would be renaming some modules: Dialog -> dialog FileDialog -> filedialog FixTk -> fixtk ... Also, I believe tkSimpleDialog and dialog could be in a single module. There are other modules like tkColorChooser and tkCommonDialog and even tkSimpleDialog (and some others) that I'm not totally sure what to do about them, but for me they should reside in possible a single module. That is. Thanks, -- -- Guilherme H. Polo Goncalves

2 1

Preparing an existing RPC mechanism for the standard library
by Guilherme Polo March 25, 2008

March 25, 2008

Hello, I've read this idea about "preparing an existing RPC mechanism for the standard library" at StandardLibrary ideas and I would be interested in doing it, but as you all know, including something into stdlib is not exactly easy and shouldn't be anyway. Also I'm not even sure if this idea is still desired. I'm considering the inclusion of rpyc, with appropriate changes (possibly lots). And would like to know your opinions towards this. Thanks, -- -- Guilherme H. Polo Goncalves

6 6

Re: [Python-ideas] [Python-Dev] The Case Against Floating Point ==
by Mark Dickinson March 19, 2008

March 19, 2008

(with apologies for the random extra level of quoting in the below...) > On Thu, Mar 13, 2008 at 11:09 AM, Imri Goldberg <lorgandon(a)gmail.com> > wrote: > > As I said earlier, I'd like static checkers (like Python-Lint) to catch > > this sort of cases, whatever the decision may be. > > > Hmm. Isn't that tricky? How does the static checker decide whether the objects being compared are floats? I guess one could be content with catching some cases where the operands to == are clearly floats... Wouldn't you have to have run-time warnings to be really sure of catching all the cases? > > It's already too late for Python 3.0. > > Still, I believe it is worth discussing. > > > Sure. I didn't mean that to come out in quite the dismissive way it did :). Apologies. Maybe a PEP aimed at Python 4.0 is in order. If you're open to the idea of just having some way to enable warnings, it could be much sooner. > > While checking against a==0.0 (and other similar conditions) before > > dividing will indeed protect from outright division by zero, it will > > enlarge any error you will have in the computation. I guess it would be > > better to do the same check for 'a is small' for appropriate values of > > 'small'. > > Still, a check for 0.0 is good enough in some cases: if a is tiny, the large intermediate values may appear and then disappear happily before giving a sensible final result. These are usually the sort of cases where just having division by 0.0 return an infinity would have "just worked" too (making the whole "if" redundant), but that's not (currently!) an option in Python. It's a truism that floating-point equality tests should be avoided, but it's just not true that floating-point equality testing is *always* wrong, and I don't think that Python should make it so. Actually, one of the reasons I thought about this subject in the first > > place, was dict lookup for floating point numbers. It seems to me that > > it's something you just shouldn't do. > > > So your proposal would presumably include making x in dict and x not in dict errors for any float x, regardless of the contents of the dictionary (or list, or set, or frozenset, or...) dict? What would you do about Decimals? A Decimal is just another floating point format (albeit base 10 instead of base 2); so presumably all these warnings/errors should apply equally to Decimal instances? If not, why not? I'm not trying to be negative here---as Aahz says, this is an interesting idea; I'm just trying to understand exactly how things might work. Mark

5 14

Faq 4.28 Suggestion -- Trailing comas
by Leszek Dubiel March 18, 2008

March 18, 2008

I would suggest to add question 4.28 to faq. Everybody who learns Python reads that and will not ask questions about "one-element tuples". I have compiled answer from responses to my last question about one-element tuple. Question: Why python allows to put comma at the end of list? This looks ugly and seems to break common rules... Answer: There are may reasons that follow. 1. If you defined multiline dictionary d = { "A": [1, 5], "B": [6, 7], # last trailing comma is optional but good style } it would be easier to add more elements, because you don't have to care about colons -- you always put colon at the end of line and don't have to reedit other lines. It eases sorting of such lines too -- just cut line and paste above. 2. Missing comma can lead to errors that are hard to diagnose. For example: x = [ "fee", "fie" "foo", "fum" ] contains tree elements "fee", "fiefoo" and "fum". So if programmer puts comma always at the end of line he saves lots of trouble in a future. 2. Nearly all reasonable programming languages (C, C++, Java) allow for an extra trailing comma in a comma-delimited list, for consistency and to make programmatic code generation easier. So for example [1,2,3,] is intentionally correct. 3. Creating one-element tuples using tuple(['hello']) syntax is much much slower (a factor of 20x here) then writing just ['hello', ]. Trailing commas when you only have one item are how python tuple syntax is defined allowing them to use commas instead of needing other tokens. If python didn't allow comma at the end of tuple, you will have to use such slow syntax. 4. The same rule applies to other type of lists, where delimiter can occur at the end. For example both strings "alfa\nbeta\n" and "alfa\nbeta" contain two lines. Sources: -- http://mail.python.org/pipermail/python-list/2003-October/231419.html -- http://mail.python.org/pipermail/python-ideas/2008-March/001478.html -- http://mail.python.org/pipermail/python-ideas/2008-March/001475.html

3 2