Mailman 3 March 2006 - Python-Dev

Re: [Python-Dev] Keep default comparisons - or add a second set?
by Robert Brewer 15 Mar '06

15 Mar '06

Noam Raphael wrote: > I don't think that every type that supports equality > comparison should support order comparison. I think > that if there's no meaningful comparison (whether > equality or order), an exception should be raised. Just to keep myself sane... def date_range(start=None, end=None): if start == None: start = datetime.date.today() if end == None: end = datetime.date.today() return end - start Are you saying the "if" statements will raise TypeError if start or end are dates? That would be a sad day for Python. Perhaps you're saying that there is a "meaningful comparison" between None and anything else, but please clarify if so. Robert Brewer System Architect Amor Ministries fumanchu(a)amor.org

3 2

str with base
by Alex Martelli 15 Mar '06

15 Mar '06

Is it finally time in Python 2.5 to allow the "obvious" use of, say, str(5,2) to give '101', just the converse of the way int('101',1) gives 5? I'm not sure why str has never allowed this obvious use -- any bright beginner assumes it's there and it's awkward to explain why it's not!-). I'll be happy to propose a patch if the BDFL blesses this, but I don't even think it's worth a PEP... it's an inexplicable though long-standing omission (given the argumentative nature of this crowd I know I'll get pushback, but I still hope the BDFL can Pronounce about it anyway;-). Alex

42 134

Topic suggestions from the PyCon feedback
by A.M. Kuchling 15 Mar '06

15 Mar '06

Those of you on python-checkins will have noticed the recent fiddling around with the feedback from PyCon. I'd like to draw the attention of the python-dev readership to the answers for the question "What 3 topics should have been covered at PyCon?" I split out core Python and web-related topics into their own lists. Here's the core Python list: Usage of new/advanced python features (12) Python Best Practices (3) Design Patterns in Python (3) Anything Alex Martelli wants to talk about. (3) Advanced Python: metaclasses, descriptors, decorators (3) Strategies for integrating C and Python (2) Problems with python (2) Iterators and Generators (2) Ways to improve performance of Python code (1) Standard library gems/ unsung modules (1) Python Language perspective from non-Guido PEP members (1) Py3K (1) More on Python internals (1) Language howtos (I really enjoyed Alex Martelli's talk last year on itertools) (1) Getting started hacking python source (1) Decimal for Experts (1) Better Development Practices with Python (1) Being more productive with Python (1) In the first entry, "new/advanced features" = a fuzzy set containing generators, iterators, metaclasses, __slots__, and decorators. When 2.5 is released, this set will probably grow to include the 'with' statement's context managers and coroutines. Some of the other entries in the above list overlap with the first entry. The conclusion I draw from these results: intermediate- or advanced-level topics of program design are not covered enough, whether in the Python documentation, in published books and articles, or in PyCon talks. Please feel free to mine the above list, or the rest of the PyCon feedback, for topic ideas. In particular: if you're going to attend PyCon 2007, EuroPython, or some other conference (even a non-Python one), please consider submitting a talk proposal covering one of the above topics. Such presentations would find a receptive audience, I think. --amk

4 3

Py3k branch - please stay out :-)
by Guido van Rossum 15 Mar '06

15 Mar '06

So I created a Py3K branch in subversion. (Due to my slippery fingers it's actually called p3yk -- that's fine, it may keep bystanders out, and it means we can rename it to the proper name when it's more ready for public consumption. :-) My current plans for this branch are simple: I'm going to rip out some obvious stuff (string exceptions, classic classes) and get a "feel" for what Python 3000 might look like. I'm not particularly looking for help -- if all goes well this is going to be my personal hobby project for the next few months, and then we'll see where we stand. I promised OSCON a keynote on Python 3000 so that's a convenient deadline. In other news, I'd like to nominate Neal Norwitz as the Python 2.5 "release coordinator". He's already doing a great job doing exactly what I think a coordinator should be doing. Anthony will remain release manager, Tim, Martin, Fred and others will do their stuff; but Neal can be the public face. He and Anthony should probably get together on IM and decide on the actual release schedule. For all Python 2.5 issues, please look to Neal. Also, if your proposal doesn't already have a PEP number, it shouldn't be going into Python 2.5. It's time to start releasing and stop evolving for a few months... -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Another threading idea
by Raymond Hettinger 14 Mar '06

14 Mar '06

FWIW, I've been working on a way to simplify the use of queues with daemon consumer threads Sometimes, I launch one or more consumer threads that wait for a task to enter a queue and then work on the task. A recurring problem is that I sometimes need to know if all of the tasks have been completed so I can exit or do something with the result. If each thread only does a single task, I can use t.join() to wait until the task is done. However, if the thread stays alive and waits for more Queue entries, then there doesn't seem to be a good way to tell when all the processing is done. So, the idea is to create a subclass of Queue that increments a counter when objects are enqueued, that provides a method for worker threads to decrement the counter when the work is done, and offers a blocking join() method that waits until the counter is zero # main thread q = TaskQueue() for t in worker_threads(): t.start() for task in tasklist: q.put(task) # increments the counter and enqueues a task q.join() # all of the tasks are done (counter is zero) do_work_on_results() # worker thread while 1: task = q.get() # task is popped but the counter is unchanged do_work(task) q.decrement() # now the counter gets reduced The idea is still in its infancy (no implementation and it hasn't been tried in real-world code) but I would like to get some feedback. If it works out, I'll post a recipe to ASPN and see how it goes. Raymond

5 7

Threading idea -- exposing a global thread lock
by Raymond Hettinger 14 Mar '06

14 Mar '06

A user on comp.lang.python has twisted himself into knots writing multi-threaded code that avoids locks and queues but fails when running code with non-atomic access to a shared resource. While his specific design is somewhat flawed, it does suggest that we could offer an easy way to make a block of code atomic without the complexity of other synchronization tools: gil.acquire() try: #do some transaction that needs to be atomic finally: gil.release() The idea is to temporarily suspend thread switches (either using the GIL or a global variable in the eval-loop). Think of it as "non-cooperative" multi-threading. While this is a somewhat rough approach, it is dramatically simpler than the alternatives (i.e. wrapping locks around every access to a resource or feeding all resource requests to a separate thread via a Queue). While I haven't tried it yet, I think the implementation is likely to be trivial. FWIW, the new with-statement makes the above fragment even more readable: with atomic_transaction(): # do a series of steps without interruption Raymond

9 15

Re: [Python-Dev] Threading idea -- exposing a global thread lock
by Michael Chermside 14 Mar '06

14 Mar '06

Josiah Carlson writes: > It would be nice if Jython or IronPython could (and would) implement > these 'critical sections'. Whether they can or not, I think that it > would be a useful feature in the CPython runtime. The issue is not whether Jython and IronPython "will", it's whether they "can". Essentially, Jython and IronPython both use locking within their fundamental data structures. This then allows them to freely allow threads to run on multiple processors. Meanwhile, CPython lacks locks in its fundamental data structures, so it uses the GIL to ensure that code which might touch Python data structures executes on only one CPU at a time. The concept of a "critical section" makes great sense when there is effectively only one CPU: just stop switching threads. But if code is using multiple CPUs, what does it mean? Shut down the other CPUs? To do such a thing cooperatively would require checking some master lock at every step... (a price which CPython pays, but which the implementations built on good thread-supporting VMs don't have to). To do such a thing non-cooperatively is not supported in either VM. Since I doubt we're going to convince Sun or Microsoft to change their approach to threading, I think it is unwise to build such a feature into the Python language. Supporting it in CPython only requires (I think) no more than a very simple C extension. I think it should stay as an extension and not become part of the language. -- Michael Chermside

4 3

Re: [Python-Dev] Threading idea -- exposing a global thread lock
by Phillip J. Eby 14 Mar '06

14 Mar '06

At 02:21 PM 3/14/2006 -0500, Tim Peters wrote: >There _is_ some variation in what "critical section" means, exactly, >to different thread programming cultures, but in none does it mean: > > a section of code such that, once a thread enters it, all other > threads are blocked from doing anything for the duration Well, I'm showing my age here, but in the good ol' days of the 8086 processor, I recall it frequently being used to describe a block of assembly code which ran with interrupts disabled - ensuring that no task switching would occur. Obviously I haven't been doing a lot of threaded programming *since* those days, except in Python. :) >The common meaning is: > > a section of code such that, once a thread enters it, all other > threads are blocked from entering the section for the duration That doesn't seem like a very useful definition, since it describes any piece of code that's protected by a statically-determined mutex. But you clearly have more experience in this than I.

3 2

Strange behavior in Python 2.5a0 (trunk) --- possible error in AST?
by Travis E. Oliphant 14 Mar '06

14 Mar '06

I'm seeing strange behavior in the Python 2.5a0 trunk that is causing the tests for numpy to fail. Apparently obj[...] = 1 is not calling PyObject_SetItem Here is a minimal example to show the error. Does anyone else see this? class temp(object): def __setitem__(self, obj, v): print obj, v mine = temp() mine[...] = 1 mine[0] = 1 Output from Python 2.4.2: Ellipsis 1 0 1 Output from Python 2.5a0: 0 1 In other words, the first line does nothing in Python 2.5a0. Does anybody else see this? Thanks, -Travis

3 5

Developing/patching ctypes (was: Re: integrating ctypes into python)
by Thomas Heller 14 Mar '06

14 Mar '06

[CC to python-dev again] Hye-Shik Chang wrote: > On 3/10/06, Thomas Heller <theller(a)python.net> wrote: >> Hye-Shik Chang wrote: >>> On 3/9/06, Thomas Heller <theller(a)python.net> wrote: >>>> I've now committed ctypes 0.9.9.4 into SVN. It seems to build, at least on the buildbots >>>> that are online. >>> Good work! It's all green on my machines (FreeBSD amd64/sparc64). >>> But gcc outputs some warnings: >>> >>> 1) libffi/include/ffi.h.in: void (*fn)() -> void (*void)(void) >> This is not the only file that would have to be changed. I wonder why they don't change >> it in gcc itself. > > This *is* the only change to calm gcc warnings down except Py_ssize_t > stuff. :-) > >>> 2) many of Py_ssize_t / int prototype mismatches. (eg. sq_repeat) >>> Could we make ctypes Py_ssize_t clean? >>> 3) ctypes doesn't support Py_ssize_t nor ssize_t yet. Because major >>> portion of our pythonapi uses it now, it would be better to have it. >> This should be done in ctypes CVS, and the result imported into python SVN after that. >> At least ctypes (the Python module) has an c_size_t type now ;-). > > I'll do it in this weekend. I saw that branch_1_0 is merged into HEAD. > Which branch are you using for Python integration at now? I did 'cvs export -r release_0_9_9_4' from the SF repository and then imported this into python SVN external/ctypes-0.9.9.4. The 'release_0_9_9_4' tag is on the 'branch_1_0' branch. I moved on a little bit on that branch, later merged everything into SF CVS HEAD, and continued to work there. We'll have to coordinate this. Different people working in different repositories will become a maintaince nightmare, so there should be an official 'master' repository for ctypes. Question for python-dev: Would it be a solution to move the 'official' ctypes development into Python SVN external/ctypes, or would this be considered abuse? Another location in SVN could be used as well, if external is though to contain only vendor drops... Thomas

7 22