Mailman 3 December 2009 - Python-ideas

Alternative weakref implementation
by Greg Ewing 20 Dec '09

20 Dec '09

I've been thinking about an alternative implementation of weak references that would remove the limitation of only certain types being weakly referenceable. This limitation can be a nuisance sometimes. The justification for it appears to be that you can't build a cycle exclusively out of objects that don't contain any mutable references, so there is no need to create weakrefs to such objects. However, avoiding cycles is not the only reason for using weakrefs. They can also be useful for setting up callbacks to be triggered when an object is deallocated. The restriction would be unnecessary if weakrefs could be implemented without incurring any overhead in the object being weakly referenced. This could be done by keeping a global dictionary, with keys that are uncounted references to weakly referenced objects, and values that are lists of weakref objects. Whenever an object is deallocated, the dict would be checked to see if there are any weak references to it, and if so they would be made dead and their callbacks called, and then the dict entry would be removed. Can anyone see any serious flaws in this scheme? -- Greg

2 1

Adding a `object.__deepcopy__`
by Ram Rachum 20 Dec '09

20 Dec '09

Do you think there should be a `__deepcopy__` method for `object`? This will solve a little problem I have here: http://stackoverflow.com/questions/1933621/deepcopy-a-simple-python-object I also think it's a more elegant solution than the way it works now. Ram.

3 2

Being able to specify "copy mode" to copy.deepcopy
by cool-RR 20 Dec '09

20 Dec '09

This is about the `copy.deepcopy` function. With the __deepcopy__ method, user-defined objects can specify how they will be copied. But it is assumed that you will always want to copy them the same way. What if sometimes you want to copy them in one way and sometimes in another? I am now being held back by this limitation. I will give some background to what I'm doing: I'm developing a simulations framework called GarlicSim. You can see a short video here: http://garlicsim.org/brief_introduction.html The program handles world states in simulated worlds. To generate the next world state in the timeline, the last world state is deepcopied and then modified. Now sometimes in simulations there are big, read-only objects that I don't want to replicate for each world state. For example, a map of the environment in which the simulation takes place. So I have defined a class called `Persistent`, for which I have defined a __deepcopy__ that doesn't actually copy it, but gives a reference to the original object. So now I can use `Persistent` as a sub-class to these big objects that I don't want to replicate. But in some cases I do want to replicate these objects, and I can't! So I suggest that it will be possible to specify a "mode" for copying. User defined objects will be able to specify how they will be deepcopied in each mode. What do you think? Ram.

8 11

disabling .pyc and .pyo files
by Kristján Valur Jónsson 16 Dec '09

16 Dec '09

Hello there. We have a large project involving multiple perforce branches of hundreds of .py files each. Although we employ our own import mechanism for the bulk of these files, we do use the regular import mechanism for an essential core of them. Repeatedly we run into trouble because of stray .pyo (and/or .pyc) files. This can happen for a variety of reasons, but most often it occurs when .py files are being removed, or moved in the hierarchy. The problem is that the application will happily load and import an orphaned .pyo file, even though the .py file has gone or moved. I looked at the import code and I found that it is trivial to block the reading and writing of .pyo files. I am about to implement that patch for our purposes, thus forcing recompilation of the .py files on each run if so specified. This will ensure that the application will execute only the code represented by the checked-out .py files. But it occurred to me that this functionality might be of interest to other people than just us. I can imagine, for example, that buildbots running the python regression testsuite might be running into problems with stray .pyo files from time to time. Do you think that such a command line option would be useful for Python at large? Cheers, Kristján

19 50

Re: [Python-ideas] Importing orphaned bytecode files
by Eric Smith 10 Dec '09

10 Dec '09

Sorry for top posting. My phone makes me! You're right: I misread. Sorry about that. -- Eric. "Ben Finney" <ben+python(a)benfinney.id.au> wrote: >Eric Smith <eric(a)trueblade.com> writes: > >> Ben Finney wrote: >> > I suggest: >> > >> > * A new attribute ‘sys.import_orphaned_bytecode’. If set ‘True’, the >> > interpreter follows the current behaviour. If ‘False’, any bytecode >> > file satisfies an import only if it has a corresponding source file >> > (where “corresponding” means “this source file would, if compiled, >> > result in a bytecode file replacing this one”). >> >> I agree with this in principle > >Thanks. > >> but I don't see how you're going to implement it. In order to actually >> check this condition, aren't you going to have to compile the source >> code anyway? If so, just skip the bytecode file. Although I guess you >> could store a hash of the source in the compiled file, or other >> similar optimizations. > >You seem to be seeing something I was careful not to write. The check >is: > > this source file would, if compiled, result in a bytecode file > replacing this one > >Nowhere there is there anything about the resulting bytecode files being >equivalent. I'm limiting the check only to whether the resulting >bytecode file would *replace* the existing bytecode file. > >This doesn't require knowing anything at all about the contents of the >current bytecode file; indeed, my intention was to phrase it so that >it's checked before bothering to open the existing bytecode file. > >Is there a better term for this? I'm not well-versed enough in the >Python import internals to know. > >-- > \ “Philosophy is questions that may never be answered. Religion | > `\ is answers that may never be questioned.” —anonymous | >_o__) | >Ben Finney > >_______________________________________________ >Python-ideas mailing list >Python-ideas(a)python.org >http://mail.python.org/mailman/listinfo/python-ideas

5 5

Heap data type
by Facundo Batista 07 Dec '09

07 Dec '09

Using the module heapq, I found that it's not easy to use. Or, at least, that it could be straightforward to use. My main issue is that for correct usage: - user should provide an external list, that shouldn't use for anything else to don't break the invariant - to alter the heap queue, a bunch of functions must be used, passing always the external list I think that changing "external list" for "internal attribute", and "bunch of functions " for "methods", it will leave the module easier to use, safer, and more object oriented. So, I basically coded it. What do you think about including this class in the heap module? """ class Heap(object): '''Maintains a heapified list, always conserving the invariant. Heaps are arrays for which heap[k] <= heap[2*k+1] and heap[k] <= heap[2*k+2] for all k, counting elements from zero. ''' def __init__(self, iterable=None): '''Initializes the heap from any iterable. >>> Heap([1, 2]) Heap([1, 2]) >>> Heap([]) Heap([]) >>> Heap() Heap([]) >>> Heap((1, 2, 3)) Heap([1, 2, 3]) >>> Heap(x**2 for x in range(3)) Heap([0, 1, 4]) ''' if iterable is None: self._queue = [] else: self._queue = list(iterable) heapq.heapify(self._queue) def __repr__(self): return "Heap(%s)" % self._queue def push(self, item): '''Push the item to the heap queue. >>> h = Heap() >>> h.push(5) >>> h.push(4) >>> h.push(3) >>> h.push(2) >>> h.push(1) >>> h.push(0) >>> h Heap([0, 2, 1, 5, 3, 4]) ''' heapq.heappush(self._queue, item) def pop(self): '''Pop one item from the heap queue. >>> h = Heap([0, 2, 1, 5, 3, 4]) >>> h.pop() 0 >>> h.pop() 1 >>> h.pop() 2 >>> h.pop() 3 >>> h.pop() 4 >>> h.pop() 5 >>> h Heap([]) >>> h.pop() Traceback (most recent call last): ... IndexError: index out of range ''' return heapq.heappop(self._queue) def pushpop(self, item): '''Push the item and pop one from the heap queue. Note that this method is more efficient than calling both push() and pop() separately. >>> h = Heap() >>> h.pushpop(7) 7 >>> h.push(5) >>> h.push(4) >>> h.push(3) >>> h.pushpop(7) 3 >>> h.pushpop(7) 4 >>> h Heap([5, 7, 7]) ''' return heapq.heappushpop(self._queue, item) def replace(self, item): '''Pop one item and push the received one into the heap queue Note that this method is more efficient than calling both pop() and push() separately. >>> h = Heap() >>> h.replace(3) Traceback (most recent call last): ... IndexError: index out of range >>> h.push(7) >>> h.replace(2) 7 >>> h Heap([2]) >>> h.push(3) >>> h Heap([2, 3]) >>> h.replace(1) 2 >>> h.replace(9) 1 >>> h Heap([3, 9]) ''' return heapq.heapreplace(self._queue, item) """ Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

11 14

Why does `sum` use a default for the `start` parameter?
by Ram Rachum 06 Dec '09

06 Dec '09

I noticed that `sum` tries to add zero to your iterable. Why? Why not just skip adding any start value if none is specified? This current behavior is preventing me from using `sum` to add up a bunch of non- number objects. Ram.

12 30