Using the module heapq, I found that it's not easy to use. Or, at
least, that it could be straightforward to use.
My main issue is that for correct usage:
- user should provide an external list, that shouldn't use for
anything else to don't break the invariant
- to alter the heap queue, a bunch of functions must be used, passing
always the external list
I think that changing "external list" for "internal attribute", and
"bunch of functions " for "methods", it will leave the module easier
to use, safer, and more object oriented.
So, I basically coded it. What do you think about including this class
in the heap module?
"""
class Heap(object):
'''Maintains a heapified list, always conserving the invariant.
Heaps are arrays for which heap[k] <= heap[2*k+1] and
heap[k] <= heap[2*k+2] for all k, counting elements from zero.
'''
def __init__(self, iterable=None):
'''Initializes the heap from any iterable.
>>> Heap([1, 2])
Heap([1, 2])
>>> Heap([])
Heap([])
>>> Heap()
Heap([])
>>> Heap((1, 2, 3))
Heap([1, 2, 3])
>>> Heap(x**2 for x in range(3))
Heap([0, 1, 4])
'''
if iterable is None:
self._queue = []
else:
self._queue = list(iterable)
heapq.heapify(self._queue)
def __repr__(self):
return "Heap(%s)" % self._queue
def push(self, item):
'''Push the item to the heap queue.
>>> h = Heap()
>>> h.push(5)
>>> h.push(4)
>>> h.push(3)
>>> h.push(2)
>>> h.push(1)
>>> h.push(0)
>>> h
Heap([0, 2, 1, 5, 3, 4])
'''
heapq.heappush(self._queue, item)
def pop(self):
'''Pop one item from the heap queue.
>>> h = Heap([0, 2, 1, 5, 3, 4])
>>> h.pop()
0
>>> h.pop()
1
>>> h.pop()
2
>>> h.pop()
3
>>> h.pop()
4
>>> h.pop()
5
>>> h
Heap([])
>>> h.pop()
Traceback (most recent call last):
...
IndexError: index out of range
'''
return heapq.heappop(self._queue)
def pushpop(self, item):
'''Push the item and pop one from the heap queue.
Note that this method is more efficient than calling both
push() and pop() separately.
>>> h = Heap()
>>> h.pushpop(7)
7
>>> h.push(5)
>>> h.push(4)
>>> h.push(3)
>>> h.pushpop(7)
3
>>> h.pushpop(7)
4
>>> h
Heap([5, 7, 7])
'''
return heapq.heappushpop(self._queue, item)
def replace(self, item):
'''Pop one item and push the received one into the heap queue
Note that this method is more efficient than calling both
pop() and push() separately.
>>> h = Heap()
>>> h.replace(3)
Traceback (most recent call last):
...
IndexError: index out of range
>>> h.push(7)
>>> h.replace(2)
7
>>> h
Heap([2])
>>> h.push(3)
>>> h
Heap([2, 3])
>>> h.replace(1)
2
>>> h.replace(9)
1
>>> h
Heap([3, 9])
'''
return heapq.heapreplace(self._queue, item)
"""
Regards,
--
. Facundo
Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Hi,
I've created a Python Bytecode Verifier in Python. I'm not a Python
guru so I borrowed coding patterns from C/C++. I also created this
with C portability in mind. The only reason I used Python was to
experiment with Python and was easier to morph code during
development.
If this program were ported to C it would only need 8 bytes per opcode
(some additional storage to track blocks) and a single pass. I haven't
found backward jumps to previously unused code in any compiled Python
code but it can easily be supported. In that case some more partial
passes are required.
I also was able to successfully verify all Python files in the
Python-2.5.2 source package.
The verification algorythm should be quite complete but I may have
missed some limitations of the interpreter that could be checked by
the verifier as well.
The ability to create this verfier proved that although Python
bytecode is designed for a dynamically typed interpreter, is still
easily verifiable. I am willing port this code C but only in the case
if there is any chance to be included in Python.
I believe that Python in general would benefit having the ability to
safely load .pyc files and create code objects on the fly.
Both Java and .NET have the ability to safely load compiled byte code.
.NET Framework, just like Python also has the ability to create and
execute new code at run-time.
You may feel that enabling closed source applications and/or creating
a multi-language runtime would hurt Python but both of these have
contributed to the success of Java (both the language and the
runtime).
Kornél
I believe int(s, base) needs an inverse function to allow string
representation with different bases. An example use case is 'hashing' a
counter like video ID's on youtube, you could use a regular int internally
and publish a shorter base-62 id
for links.
This subject was discussed 2.5 years ago:
http://mail.python.org/pipermail/python-dev/2006-January/059789.html
I opened a feature request ticket:
http://bugs.python.org/issue6783
Some of the questions that remain:
1. Whether this should be a method for int or a regular function in a
standard library module like math.
2. What should the method/function be called? (base_convert, radix, etc)
What do you guys think?
RunThePun
So there has been a little discussion on the python mailing list regarding
a request I had to expose {set,get}res{uid,gid}.
The general consensus there is that even though it isn't a POSIX standard,
it is shared among many Unix OSes and belongs in the same module as setuid
and friends, namely os, which maps to posixmodule.c
Previous discusion and background as to why this is desirable:
http://bugs.python.org/issue6508
If this is the Right Way to do it, I'll brush up on autotools and see
about submitting a patch...
--
Obama Nation | My emails do not have attachments; it's a digital signature
that your mail program doesn't understand. | http://www.subspacefield.org/~travis/
If you are a spammer, please email john(a)subspacefield.org to get blacklisted.
Hello,
As previously mentioned on python-ideas [1] (circa 2006), it would
make sense to be able to perform bitwise operations on
bytes/bytearray.
Stealing the example from the original suggestion:
Suppose I have a string (say, read in from a binary file) that has a
4-byte field in it. Bit 11 indicates whether the accompanying data is
in glorped form (something I'm just making up for this example).
For example, if the field has 0x000480c4, the data is not glorped. If
the data is glorped, bit 11 would be on, i.e., 0x001480c4.
Let's say I want to turn on the glorp bit; what I have to do now:
GLORPED = 0x10
newchar = flags[1] | GLORPED
flags = flags[0] + newchar + flags[2:]
What I'd like to be able to do is something like:
GLORPED = b"\x00\x10\x00\x00"
flags |= GLORPED
# test if the glorped bit is on
any(flags & GLORPED)
I have run into this a few times, at least when reading/writing binary
formats etc. This approach is more intuitive than the typical/archaic
way of converting to an integer, performing a bitwise operation on the
integer, converting back to bytes. Arguably, bitwise operations on a
high-level integer type don't make sense, as base 2 is an
implementation detail.
At the very least, bytes and bytearray should be usable with the ~ ^ |
& operators
Example behavior:
>>> ~b'\x55\xff'
b'\xaa\x00'
>>> b'\xf5\x60' & b'\xa9\x3c'
b'\xa1\x20'
Unresolved problems:
If the two arguments are of different length, either it could either
raise a ValueError or mimic the behavior of ints.
Xoring an int to a byte seems less than well defined in general, due
to endianness ambiguity of the int and size ambiguity. I would think
this should not be allowed.
Also conceivable is using the shift operators >> and << on bytes, but
I personally would use that less often, and the result of such an
operation is ambiguous due to endianness.
-Eric Eisner
[1] http://mail.python.org/pipermail/python-ideas/2006-December/000001.html
I would like to propose an expression, similar to the if-else expression,
that responds to exceptions.
I had originally posted this (probably mistakenly) on py-dev. This
current posting is a cleaned up
version of the idea, based on responses I got on from the earlier posting.
_*Abstract:
*_Proposal for a conditional expression, similar to the if-else
expression, that responds to exceptions.
_*Motivation:
*_An expression syntax that responds to exceptions, and which reproduces
the readability and conciseness of the if-else conditional expression,
would simplify some exception-handling cases, especially within list
comprehensions.
_*Very Simple Example - type coercion:
*_Current approach:
try:
x = float(string)
except:
x = float('nan')
Proposed solution using exception-based conditional expression:
x = float(string) except ValueError: float('nan')
_*Simple Example - type coercion in a list comprehension:
*_Current approach:
def safe_float(string):
try:
x = float(string)
except ValueError:
x = float('nan')
return x
...
xs = (safe(float(string)) for string in strings)
Proposed solution using exception-based conditional expression:
xs = ((float(string) except ValueError: float('nan')) for string in
strings)
_*Discussion:
*_In my own python coding, I find I make common use of the if-else
conditional expression, especially within list comprehensions. (In one
of my packages, which has ~5800 lines of code, I found if-else
expressions in ~1% of the lines.)
Here is a slightly more involved example than the examples presented
above. In data processing, I often string together a sequence of
iterable list comprehensions, corresponding to a sequence of operations
on a given dataset "ys" to produce a processed dataset "x":
xs = (operation_A(x) for x in ys)
xs = (operation_B(x) for x in xs if filter_B(x))
xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for
x in xs if filter_C(x))
# final, explicit list of values
xs = [ x for x in xs ]
This is often a flexible way for me to define processing and filtering
sequences which also seems
to have good performance on very large datasets. One advantage is that
I can quickly mix-and-match from existing processes like this to make a
new process. An exception-based conditional would go nicely
into many of these process sequences, keeping them both robust and flexible.
xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
I also often have object classes which have some common method or
attribute. For instance, some of my objects have scope-dependent values:
x = y.evaluate(scope))
where scope is typically locals(), globals(), or some other
dictionary-like container. But, to keep my code modular, I want to
handle, in the same lines of code, objects which do not have some
particular method, which leads me to lines of code like:
x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
This seems not very "Pythonic", similar to using type-testing instead of
try-except. (My impression was that there was a long-standing trend in
the evolution of Python to remove tests like this, and I thought that
was the original motivation for the try-except syntax.)
I would much rather write:
x = y.evaluate(locals()) except AttributeError: y
or, in the list comprehension example:
xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
Clearly this can be handled in several ways with the language as it is.
One way is to define a new function, as in the second simple example above:
def safe_evaluate(y,scope):
try:
x = y.evaluate(scope)
except AttributeError:
x = y
return x
...
xs = (safe_evaluate(y,locals()) for y in ys)
but this quickly (in my packages at least) leads to an annoying
proliferation of "safe_" functions.
Again, this seems not to be in the "Pythonic" spirit, and is also less
concise, less readable. (I also suspect, but have not verified, that
this is in general less efficient than in-line expressions -- wasn't
that part of the original motivation for list comprehensions?).
In the thread of my previous post to py-dev, there were comments,
questions, and suggestions concerning the details of the syntax. Having
reflected on this for a couple weeks, I am now most strongly supportive
of what is essentially just an inline compression of the current
try-except syntax. So the following examples would be allowed:
x = expression0 except: default_expression
x = expression0 except exception1: expression1 except exception2:
expression2 except: default_expression
Or, more generally:
x = expression0\
except exception1: expression1\
except exception2: expression2\
...
except exceptionI: expressionI\
...
except: default_expression
In this last example, the behaviour would be as follows:
- evaluate expression0.
If no exception is encountered, return the result.
- if an exception is encountered,
search for the matching exception in the except clauses.
- if a matching exception ("exceptionI") is found,
evaluate the corresponding expression ("expressionI"), and
return the result.
- if no matching exception is found, and a default except: clause
(i.e., one without and exception)
is given, evaluate default_expression, and return the result.
- if no matching exception is found, and no default except clause if
given,
pass the exception on to the caller.
- if a new exception is encountered while evaluating an an except
expression ("expressionI"),
pass the exception on to the caller.
I hope I have made a convincing case here. This seems to me to be a
natural ("Pythonic") addition to the language.
Jeff McAninch
--
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch(a)lanl.gov
==========================
A while back i had learned about the 'with' statement and at first I naively
thought that it worked similarly to a with statement I was familiar with,
from delphi/pascal - obviously it doesn't and I was instantly hit with the
idea of how useful it would be to have a keyword that could use a namespace
from an object, hitting the familiar __getattr__ functions and related.
the keyword I proposed would be 'interrogate' (which is rather long, could
use another one like 'using' or something) but.. thats not really important
to me, the idea is, so
class test():
def __init__(self):
self.x = 42
foo = test()
bar = test()
x = 33
bar.x = 22
interrogate foo, bar:
print x
-----
output is 42 since x is found inside of foo before bar, and locals would be
interrogated last, but a more usefull example would be ..
class test():
def __getattr__(self, name):
if len(name) == 1 and (ord(name) in range(ord('a'), ord('z'))) :
return ord(name)
test_ns = test()
interrogate test_ns:
print a
----
output is 97
using the above example lexical closures may be awkward to implement in the
language itself, this is a bit of a contrived example.. but.. eh..
# if I understand python enough, then I believe that when we enter this
block, and create variables, they go out of scope,
# and there are no exceptions to this rule, ever so .. in order to implement
mine obviously scope would be the same
# mechanism so I have oh_bother declared in the local namespace first
oh_bother = None
interrogate test_ns:
def geronimo():
return z
oh_bother = geronimo
print oh_bother()
---
output is 122
--
-Prozacgod
"Prozac may heal the mind, but friends can mend the soul"
Calvin Spealman wrote:
> -1 on colons in the expression like that. I like the idea of being
> able to handle an exception in generator expressions and the like, but
> I've never seen a syntax I liked. I think I've favored the idea of
> something like `float(x) except float('nan') if ValueError` thinking
> it reads more naturally as an expression, puts the real logic
> ("convert x to a float or get a NaN float") together, which I think
> makes sense.
>
>
Yes, I agree about the colons. They have no purpose. I was just
blindly following the try-except. (Duh! on my part)
So, in the simple example:
x = float(string) except ValueError float('nan')
But possibly the exception tuples now have to be explicitly tuples?
x = float(string) except (ValueError,) float('nan')
So the general case would be something like
exception_expression :== nominal_value {except exception_tuple exception_value}* {except default_value}
Jeff McAninch
--
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch(a)lanl.gov
==========================
>
> -1, if this goes through it will be extended to all other compound
> statements over time and we'll end up with 2 or 3 ways to do every
> thing.
>
> A much better idea would be to find a way to make all compound
> statements into expressions, future-proofing the decision and avoiding
> the redundancy between "compound statement statement" and "compound
> statement expression".
>
Python's list of compound statements is pretty thin (just 7),
and 4 of these already have expression forms.
for ==> list comprehension, iterator expression
while ==> list comprehension, iterator
if ==> if-else conditional expression
def ==> already have lambda expressions
try ==> my proposal
class ==> Okay, so this one doesn't have a compound statement,
but where would you use it?
with ==> I haven't had an occasion to use with,
so can't think of an example where it could
be useful as a compound expression.
Seems to me try-except is kind of sticking out here, and I've
tried to show what I think are natural and common places where
a try-except expression would be appropriate.
--
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch(a)lanl.gov
==========================
I was thinking about a good syntax for implicit lambdas for a while
and today I had this idea: make ``_:`` a shortcut for ``lambda
_=None:``
For example:
map( _: _ + 5, some_list)
register_callback( _: True)
def apply_transform(..., transform = _:_, ... ):
but still
addition = lamba x, y: x + y
The rationale is that you only want to get rid of lambda keyword to
create a *very* simple function, the one that will be called either
without parameters or with only one parameter. For everything more
complicated, you really should go and write the explicit function
signature using lambda.
Even though ``:`` could theoretically denote implicit lambda, it's too
easy to miss it. The combination ``_:`` is much easier to notice. It
also makes explicit that there is at most one parameter and it's name
is ``_``. Since it's very short, it can easily be used in a long
function call or as a default parameter, as above
Your thoughts?
