Mailman 3 February 2015 - Python-ideas

Re: [Python-ideas] Add orderedset as set(iterable, *, ordered=False) and similarly for frozenset.
by Ed Kellett 08 Feb '15

08 Feb '15

On Fri Feb 06 2015 at 01:17:03 Nathaniel Smith <njs(a)pobox.com> wrote: > > Surely a.union(b) should produce the same result as a.update(b) except > out-of-place, and .update() already has well-defined semantics for > OrderedDicts. > I don't think they are equivalent. update() makes sense because it's described as a procedure mutating one operand - among other implications, it fundamentally doesn't commute. On the other hand, set union is commutative, and changing that seems strange (and wrong) to me. I think it'd be more sensible to add .extend() and + operations to ordered sets if that's a useful thing to do to them. It mirrors the fact that they are (more or less) mutable sequences, and it's more obvious what the order of the result is (and + on sequences is already non-commutative). In any case, I'm evidently wrong about this being a trivial decision. :) Ed Kellett

5 12

Add orderedset as set(iterable, *, ordered=False) and similarly for frozenset.
by Raymond Hettinger 07 Feb '15

07 Feb '15

Resending this earlier post. It was mistakenly sent to python-ideas(a)googlegroups.com instead of python-ideas(a)python.org. Sorry for the confusion and for the posts arriving out-of-sequence. ---------------------- > On Feb 4, 2015, at 1:14 PM, Neil Girdhar <mistersheik(a)gmail.com> wrote: > > Hundreds of people want an orderedset (http://stackoverflow.com/questions/1653970/does-python-have-an-ordered-set) and yet the ordered sets that are in PyPI (oset, ordered-set) are incomplete. Please consider adding an ordered set that has *all* of the same functionality as set. > > The only major design decision is (I think) whether to store the elements in a linked list or dynamic vector. > > My personal preference is to specify the ordered set via a flag to the constructor, which can be intercepted by __new__ to return a different object type as necessary. > > (If anyone wants to add a complete ordered set to PyPI that would also be very useful for me!) Long ago, I ago considered adding an OrderedSet() and decided to hold-off for several reasons: * The primary use cases for sets don't need order. The main use cases being: uniquification, fast membership testing, and classic set-to-set operations (union, intersection, and difference). * As far as I can tell, the primary benefit of an ordered set is that it is easier on the eyes when interactively experimenting with trivial schoolbook examples. That said, an OrderedSet __repr__ looks uglier because it would lack the {10,20,30} style literals we have for regular sets). * It isn't entirely clear what the set-to-set operations should do to maintain order, whether equality tests should require matching order, whether we care about loss of commutativity, and whether we're willing to forgo the usual optimizations on set-to-set operations (i.e. the intersection algorithm loops over the smaller of the two input sets and checks membership in the larger set -- that is fast, but it would lose order if the second set were the smaller of the two). * The standard library should provide fast, clean fundamental building blocks and defer all of the myriad of variants and derivations to PyPI. We already provide OrderedDict and MutableSet so that rolling your own OrderedSet isn't hard (see the recipe below). * We're likely to soon have an OrderedDict written in C (I believe Eric Snow has been working on that for some time), so anything build on top of an OrderedDict will inherit all the speed benefits. Also, there is some chance that PyPy folks will contribute their work making regular sets and dicts ordered by default; in which case, having a separate OrderedSet would have been a waste and the question would be moot. * Also, I wouldn't read too much in the point score on the StackOverflow question. First, the question is very old and predates simpler solutions now available using OrderedDict and MutableMapping. Second, StackOverflow tends to award high scores to the simplest questions and answers (such as http://stackoverflow.com/a/8538295/1001643 ). A high score indicates interest but not a need to change the language. A much better indicator would be the frequent appearance of ordered sets in real-world code or high download statistics from PyPI or ActiveState's ASPN cookbook. * One thing I've used to test the waters is to include a link in the documentation to an OrderedSet recipe (it's in the "See also" section at the very bottom of the page at https://docs.python.org/2/library/collections.html ). As far as I can tell, there has been nearly zero uptake based on this link. That gives us a good indication that there is very little real-world need for an OrderedSet. What I suggest is adding the recipe (like the one shown below) directly to the docs in this section: https://docs.python.org/2/library/collections.html#ordereddict-examples-and… There we already have code for a LastUpdatedOrderedDict() and an OrderedCounter(). In the past, I've used these recipes as a way to gauge interest. When the interest was sufficient, it got promoted from a recipe in the docs to a full implementation living in a standard library module. For example, itertools.combinations_with_replacement() and itertools.accumulate() started out as recipes in the docs. In general, there shouldn't be a rush to push more things in the standard library. We get many suggestions in the form, "it might be nice to have XXX in the standard library", however, in many cases there has been so little uptake in real-world code that it is isn't worth the maintenance burden or for the burden on the working programmer for having too many choices in the standard library (I encounter very few people who actually know most of what is available in the standard library). PyPI is a successful secondary source of code and has become part of the working programmer's life. We're gradually making it easier and easier to pull that code on-demand. IMO, that is where most of the collections variants should live. Otherwise, it would be easy to fill the collections module with dozens of rarely used dict/set variants, red-black trees, pairing heaps, tries, graph implementations, bitarrays, bitsets, persistent collections, etc.) my-two-cents-ly, Raymond ------- Rolling your own from existing parts -------------- from collections import OrderedDict, MutableSet class OrderedSet(MutableSet): def __init__(self, iterable=()): self._data = OrderedDict.fromkeys(iterable) def add(self, key): self._data[key] = None def discard(self, key): self._data.pop(key, None) def __len__(self): return len(self._data) def __contains__(self, key): return key in self._data def __iter__(self): return iter(self._data) def __repr__(self): return '%s(%r)' % (self.__class__.__name__, set(self))

1 0

[PEP8] Predicate consistency
by Marco Buttu 06 Feb '15

06 Feb '15

Hi all, the PEP 8 says that function and method names should be lowercase, with words separated by underscores as necessary to improve readability. Maybe it could be a good idea to specify in the PEP8 that the predicates should not be separated by underscores, because it is pretty annoying to see this kind of inconsistency: str.isdigit() float.is_integer() -- Marco Buttu INAF-Osservatorio Astronomico di Cagliari Via della Scienza n. 5, 09047 Selargius (CA) Phone: 070 711 80 217 Email: mbuttu(a)oa-cagliari.inaf.it

4 8

Python Mobile-sig
by Jeff Hardy 04 Feb '15

04 Feb '15

There is now a Python SIG dedicated to running Python on mobile devices. Sign up for the mobile-sig(a)python.org mailing list and find out more at [1]. The Mobile-SIG exists to improve the usability of Python on mobile devices, as mobile platforms have their own unique requirements and constraints. There are two goals: * To collaborate on porting CPython to mobile platforms and ensure that other Python implementations (i.e. Jython, IronPython) present much the same environment as CPython when running on mobile devices. * To collaborate on the design of modules for mobile-centric features such as cameras, GPS, accelerometers, and other sensors to ensure cross-platform and cross-implementation compatibility. The intent is to work on mobile-centric features in this group before proposal to python-dev, similar to how the distutils-sig operates. - Jeff [1] https://www.python.org/community/sigs/current/mobile-sig/

1 0

change list() - branch from "More useful slices"
by Alexander Heger 02 Feb '15

02 Feb '15

On 3 February 2015 at 01:44, Rob Cliffe <rob.cliffe(a)btinternet.com> wrote: > > On 02/02/2015 12:48, João Santos wrote: >> >> L = ['foo', 'bar', 'baz'] is syntactic sugar, you can write L = >> list('foo', 'bar', 'baz') > > No you can't (at least not in Python 2.7.3). (A good advertisement for > testing your code before publishing it. ) It has to be L = list(('foo', > 'bar', 'baz')) which is using a tuple literal. Well, if something useful comes out of this discussion, maybe it would be an idea to extend list to allow the former syntax naively assumed, list('foo', 'bar', 'baz') similar to what exists for dict(): dict(a=1, b=2, c=3) though there would be excessive ambiguity for a single argument that is an iterable, like list('abc') which gives ['a','b','c'] not ['abc'] and hence a source many mistakes... Well, just a thought, maybe not a good one. I am sure this must have been discussed in the past, and rejected. -Alexander

9 9

More useful slices
by Todd 02 Feb '15

02 Feb '15

Although slices and ranges are used for different things and implemented differently, conceptually they are similar: they define an integer sequence with a start, stop, and step size. For this reason I think that slices could provide useful syntactic sugar for defining ranges without sacrificing clarity. The syntax I propose is simply to wrap a slice in parentheses. For examples, the following pairs are equivalent: range(4, 10, 2) (4:10:2) range(3, 7) (3:7) range(5) (:5) This syntax should not be easily confused with any existing syntax. Slices of sequences use square brackets, dictionaries use curly braces, and function annotations only appear in function declarations. It is also considerably more compact than the current range syntax, while still being similar to a syntax (slicing) all python users should be familiar with.

19 44

Infinite in Python (not just in datetime)
by Thomas Güttler 02 Feb '15

02 Feb '15

Thank you very much, that there are some developers who support the idea of infinite in the datetime module. Next idea: If Python would support infinite in the core, implementing infinite in datetime might be much easier. What do you think?

7 8

Adding a subprocess.CompletedProcess class
by Thomas Kluyver 02 Feb '15

02 Feb '15

The subprocess module provides some nice tools to control the details of running a process, but it's still rather awkward for common use cases where you want to execute a command in one go. * There are three high-level functions: call, check_call and check_output, which all do very similar things with different return/raise behaviours * Their naming is not very clear (check_output doesn't check the output, it checks the return code and captures the output) * You can't use any of them if you want stdout and stderr separately. * You can get stdout and returncode from check_output, but it's not exactly obvious: try: stdout = check_output(...) returncode = 0 except CalledProcessError as e: stdout = e.output returncode = e.returncode I think that what these are lacking is a good way to represent a process that has already finished (as opposed to Popen, which is mostly designed to handle a running process). So I would: 1. Add a CompletedProcess class: * Attributes stdout and stderr are bytes if the relevant stream was piped, None otherwise, like the return value of Popen.communicate() * Attribute returncode is the exit status * ? Attribute cmd is the list of arguments the process ran with (not sure if this should be there or not) * Method cp.check_returncode() raises CalledProcessError if returncode != 0, inspired by requests' Response.raise_for_status() 2. Add a run() function - like call/check_call/check_output, but returns a CompletedProcess instance 3. Deprecate call/check_call/check_output, but leave them around indefinitely, since lots of existing code relies on them. Thanks, Thomas

9 26

"atexit" equivalent for functions in contextlib
by Nikolaus Rath 31 Jan '15

31 Jan '15

Hello, The other day, I was looking for an "atexit" equivalent at the function level. I was hoping to replace code like this: def my_function(bla, fasel): with ExitStack() as cm: ... cm.push(...) ... with something like: @handle_on_return def my_function(bla, fasel, on_return): ... on_return.push(...) ... It seems that contextlib *almost* offers a way to do this with ExitStack and ContextDecorator - but as far as I can tell the final piece is missing, because ContextDecorator does not offer a way to pass the context manager to the decorated function. However, the following decorator does the job: def handle_on_return(fn): @functools.wraps(fn) def wrapper(*a, **kw): with contextlib.ExitStack() as on_return: kw['on_return'] = on_return return fn(*a, **kw) return wrapper It's not a lot of code, but my feeling is that not anyone who might be interested in an "on_return" functionality would be able to come up with this right away. Would it make sense to add something along these lines to contextlib? Maybe instead of a new decorator, ContextDecorator could also take an additional keyword argument that tells it to pass the context manager to the decorated function? Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

2 3