Mailman 3 October 2010 - Python-ideas

Fwd: stats module Was: minmax() function ...
by Raymond Hettinger 15 Oct '10

15 Oct '10

Drat. This should have gone to python-ideas. Re-sending. Begin forwarded message: > From: Raymond Hettinger <raymond.hettinger(a)gmail.com> > Date: October 15, 2010 1:00:16 PM PDT > To: Python-Dev Dev <python-dev(a)python.org> > Subject: Fwd: [Python-ideas] stats module Was: minmax() function ... > > Hello guys. If you don't mind, I would like to hijack your thread :-) > > ISTM, that the minmax() idea is really just an optimization request. > A single-pass minmax() is easily coded in simple, pure-python, > so really the discussion is about how to remove the loop overhead > (there isn't much you can do about the cost of the two compares > which is where most of the time would be spent anyway). > > My suggestion is to aim higher. There is no reason a single pass > couldn't also return min/max/len/sum and perhaps even other summary > statistics like sum(x**2) so that you can compute standard deviation > and variance. > > A few years ago, Guido and other python devvers supported a > proposal I made to create a stats module, but I didn't have time > to develop it. The basic idea was that python's batteries should > include most of the functionality available on advanced student > calculators. Another idea behind it was that we could invisibility > do-the-right-thing under the hood to help users avoid numerical > problems (i.e. math.fsum(s)/len(s) is a more accurate way to > compute an average because it doesn't lose precision when > building-up the intermediate sums). > > I think the creativity and energy of this group is much better directed > at building a quality stats module (perhaps with some R-like capabilities). > That would likely be a better use of energy than bike-shedding > about ways to speed-up a trivial piece of code that is ultimately > constrained by the cost of the compares per item. > > my-two-cents, > > > Raymond

2 1

String Subtraction
by Dave Jakeman 15 Oct '10

15 Oct '10

I'm new to Python and this is my first suggestion, so please bear with me: I believe there is a simple but useful string operation missing from Python: subtraction. This is best described by way of example: >>> "Mr Nesbit has learnt the first lesson of not being seen." - "Nesbit " 'Mr has learnt the first lesson of not being seen.' >>> "Can't have egg, bacon, spam and sausage without the spam." - " spam" 'Can't have egg, bacon, and sausage without the spam.' >>> "I'll bite your legs off!" - "arms" 'I'll bite your legs off!' If b and c were strings, then: a = b - c would be equivalent to: if b.find(c) < 0: a = b else: a = b[:b.find(c)] + b[b.find(c)+len(c):] The operation would remove from the minuend the first occurrence (searching from left to right) of the subtrahend. In the case of no match, the minuend would be returned unmodified. To those unfamiliar with string subtraction, it might seem non-intuitive, but it's a useful programming construct. Many things can be done with it and it's a good way to keep code simple. I think it would be preferable to the current interpreter response: >>> record = line - newline Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for -: 'str' and 'str' As the interpreter currently checks for this attempted operation, it seems it would be straightforward to add the code needed to do something useful with it. I don't think there would be backward compatibility issues, as this would be a new feature in place of a fatal error.

8 9

Pythonic Dates, Times, and Deltas
by Daniel G. Taylor 15 Oct '10

15 Oct '10

Hey, I've recently been doing a lot of work with dates related to payment and statistics processing at work and have run into several annoyances with the built-in datetime, date, time, timedelta, etc classes, even when adding in relativedelta. They are awkward, non-intuitive and not at all Pythonic to me. Over the past year I've written up a library for making my life a bit easier and figured I would post some information here to see what others think, and to gauge whether or not such a library might be PEP-worthy. My original post about it was here: http://programmer-art.org/articles/programming/pythonic-date The github project page is here: http://github.com/danielgtaylor/paodate This is code that is and has been running in production environments for months but may still contain bugs. I have tried to include unit tests and ample documentation. I'd love to get some feedback and people's thoughts. I would also love to hear what others find is difficult or missing from the built-in date and time handling. Take care, -- Daniel G. Taylor http://programmer-art.org/

10 13

Re: [Python-ideas] [Python-Dev] Inclusive Range
by Nick Coghlan 07 Oct '10

07 Oct '10

On Mon, Oct 4, 2010 at 5:27 PM, Xavier Morel <python-dev(a)masklinn.net> wrote: > Same as the others: > 0. This is a discussion for python-ideas, I'm CCing that list > 1. This is a major backwards compatibility breakage, and one which is entirely silent (`print` from keyword to function wasn't) > 2. It loses not only well-known behavior but interesting properties as well (`range(n)` has exactly `n` elements. With your proposal, it has ``n+1`` breaking ``for i in range(5)`` to iterate 5 times as well as ``for i in range(len(collection))`` for cases where e.g. ``enumerate`` is not good enough or too slow) > 3. As well as the relation between range and slices > 4. I fail to see how it is more intuitive (let alone more practical, see previous points) > 5. If you want an inclusive range, I'd recommend proposing a flag on `range` (e.g. ``inclusive=True``) rather than such a drastic breakage of ``range``'s behavior. That, at least, might have a chance. Changing the existing default behavior of range most definitely doesn't. A flag doesn't have any chance either - you spell inclusive ranges by including a "+1" on the stop value. Closed ranges actually do superficially appear more intuitive (especially to new programmers) because we often use inclusive ranges in ordinary speech ("10-15 people" allows 15 people, "ages 8-12" includes 12 year olds, "from A-Z" includes items starting with "Z"). However, there are some cases where we naturally use half-open ranges as well (such as "between 10 and 12" excluding 12:01 to 12:59) or explicitly invoke exclusive ranges as being easier to deal with (such as the "under 13s", "under 19s", etc naming schemes used for age brackets in junior sports) However, as soon you move into the mathematical world (including programming), closed ranges turn out to require constant adjustments in the arithmetic, so it far more natural to use half-open ranges consistently. Xavier noted the two most important properties of half-closed ranges for Python: they match the definition of subtraction, such that len(range(start, stop)) = (stop - start), and they match the definition of slicing as being half-open. As to whether slicing itself being half-open is beneficial, the value of that becomes clear ones you start trying to manipulate ranges: With half-open slices, the following is true: s == s[:i] + s[i:] With inclusive slices (which would be needed to complement inclusive range), you would need either a -1 on the stop value of the first slice, or a +1 on the start value of the second slice. Similarly, if you know the length of the slice you want, then you can grab it via s[i:i+slice_len], while you'd need a -1 correction on the stop value if slices were inclusive. There are other benefits to half-open ranges when it comes to (approximately) continuous spectra like time values, floating point numbers and lexically ordered strings. Being able to say things like "10:00" <= x < '12:00", 10.0 <= x < 12.0, "a" <= x < "n" are much clearer than trying to specify their closed range equivalents. While that isn't specifically applicable to the range() builtin, it is another factor in why it is important to drink the "half-open ranges are your friend" Kool-aid as a serious programmer. Cheers, Nick. P.S. many of the points above are just rephrased from http://www.siliconbrain.com/ranges.htm, which is the first hit when Googling "half-open ranges" -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

16 28

A general purpose range iterator (Re: improvements to slicing)
by Nick Coghlan 06 Oct '10

06 Oct '10

On Thu, Oct 7, 2010 at 5:21 AM, Ron Adam <rrr(a)ronadam.com> wrote: > I think these are easier to use than the current behavior. It doesn't > change slices using positive indexes and steps so maybe it's not so backward > incompatible to sneak in. ;-) I think that sound you just heard was thousands of SciPy users crying out in horror ;) Given a "do over", there a few things I would change about Python's range generation and extended slicing. Others would clearly change a few different things. Given the dual barriers of "rough consensus and running code", I don't think there are any *specific* changes that would make it through the gauntlet. The idea of a *generalised* range generator is in interesting one though. One that was simply: _d = object() def irange(start=_d, stop=_d, step=1, *, include_start=True, include_stop=False): # Match signature of range while still allowing stop=val as the only keyword argument if stop is _d: start, stop = 0, start elif start is _d: start = 0 if include_start: yield start current = start while 1: current += step if current >= stop: break yield current if include_stop and current == stop: yield stop Slower than builtin range() for the integer case, but works with arbitrary types (e.g. float, Decimal, datetime) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

Re: [Python-ideas] [Python-Dev] Inclusive Range
by Masklinn 04 Oct '10

04 Oct '10

On 2010-10-04, at 05:04 , Eviatar Bach wrote: > Hello, > > I have a proposal of making the range() function inclusive; that is, > range(3) would generate 0, 1, 2, and 3, as opposed to 0, 1, and 2. Not only > is it more intuitive, it also seems to be used often, with coders often > writing range(0, example+1) to get the intended result. It would be easy to > implement, and though significant, is not any more drastic than changing > print to a function in Python 3. Of course, if this were done, slicing > behaviour would have to be adjusted accordingly. > > What are your thoughts? Same as the others: 0. This is a discussion for python-ideas, I'm CCing that list 1. This is a major backwards compatibility breakage, and one which is entirely silent (`print` from keyword to function wasn't) 2. It loses not only well-known behavior but interesting properties as well (`range(n)` has exactly `n` elements. With your proposal, it has ``n+1`` breaking ``for i in range(5)`` to iterate 5 times as well as ``for i in range(len(collection))`` for cases where e.g. ``enumerate`` is not good enough or too slow) 3. As well as the relation between range and slices 4. I fail to see how it is more intuitive (let alone more practical, see previous points) 5. If you want an inclusive range, I'd recommend proposing a flag on `range` (e.g. ``inclusive=True``) rather than such a drastic breakage of ``range``'s behavior. That, at least, might have a chance. Changing the existing default behavior of range most definitely doesn't. I'd be −1 on your proposal, −0 on adding a flag to ``range`` (I can't recall the half-open ``range`` having bothered me recently, if ever)

1 0