From greg.ewing at canterbury.ac.nz  Sat Mar  1 00:34:05 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 01 Mar 2014 12:34:05 +1300
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
Message-ID: <53111CED.2050801@canterbury.ac.nz>

Chris Angelico wrote:
> With this proposal, your
> star_lambda function's declaration changes the call site - instead of
> evaluating a*b+c, it has to construct an anonymous function and pass
> it along.

And when you consider that this could happen with any
argument to any function, depending on what the function
turns out to be like at run time, it means that *all*
function arguments would need to be passes as anonymous
functions. That would be very awkward an inefficient.

-- 
Greg

From ron3200 at gmail.com  Sat Mar  1 03:13:50 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 28 Feb 2014 20:13:50 -0600
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
 <leqork$it2$1@ger.gmane.org>
 <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com>
Message-ID: <lerfog$9ej$1@ger.gmane.org>



On 02/28/2014 04:33 PM, Nick Coghlan wrote:
> On 1 Mar 2014 05:43, "Ron Adam"
> <ron3200 at gmail.com
> <mailto:ron3200 at gmail.com>> wrote:
>  >
>  >
>  >
>  > On 02/28/2014 11:54 AM, Chris Angelico wrote:
>  >>
>  >> On Sat, Mar 1, 2014 at 4:17 AM, Ron
> Adam<ron3200 at gmail.com
> <mailto:ron3200 at gmail.com>>  wrote:
>  >>>
>  >>> >This returns a lambda-like function.
>  >>> >
>  >>> >       def star_lambda(***expr): return expr
>  >>> >
>  >>> >
>  >>> >And is used this way...
>  >>> >
>  >>> >       result = star_lambda(a * b + c)  # captures expression.
>  >>> >
>  >>> >       actual_result = ***result        # *** resolves "result" here!
>  >>> >
>  >>
>  >> Interesting, but I don't like the way the interpretation of a function
>  >> call depends on the target function. With both * and ** notations,
>  >> there's absolutely no difference: the function is called with these
>  >> positional and those keyword arguments, whether they came from actual
>  >> args or from * or ** unpack/repacks;and there's no difference between
>  >
>  > > a function that collects args with *args,**kwargs and one that
>  > > collects them with individual names (or a C-level function that might
>  > > do something altogether different).
>  >
>  >
>  > It's not clear what differences you mean here... can you show some examples?
>
> Remember that at compile time, Python has *no idea* what the actual
> signature of the target function is. Thus, all Python function calls use
> the following sequence (ignoring optimisations of special cases):
>
> 1. At the call site, the arguments are collected into a tuple of positional
> arguments and a dict of keyword arguments.
> 2. The interpreter hands that tuple and dict over to the target callable
> 3. The *target callable* then maps the supplied arguments to the defined
> parameters including filling in any default values.
>
> Any function related proposals need to account for the fact that from the
> compiler's point of view *every* function signature looks like "(*args,
> **kwds)" (although it may have optimised paths for the no-args case and the
> positional-args-only case), and that the target callable may not even be
> written in Python.

So functions can't be extended to take a triplet instead of a pair...

       (*args, **kwds, ***expr)


Looking up...  I think this is what you're referring to in ceval.c.

#-------------------

/* External interface to call any callable object.
    The arg must be a tuple or NULL.  The kw must be a dict or NULL. */

PyObject *
PyEval_CallObjectWithKeywords(PyObject *func, PyObject *arg, PyObject *kw)
{

#--------------------

And it wouldn't work in the normal case any way, as expressions are 
evaluated as they are put on the stack before calling the function.

Somehow I was thinking this morning the code inside the function call 
parentheses f(...), could be parsed later than it actually is.  And in the 
context of the function definition, possibly similar to how a comprehension 
is evaluated.  But it would take some pretty big changes to do that I supose.

Cheers,
    Ron


From rosuav at gmail.com  Sat Mar  1 03:14:52 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Mar 2014 13:14:52 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <leqork$it2$1@ger.gmane.org>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
 <leqork$it2$1@ger.gmane.org>
Message-ID: <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com>

On Sat, Mar 1, 2014 at 6:42 AM, Ron Adam <ron3200 at gmail.com> wrote:
> It's not clear what differences you mean here... can you show some examples?

These are exactly the same:

x = (1,2)
f(*x)

f(1,2)

I played with dis.dis() and it seems there are special-case opcodes
for calling a function with a variable number of arguments and/or with
variable keyword args; but once it arrives at the other side, the two
are identical:

>>> def f(*args):
    print(args)
>>> f(1,2)
(1, 2)
>>> x=(1,2)
>>> f(*x)
(1, 2)

The runtime fetches some callable, gives it some args, and says "Go do
your stuff!". It doesn't care what that callable is - it could be a
classic function defined with 'def' or 'lambda', it could be a type,
it could be an object with __call__, it could be a built-in that's
backed by a C function, anything. All that ends up arriving on the
other side is: You have these positional args and these keyword args.

Adding the tri-star to the mix suddenly changes that. A function is
now capable of taking an expression, rather than an object. That's
completely different, and it depends on the called function to
distinguish one from the other.

ChrisA

From rosuav at gmail.com  Sat Mar  1 03:17:10 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Mar 2014 13:17:10 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <lerfog$9ej$1@ger.gmane.org>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
 <leqork$it2$1@ger.gmane.org>
 <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com>
 <lerfog$9ej$1@ger.gmane.org>
Message-ID: <CAPTjJmrvQUihCG6kuPA3Y_9mzH5O3PqH5VKHk1dQC1Ahdzzt0Q@mail.gmail.com>

On Sat, Mar 1, 2014 at 1:13 PM, Ron Adam <ron3200 at gmail.com> wrote:
> Somehow I was thinking this morning the code inside the function call
> parentheses f(...), could be parsed later than it actually is.  And in the
> context of the function definition, possibly similar to how a comprehension
> is evaluated.  But it would take

The best way to do it would be to adorn the call site. We can
currently do that with lambda:

f(a * b + c)
f(lambda: a * b + c)

Those are completely different from each other, but completely
consistent with themselves - it doesn't matter what f is, one of them
passes the sum of the product and the other passes a callable.

ChrisA

From ian.team.python at gmail.com  Sat Mar  1 04:09:02 2014
From: ian.team.python at gmail.com (ian o)
Date: Fri, 28 Feb 2014 19:09:02 -0800 (PST)
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
Message-ID: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>



>
> And we don't actually mind all that much if applications don't migrate 
> in the near term - Python 2 will have commercial support available 
> well past 2020. (The developers of *those applications* might mind, 
> though, just as anyone maintaining Python 2.4 compatibility for the 
> benefit of RHEL/CentOS 5 typically isn't happy about it) 
>
>
> Your response is most dissapointing, and I would hope does not represent 
the community overall.

We have 3 teams working server systems currently with python as the 
implementation model.  All work is currently in python 2.x and with a 
solution to this the work could immediately move to 3.x.

However, you state that our situation, and that of the other 60% of python 
programmers as at end January 2014 who are constrained from using python 3 
by python 2 dependencies:  "we don't mind about those developers"

And of course the dependent library module developers will not update the 
library, because 'no one out there uses python 3.x'.   And from your 
perspective this is simply not of any interest.

No wonder Tiobe index shows python dropping and in the words of those 
trying to persuade me to move the projects to another language 'python 3.x' 
came out 5 years ago and we still cannot move to it, this is going no where.

I would like to still have the developers work in python, and to do that 
there needs to be a path to python 3.  I also would prefer to be in python 
3 anyway.  But even if a workable solution is there, your answer is who 
cares about 60% of developers anyway!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/d60d0773/attachment.html>

From steve at pearwood.info  Sat Mar  1 04:46:11 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 1 Mar 2014 14:46:11 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <leqgah$b0o$1@ger.gmane.org>
References: <leqgah$b0o$1@ger.gmane.org>
Message-ID: <20140301034611.GD28804@ando>

On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote:
> 
> Starting new thread because this bike has a different shape and color.
> 
> Yesterday I was thinking that just making the keyword lambda assignable 
> like True, False, and None, would be enough.

You can't assign to True, False or None. (You can assign to True and 
False in Python 2, but shouldn't.)


[...]
> This morning I thought we could have in a functions definition something, 
> like "*", and "**", to take an expression.  Similar to Nicks idea with =:, 
> but more general.
> 
> The idea is to have "***" used in def mean to take "any" call expression 
> and not evaluate it until *** is used on it.
[...]
> A function call that captures an expression may be tricky to do. Here's one 
> approach that requires sugar when a function defined with "***" is called.

I think it would be useful to have a way to delay execution of an 
expression, that is to say, have a way to capture an expression for 
later evaluation, something more lightweight than a function, but I 
don't think that limiting it to inside function calls is the right 
approach.

Something perhaps like a thunk might be appropriate? We can *almost* do 
that now, since Python has a compile function:

thunk = compile("x + 3", "", "eval")
# much later
eval(thunk)

Some problems with that approach:

- You have to write the expression as a string, which means you lose any 
possibility of syntax highlighting.

- The temptation is to pass some arbitrary untrusted string, which leads 
to serious security implications. The argument here should be limited 
to an actual expression, not a string containing an expression which 
might have come from who knows where.

- It should be as lightweight as possible. The actual compilation of the 
expression should occur at compile-time, not run-time. That implies some 
sort of syntax for making thunks, rather than a function call.

- Likewise actually evaluating the thunk should be really lightweight, 
which may rule out a function call to eval.

- How should scoping work? I can see use-cases for flat scoping, static 
scoping, dynamic scoping, and the ability to optionally provide custom 
globals and locals, but I have no idea how practical any of them would 
be or what syntax they should use.


All of this is pie-in-the-sky at the moment, and not a serious proposal 
for Python 3.5.


-- 
Steven

From ncoghlan at gmail.com  Sat Mar  1 04:57:42 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 1 Mar 2014 13:57:42 +1000
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
Message-ID: <CADiSq7fo-gDCtZ0NrNkk4BftBgaCTSvR1X8FBs-97AT914WxFA@mail.gmail.com>

On 1 March 2014 13:09, ian o <ian.team.python at gmail.com> wrote:
>> And we don't actually mind all that much if applications don't migrate
>> in the near term - Python 2 will have commercial support available
>> well past 2020. (The developers of *those applications* might mind,
>> though, just as anyone maintaining Python 2.4 compatibility for the
>> benefit of RHEL/CentOS 5 typically isn't happy about it)
>>
> Your response is most dissapointing, and I would hope does not represent the
> community overall.

My apologies, I wrote my post assuming you had already read
http://docs.python.org/dev/howto/pyporting.html and decided that the
existing approaches described there weren't sufficient for your
purposes. Those cases where the recommended migration process isn't
considered acceptable for some reason (usually non-technical ones) are
the cases that I consider to fall into the "it's OK to just keep using
Python 2" category.

However, I just realised two things:

  * that version of the guide is currently only in the Python 3.4
docs, and so the page at http://docs.python.org/3/howto/pyporting.html
is still displaying the older 3.3 version of the guide (the page in
the Python 2.7 docs is similarly outdated).

  * that guide is currently aimed primarily at library and framework
authors, and doesn't explicitly discuss the migration of integrated
applications.

I have now filed http://bugs.python.org/issue20812 and
http://bugs.python.org/issue20813 to address those limitations, but in
the meantime, I will provide the missing information here:

For developers of integrated applications that currently still have
some dependencies on Python 2, the preferred migration path is to use
tools like python-modernize or python-future to shift first into the
large common subset of Python 2 and Python 3, and then only later
switch fully to Python 3. This approach permits application developers
to take the following path:

1. Python 2 only (status quo)
2. Python 2/3 compatible on Python 2 (waiting for dependencies)
3. Python 2/3 compatible on Python 3 (dependencies ported or replaced)
4. Python 3 only (drop Python 2 support)

Brett Cannon's "caniusepython3" tool
(https://pypi.python.org/pypi/caniusepython3/) is designed to automate
the dependency analysis to see if all declared dependencies are Python
3 compatible (or have suitable alternatives available). However, if
you're using system packages for dependency management, some data
transformations will be needed to convert them to a form that the tool
understands.

My original reply assumed that you were already aware of the details
of this preferred approach before posting your proposal, and I now
realise that assumption was likely incorrect. Once again, my
apologies.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Sat Mar  1 04:58:08 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 28 Feb 2014 19:58:08 -0800
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <lerfog$9ej$1@ger.gmane.org>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
 <leqork$it2$1@ger.gmane.org>
 <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com>
 <lerfog$9ej$1@ger.gmane.org>
Message-ID: <D88EB01E-48A7-418E-AF5F-3C1895CAA568@yahoo.com>

On Feb 28, 2014, at 18:13, Ron Adam <ron3200 at gmail.com> wrote:

> So functions can't be extended to take a triplet instead of a pair...
> 
>     (*args, **kwds, ***expr)

The more you elaborate this, the more this looks like Ruby procs: You get any number of normal arguments, then at most one special kind of callback thing which must come at the end.

I'm not sure which use cases this solves. It doesn't work for expressions that need an argument, like a sorting key function. It also doesn't work for pre-existing functions that weren't designed to take ***expr, like Button or takewhile. And wrapping the expression in a call to a forwarding function doesn't seem any less verbose or more readable than just using lambda.

Anyway, there are two fundamental problems with anything that doesn't require any syntax at the call site. First, it's not obvious, or even easily discoverable, to the reader when you're passing a quoted expression and when you're passing a value. Second, it's not discoverable to the compiler.

Going back to the analogies from other languages again, Lisp gets around this by making functions and macros different things. Among other differences, a macro always gets expressions instead of values, without any need to quote them at the call site. And, as Haoyi Li has pointed our multiple times on the other threads, you can already do the same thing in Python (with MacroPy).

From ron3200 at gmail.com  Sat Mar  1 05:07:27 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 28 Feb 2014 22:07:27 -0600
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org>
 <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com>
 <leqork$it2$1@ger.gmane.org>
 <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com>
Message-ID: <lermdh$9lh$1@ger.gmane.org>



On 02/28/2014 08:14 PM, Chris Angelico wrote:
> On Sat, Mar 1, 2014 at 6:42 AM, Ron Adam <ron3200 at gmail.com> wrote:
>> It's not clear what differences you mean here... can you show some examples?
>
> These are exactly the same:
>
> x = (1,2)
> f(*x)
 >
> f(1,2)
 >
> I played with dis.dis() and it seems there are special-case opcodes
> for calling a function with a variable number of arguments and/or with
> variable keyword args; but once it arrives at the other side, the two
> are identical:
>
>>>> def f(*args):
>      print(args)
>>>> f(1,2)
> (1, 2)
>>>> x=(1,2)
>>>> f(*x)
> (1, 2)
>
> The runtime fetches some callable, gives it some args, and says "Go do
> your stuff!". It doesn't care what that callable is - it could be a
> classic function defined with 'def' or 'lambda', it could be a type,
> it could be an object with __call__, it could be a built-in that's
> backed by a C function, anything. All that ends up arriving on the
> other side is: You have these positional args and these keyword args.
>
> Adding the tri-star to the mix suddenly changes that. A function is
> now capable of taking an expression, rather than an object. That's
> completely different, and it depends on the called function to
> distinguish one from the other.

Right..  To make it work in the way I was thinking would require each 
function consisting of two parts.  One part to build a name-space, and the 
other for executing the code.  Then at call time, the first part could 
evaluate the expressions inside the function parentheses. This part would 
be new, and possibly work like a dict comprehension, except more specific 
to call signatures.  Then the constructed name space would be passed 
directly to the code part for the actual call.  Which would be quite 
different than what happens presently.


I've refactored dis once for fun, and also played around with ceval enough 
to know how this stuff works, but when I haven't looked at the code 
recently,  (like now) I tend to think more in abstract terms which helps 
with creativity and seeing new ways to do things.  (And it is why I like 
reading this list), But I sometimes misses on the objectivity side... [Note 
to self... look at the code more.]

It comes down to this...

    "Creativity and Objectivity often don't want
     to occupy the same space at the same time."
also helped test some patches to dis as well.  And
Cheers,
    Ron


From steve at pearwood.info  Sat Mar  1 05:07:40 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 1 Mar 2014 15:07:40 +1100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
Message-ID: <20140301040740.GF28804@ando>

On Thu, Feb 27, 2014 at 10:03:42PM -0800, ian o wrote:
> Chris, great reply.
> 
> Yes it is not easy. Especially if you wish to make use of python2 modules 
> painless.
> 
> And I agree wholeheartedly that serious thought is needed on the 'could it 
> work so well that it actually slow the migration of library modules?'.

I think that such a bridge will slow the migration of library modules, 
since it reduces the need to migrate. As a library author, why should I 
spend tens or hundreds of hours migrating my library to Python 3 when I 
can spend 10 minutes adding a note to the library's website telling 
people to use the bridge?

The problem with this suggestion is that it does nothing to encourage 
library authors to migrate to 3, it simply enables them to delay 
migrating even longer, possibly for so long that they lose interest 
altogether.

That's library authors. How about the developers who use those 
libraries?

Right now, there is some motivation for people to move to 3: cool new 
features, and getting cooler by the year. With this suggested bridge, 
they can migrate to 3 while still remaining dependent on 2. This might 
seem like a win if you only focus on the short term uptake of 3, but it 
misses the fact that they're still dependent on 2. They've now *trebled* 
the number of language dependencies, from Python only to two versions of 
Python plus the bridge software.

Having spent the time and effort to migrate to 3, in a few years they'll 
have to go through the same pain again to get rid of the dependency on 2 
and the bridge. Nobody will want to do that. The pressure on the core 
developers to keep supporting 2 forever will increase, not decrease.


> This is an important debate to have.  I suggest if the answer to 'can a 
> specification be created that will significantly accelerate migration?', 
> then it is worth the effort to deliver this.

I think you are making a classic mistake of technically-minded people: 
looking for a technical solution to a social problem.

The problem with Python 3 uptake is not a technical problem, or at least 
not *only* a technical problem. It is mostly a social problem: the costs 
of migrating are greater than the benefits. Python 3 is great, but for 
most people Python 2 does the job.

Another social problem is that the minority of library authors who 
haven't migrated haven't done so because they personally don't care 
about Python 3, or they don't have the time or money to spend on it. 
Adding a bridge doesn't suddenly give them more time or money.

For application developers, the social problem is less because of a few 
libraries and more to do with the supported standard Python on their 
system. That will change once Linux distros start providing 3 as their 
standard system Python; out of the big distros, Fedora is about to do 
so, so things will start changing soon.

But perhaps the biggest reason for slow uptake of Python 3 with existing 
projects is that nobody wants to spend money just to migrate to 3 for 
the sake of migrating to 3. "Written in Python 3" is not a selling 
point. Even the availability of security updates is not much of a 
selling point, although it will become more important when Python 2.7 is 
end-of-lifed. The reality is that many apps work perfectly well now 
using Python 2.7, or 2.6, or even 2.4, and they'll continue to work well 
in five or ten years so there's no real need to update. At the last US 
PyCon, there was even one fellow still using Python 1.5. It works, he 
didn't need security updates, so why migrate?

*This is not a problem to be solved.* It's okay for people to stick with 
an old version of Python. There's no rule that says Python has failed if 
some developers stick to old versions.


> As things stand, the popularity of Python has dropped by most measures 
> since the release of Python 3. Improve the language and lose market share?  
> I suggest that is all about the pain of the transition.

That sounds remarkably like FUD to me. Where is your evidence that the 
popularity of Python has fallen?

Personally, I think all the "language popularity" measures are dubious, 
or at least need to be treated with considerable skepticism. Take this 
one:

http://langpop.com/

which lists BrainF--- in the 40 most popular languages. Sure it is.

The PyPL popularity index here:

https://sites.google.com/site/pydatalog/pypl/PyPL-PopularitY-of-Programming-Language

shows Python as increasing in popularity. In contrast, TIOBE shows 
Python as losing popularity -- but so did Java, PHP, C++ and Ruby, and 
none of them have a major version change. (TIOBE also shows that despite 
a large number, possibly majority, of VB 6 developers boycotting VB.Net, 
VB.Net has just entered the top 10 most popular languages.)

And then there's CodeEval, which for the third year in a row has found 
that Python is more popular than Java, C++, C, PHP and Javascript. In 
fact more popular than Java, C and PHP *together*.

http://blog.codeeval.com/codeevalblog/2014

Bless their little white socks, but methinks their methodology is 
perhaps a tad flawed.

I think it is interesting to compare and contrast these different ways 
of measuring popularity, but then I'm a statistics wonk and I find even 
flawed statistics interesting. Remember that there is no one true 
definition of popularity, even if there were there is no accurate way of 
measuring it, and all of the sites that claim to do so are actually 
measuring subtley different things.

Most importantly, chasing popularity for its own sake is a waste of 
time. What makes Python Python is not just the syntax and std lib, but 
also the culture -- we're neither the stultifying corporate culture of 
"Java shops", nor the wild-west anything goes of cowboy PHP developers, 
but something unique. (As are other languages, of course. Perl culture 
is not Haskell culture is not Erlang culture.) In part, people take up 
languages because they like the culture. Python will never attract the 
corporate suits who like the Java philosophy, nor will it attract the 
cowbody web developers who like the PHP way of doing things. Should we 
try to ape PHP or Java to attract the cowboys or suits? Or should we 
continue to make Python the best language that *we* like, regardless of 
what other people prefer?

Untimately, apart from winning pissing contests on Slashdot and Reddit, 
what does it matter if Python is third most popular or ninth most 
popular? Its not a popularity context where the winner takes all. 
According to TIOBE, the most popular language, C, has less than 20% 
share. If you think Python is a failure because it is in position 8, 
what does that make Javascript in position 9?



> You do not have to search hard to find comments to the effect 'nobody is 
> using python3'.   Or 'even if you use python 3, do not use the new features 
> since you need to ensure code is compatible with both versions'.

You don't have to search very hard to find comments that Elvis was 
kidnapped by aliens.

https://duckduckgo.com/html/?q=elvis+kidnapped+by+aliens

A belief which has rather more going for it than either of the two 
comments you list above, since there's a miniscule, microscopic chance 
that perhaps Elvis *was* kidnapped by aliens, whereas I know for a fact 
beyond any doubt that some people are using Python 3 and using the new 
features.



-- 
Steven

From ian.team.python at gmail.com  Sat Mar  1 05:10:04 2014
From: ian.team.python at gmail.com (ian o)
Date: Fri, 28 Feb 2014 20:10:04 -0800 (PST)
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
Message-ID: <f00701bc-f041-4f65-aa24-c75bc9b835ec@googlegroups.com>



> And we don't actually mind all that much if applications don't migrate 
> in the near term - Python 2 will have commercial support available 
> well past 2020. (The developers of *those applications* might mind, 
> though, 
>
..... 

> The group we *do* really care about supporting is authors of existing 
> Python *libraries*, and "2in3" and "3in2" approaches don't really help 
> them at all, since library and framework authors with users on both 
> Python 2 and Python 3 will likely face demand to support both versions 
> natively anyway.


Cheers, 
> Nick. 
>
> Nic,
>
Lets assume  for the moment that the 'we don't really care about the 60% of 
python developers blocked from moving to python 3 by depencies' was not 
really what you meant.

I am going to assume that what you really meant was 'I do not think helping 
these people directly is the best solution'.
And this attitude is based around the fear that if the developers can move 
to python 3 before the libraries all support python 3, then that will 
remove pressure from the libraries to support python 3.

This fear i can understand.  But I suggest some real world experience shows 
this approach is actually not assisting either group.  It is certainly not 
assisting the developers wishing to move to python 3, but it is also making 
life hard for the "people we do really care about"

So not supporting a "2in3" solution forces programmers to with python2 
dependencies to hold off moving to python3 until their dependencies 
migrate, but hopefully with sound motives.

Now take a typical dependency.  Web2py.  Speak to them on migration and 
they will telly you 'none of our users have moved to python 3'.  So the 
users are held back from moving by the dependencies, and the dependencies 
are held back by the users.

I would suggest that making it easier to support both python2 and ptyhon3 
from a library is simply reducing what it a painful thing.  I would suggest 
that allowing all the end users to migrate to python3 as soon as possible 
is a far more attractive solution.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/5a908817/attachment.html>

From abarnert at yahoo.com  Sat Mar  1 06:09:03 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 28 Feb 2014 21:09:03 -0800
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
Message-ID: <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com>

On Feb 28, 2014, at 19:09, ian o <ian.team.python at gmail.com> wrote:

> However, you state that our situation, and that of the other 60% of python programmers as at end January 2014 who are constrained from using python 3 by python 2 dependencies:  "we don't mind about those developers"

You're twisting the numbers. I'm part of the 60% who have to work on Python 2.x for at least one project. But first, I have others--including shipping commercial code both on the desktop and the server--using 3.x. And second, in all but one case, the 2.x apps are 2.x not because of lacking library support, but because we have an already-working and -deployed app that's effectively in maintenance mode, and it's not worth the effort or risk to port it to 3.x. (In other words, most of my 2.x work points to the _success_ of the migration plan, not a failure.)

And you're also twisting Nick's words. He said that he cares more about getting library developers to 3.x than application developers. And he explained why that makes life better for application developers. To take that as "the Python devs don't care about application developers" is willfully misinterpreting the point.

> And of course the dependent library module developers will not update the library, because 'no one out there uses python 3.x'.  

None of the library devs I've ever dealt with has ever said that (well, not since the early 3.1 days).

Most projects that don't support 3.x today want to do so, it just isn't always the very highest thing on their priority list. Three times I've had to port a library myself rather than waiting around--but in every case, the library's dev was happy to take my code, even if it meant losing 2.5 support, reviewing large changelists, etc.

I'm sure there are some exceptions, but it's certainly not the ubiquitous case you're presenting.

There's a reason the Python 3 Wall of Shame changed it's name to Wall of Superpowers and the competing py3ksupport page stopped updating: because people have been porting important libraries to 3.x, and continue to do so.

> No wonder Tiobe index shows python dropping

And is the Python 3 migration also to blame for Python's direct competitors (Ruby, Perl, and PHP) also dropping)? Or for the general trend in Tiobe numbers to show ObjC and Windows-specific languages continually gaining on cross-platform languages?

> and in the words of those trying to persuade me to move the projects to another language 'python 3.x' came out 5 years ago and we still cannot move to it, this is going no where.

This is pure FUD. And I don't see how repeating that FUD to the Python devs is going to help you. If you want to convince these people to stay on Python, you need to start talking specifics instead of talking points. What libraries are stopping you from moving to 3.x? Does Ruby or Node.js or whatever actually have a better alternative for all of those libraries than Python 3 does? Would it really be easier to write those missing libraries yourself in a new language than to port existing Python 2.x libraries to 3.x?

For that matter, have you actually checked whether those libraries really are blocking you? I've lost count of the number of people who've claimed they can't switch to Python 3 because of some library that's been ported since 2010 or that has a 3.x-compatible drop-in replacement.


From ian.team.python at gmail.com  Sat Mar  1 06:53:13 2014
From: ian.team.python at gmail.com (ian o)
Date: Fri, 28 Feb 2014 21:53:13 -0800 (PST)
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <20140301040740.GF28804@ando>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
Message-ID: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>

Steven,
Great post.  Arguments can be enjoyed when people with differing views 
argue them well.
Let me try and put the counter case as well.

I have to start with the Elvis abducted by aliens just because it is such a 
great place to start.
I enjoyed your point. But let me counter by saying that the information 
that some people say 'Elvis was abducted by aliens' does not mean this is 
what actually happened, but it does tell us that some people believe this!  
And there is rather surprising information in the fact that 'some people 
believe Elvis was abducted by Aliens'.  Similarly I do not believe 'no-one 
is using python 3'.  I program in python3 (and wish i could for all 
programming).  But it is still interesting that people are saying it.

Other comments inline below:

I think that such a bridge will slow the migration of library modules, 
> since it reduces the need to migrate. As a library author, why should I 
> spend tens or hundreds of hours migrating my library to Python 3 when I 
> can spend 10 minutes adding a note to the library's website telling 
> people to use the bridge? 
>
> The problem with this suggestion is that it does nothing to encourage 
> library authors to migrate to 3, it simply enables them to delay 
> migrating even longer, possibly for so long that they lose interest 
> altogether. 
>
> I suggest the counter point here is to consider life a library developer.  
Which motivates you: that fact that people using your library all program 
in python3, or hearing that all your users program in python2?  
  

> That's library authors. How about the developers who use those 
> libraries? 
>
> Right now, there is some motivation for people to move to 3: cool new 
> features, and getting cooler by the year. With this suggested bridge, 
> they can migrate to 3 while still remaining dependent on 2. This might 
> seem like a win if you only focus on the short term uptake of 3, but it 
> misses the fact that they're still dependent on 2. They've now *trebled* 
> the number of language dependencies, from Python only to two versions of 
> Python plus the bridge software. 
>
> Are you assuming that it will always be either a) all dependencies require 
python2 
or b) all dependencies are available in python 3.   Rather than c) some 
dependencies are available in python3 but the python3 version cannot be 
used because I am being held back to python2 by some other critical 
dependencies.
For the projects i work on, it is c).  It is not like there has been no 
success getting packages to migrate.  The problem is that an entire project 
is held back by one single dependency.

And for us we would wear some pain right now and make that move.  Most of 
the project doesn't have tow steps and can be moved immediately. If a 
bridge is offered, it would not be a great idea to force everyone to use 
it.  If EVERY imported module is still in python2 then I suggest it is too 
early to move.  Demonstrably, some people like the idea, and I suggest it 
would be interesting to discover how many.  There may be cases where it is 
not the right choice because too many dependencies are only available in 
python3 , but alone that does not mean the choice should not be offered.  
Personally I beleive that now many dependencies can be found in python3, 
hence the frustration of having to wait until every single one moves seems 
unfortunate.

Having spent the time and effort to migrate to 3, in a few years they'll 
> have to go through the same pain again to get rid of the dependency on 2 
> and the bridge. Nobody will want to do that. The pressure on the core 
> developers to keep supporting 2 forever will increase, not decrease. 
>
> I cannot see that getting rid of the bridge would be that difficult, 
particularly if it is not for many modules.   The real pain is living in 
world where there is so much holding everything back to the older version.  
Again, let me declare the self interest here.  We have development teams 
and projects we wish to move to python3.  A bridge like this would allow us 
to move now, and that would allow pressure on the module providers.  
Stopping a step like this on the basis it stops people like us moving to 
python3 just seems counter productive if the goal is to move to python3.  

>
> > This is an important debate to have.  I suggest if the answer to 'can a 
> > specification be created that will significantly accelerate migration?', 
> > then it is worth the effort to deliver this. 
>
> I think you are making a classic mistake of technically-minded people: 
> looking for a technical solution to a social problem. 
>
> The problem with Python 3 uptake is not a technical problem, or at least 
> not *only* a technical problem. It is mostly a social problem: the costs 
> of migrating are greater than the benefits. Python 3 is great, but for 
> most people Python 2 does the job. 
>
> I am a person experiencing the problem.  And our experience is that our 
core developers use code supplied by a third party who does not produce a 
python3 version. So the core has to be done in python2.  The code produced 
by these core developers is then supplied to other companies - forcing them 
to stay in python2.  It is dependency hell.  One module anywhere down the 
chain that cannot be moved and the whole project is forced to stay 
python2.   
 

> Another social problem is that the minority of library authors who 
> haven't migrated haven't done so because they personally don't care 
> about Python 3, or they don't have the time or money to spend on it. 
> Adding a bridge doesn't suddenly give them more time or money. 
>

Agreed.  But in the real world, an example of a major dependency requiring 
python2, web2py,  states they will be motivated to python3 when the users 
of their framework move.   But this policy of forcing all dependencies to 
move before the end users move means this will never happen.  What it does 
mean is many of their customers can move.  And it is even possible for them 
to ask their holdout customers to either stay with the old version or move, 
since these customers are no longer held back by other dependencies.

For application developers, the social problem is less because of a few 

> libraries and more to do with the supported standard Python on their 
> system. That will change once Linux distros start providing 3 as their 
> standard system Python; out of the big distros, Fedora is about to do 
> so, so things will start changing soon. 
>
> But perhaps the biggest reason for slow uptake of Python 3 with existing 
> projects is that nobody wants to spend money just to migrate to 3 for 
> the sake of migrating to 3. "Written in Python 3" is not a selling 
> point. Even the availability of security updates is not much of a 
> selling point, although it will become more important when Python 2.7 is 
> end-of-lifed. The reality is that many apps work perfectly well now 
> using Python 2.7, or 2.6, or even 2.4, and they'll continue to work well 
> in five or ten years so there's no real need to update. At the last US 
> PyCon, there was even one fellow still using Python 1.5. It works, he 
> didn't need security updates, so why migrate? 
>
> Perhaps the biggest reason .....  Why not ask? Well, python.org conducted 
a survey as of January 2014 an answer given was that 60% of developers were 
held back from moving projects because of dependencies. This is a big 
number and it is not just guessing but real figures.

*This is not a problem to be solved.* It's okay for people to stick with 
> an old version of Python. There's no rule that says Python has failed if 
> some developers stick to old versions. 
>
> OK. So you are telling us forget moving our developers to python3 until 
every last dependency has moved. And we should also force the people who in 
turn use our code to stay with python2 and it is not a problem.

>
> > As things stand, the popularity of Python has dropped by most measures 
> > since the release of Python 3. Improve the language and lose market 
> share?   
> > I suggest that is all about the pain of the transition. 
>
> That sounds remarkably like FUD to me. Where is your evidence that the 
> popularity of Python has fallen? 
>
> Personally, I think all the "language popularity" measures are dubious, 
> or at least need to be treated with considerable skepticism. Take this 
> one: 
>
> http://langpop.com/ 
>
> which lists BrainF--- in the 40 most popular languages. Sure it is. 
>
> The PyPL popularity index here: 
>
>
> https://sites.google.com/site/pydatalog/pypl/PyPL-PopularitY-of-Programming-Language<https://www.google.com/url?q=https%3A%2F%2Fsites.google.com%2Fsite%2Fpydatalog%2Fpypl%2FPyPL-PopularitY-of-Programming-Language&sa=D&sntz=1&usg=AFQjCNEJ__tv-fiB_BTXtt3qu6Y_UD49kg> 
>
> Thanks for that- useful link!  Please remember I am the converted, so it 
helps to have arguments to push back with when others push tiobe in front 
of me.
 

> shows Python as increasing in popularity. In contrast, TIOBE shows 
> Python as losing popularity -- but so did Java, PHP, C++ and Ruby, and 
> none of them have a major version change. (TIOBE also shows that despite 
> a large number, possibly majority, of VB 6 developers boycotting VB.Net, 
> VB.Net has just entered the top 10 most popular languages.) 
>
> And then there's CodeEval, which for the third year in a row has found 
> that Python is more popular than Java, C++, C, PHP and Javascript. In 
> fact more popular than Java, C and PHP *together*. 
>
> http://blog.codeeval.com/codeevalblog/2014<http://www.google.com/url?q=http%3A%2F%2Fblog.codeeval.com%2Fcodeevalblog%2F2014&sa=D&sntz=1&usg=AFQjCNEcIR2Zkg32DfKkhns8eq7KVgFJ1Q> 
>
> Perhaps overstated, but it helps to have sources like this.   It does say 
coding tests and challenges.  I live in a country where school children 
have one national coding contest/challenge, and it is in python.  (And in 
python 3, no dependencies :) )

Bless their little white socks, but methinks their methodology is 
> perhaps a tad flawed. 
>
> I think it is interesting to compare and contrast these different ways 
> of measuring popularity, but then I'm a statistics wonk and I find even 
> flawed statistics interesting. Remember that there is no one true 
> definition of popularity, even if there were there is no accurate way of 
> measuring it, and all of the sites that claim to do so are actually 
> measuring subtley different things. 
>
> Most importantly, chasing popularity for its own sake is a waste of 
> time. What makes Python Python is not just the syntax and std lib, but 
> also the culture -- we're neither the stultifying corporate culture of 
> "Java shops", nor the wild-west anything goes of cowboy PHP developers, 
> but something unique. (As are other languages, of course. Perl culture 
> is not Haskell culture is not Erlang culture.) In part, people take up 
> languages because they like the culture. Python will never attract the 
> corporate suits who like the Java philosophy, nor will it attract the 
> cowbody web developers who like the PHP way of doing things. Should we 
> try to ape PHP or Java to attract the cowboys or suits? Or should we 
> continue to make Python the best language that *we* like, regardless of 
> what other people prefer? 
>

All agreed on the culture. But i suggest this is the biggest point.  
Stopping measures like a bridge that would allow many more end users to 
code in python3 just seems a negative culture.  Effectively you are telling 
us that our team should not move to python3.
 

>
> Untimately, apart from winning pissing contests on Slashdot and Reddit, 
> what does it matter if Python is third most popular or ninth most 
> popular? Its not a popularity context where the winner takes all. 
> According to TIOBE, the most popular language, C, has less than 20% 
> share. If you think Python is a failure because it is in position 8, 
> what does that make Javascript in position 9? 
>
>
>
> > You do not have to search hard to find comments to the effect 'nobody is 
> > using python3'.   Or 'even if you use python 3, do not use the new 
> features 
> > since you need to ensure code is compatible with both versions'. 
>
> You don't have to search very hard to find comments that Elvis was 
> kidnapped by aliens. 
>
> https://duckduckgo.com/html/?q=elvis+kidnapped+by+aliens 
>
> A belief which has rather more going for it than either of the two 
> comments you list above, since there's a miniscule, microscopic chance 
> that perhaps Elvis *was* kidnapped by aliens, whereas I know for a fact 
> beyond any doubt that some people are using Python 3 and using the new 
> features. 
>
> I think I already covered this one.  Again the point is that some people 
say it. We both know it is not literally true.  If i did not program in 
python3 I would not bother posting suggestions to try and allow more 
programming in python3. 

Ian


>
> -- 
> Steven 
> _______________________________________________ 
> Python-ideas mailing list 
> Python... at python.org <javascript:> 
> https://mail.python.org/mailman/listinfo/python-ideas 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>

Steven,


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/c5c0e15b/attachment-0001.html>

From ron3200 at gmail.com  Sat Mar  1 07:07:14 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 01 Mar 2014 00:07:14 -0600
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <20140301034611.GD28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
Message-ID: <lerte5$4bn$1@ger.gmane.org>



On 02/28/2014 09:46 PM, Steven D'Aprano wrote:
> On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote:
>> >
>> >Starting new thread because this bike has a different shape and color.
>> >
>> >Yesterday I was thinking that just making the keyword lambda assignable
>> >like True, False, and None, would be enough.
> You can't assign to True, False or None. (You can assign to True and
> False in Python 2, but shouldn't.)

I meant it the other way around.

For example... def is not assignable to anything..

      D = def     # won't work

      T = True    # works
      F = False
      N = None

      L = lambda  # won't work

lambda isn't an object like True, False, and None.

> [...]
>> >This morning I thought we could have in a functions definition something,
>> >like "*", and "**", to take an expression.  Similar to Nicks idea with =:,
>> >but more general.
>> >
>> >The idea is to have "***" used in def mean to take "any" call expression
>> >and not evaluate it until *** is used on it.
> [...]
>> >A function call that captures an expression may be tricky to do. Here's one
>> >approach that requires sugar when a function defined with "***" is called.

> I think it would be useful to have a way to delay execution of an
> expression, that is to say, have a way to capture an expression for
> later evaluation, something more lightweight than a function, but I
> don't think that limiting it to inside function calls is the right
> approach.

The expressing part isn't limited to inside function calls.. Or wouldn't be 
if it was a doable idea.

> Something perhaps like a thunk might be appropriate? We can*almost*  do
> that now, since Python has a compile function:
>
> thunk = compile("x + 3", "", "eval")
> # much later
> eval(thunk)


     def thunk(***expr):
         return expr

     def do thunks(*args):
         for expr in args:
             ***expr

     start = t = time()
     update_timer = thunk(t = time())
     show_timer = thunk(print(t-start))
     show_status = thunk(do_thunks(upate_timer, show_timer))


Then as you do things, possibly in different functions as well.

     ...
     ***show_status        # update t, and prints elapsed time.
     ...
     ***show_status
     ...
     ***show_status
     ...


Yes, it could be done with lambda too.




Here's where it differs...

So if we have this, where obj is nested expressions.

      While 1:
          try:
              obj = ***obj
          except TypeError:
              break

Then the result of the obj expression, could be a callable without any 
conflict.  Only delayed expressions would be expressed.

This was one of the properties I was looking for.


> Some problems with that approach:
>
> - You have to write the expression as a string, which means you lose any
> possibility of syntax highlighting.
>
> - The temptation is to pass some arbitrary untrusted string, which leads
> to serious security implications. The argument here should be limited
> to an actual expression, not a string containing an expression which
> might have come from who knows where.
>
> - It should be as lightweight as possible. The actual compilation of the
> expression should occur at compile-time, not run-time. That implies some
> sort of syntax for making thunks, rather than a function call.
>
> - Likewise actually evaluating the thunk should be really lightweight,
> which may rule out a function call to eval.
>
> - How should scoping work? I can see use-cases for flat scoping, static
> scoping, dynamic scoping, and the ability to optionally provide custom
> globals and locals, but I have no idea how practical any of them would
> be or what syntax they should use.

That's where the idea I mentioned in another response to this thread comes 
in.  Separating the namespace constructing from the code part of a function 
in a well defined way.

If that can be done... also pie in the sky. Then we could also evaluate an 
expression in a previously defined namespace.  It's hypothetical, so supply 
your own syntax for it.

The compiler will still compile the code, so most of the work is still done 
at compile time.  But we have these useful pieces we can take apart and put 
back together again.  (without calling eval, or exec.)

BTW, you can do that now, but it's very hard to get right.

      help(type(lambda:1))


> All of this is pie-in-the-sky at the moment, and not a serious proposal
> for Python 3.5.

Agree.
   Cheers,
      Ron




From rosuav at gmail.com  Sat Mar  1 07:43:50 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Mar 2014 17:43:50 +1100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
Message-ID: <CAPTjJmo2i3uZ=x_LyiB+f2vijAPRS3feA7eh1GosgX26s=wE7Q@mail.gmail.com>

On Sat, Mar 1, 2014 at 4:53 PM, ian o <ian.team.python at gmail.com> wrote:
> A bridge like this would allow us to move now, and that would allow pressure
> on the module providers.

Think about what the bridge really means. It means that you move now,
yes, but it also means that the existing module is working fine. That
means there is *no* pressure on the module provider. The pain of
maintaining the bridge is all yours. You want to put pressure on the
module provider? *Don't* have the bridge. Then you have an incentive
to get that module ported, because then you could migrate. Maybe that
means you lean on the developers; maybe that means you run the code
through 2to3 and then start dealing with test suite failures, and then
submit the code back to them and say "Here's Python 3.3 support, these
are the downsides, will you accept it?". That's the sort of pressure
that's likely to work. Saying "Hey, we can use your module in Python 3
now, thanks to a Python-in-C-in-Python bridge" will evoke the response
"Okay fine, no problem then!".

ChrisA

From ncoghlan at gmail.com  Sat Mar  1 07:55:37 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 1 Mar 2014 16:55:37 +1000
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
Message-ID: <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>

On 1 March 2014 15:53, ian o <ian.team.python at gmail.com> wrote:
> Steven,
>> *This is not a problem to be solved.* It's okay for people to stick with
>> an old version of Python. There's no rule that says Python has failed if
>> some developers stick to old versions.
>>
> OK. So you are telling us forget moving our developers to python3 until
> every last dependency has moved. And we should also force the people who in
> turn use our code to stay with python2 and it is not a problem.

Ian, please keep in mind that you are asking people to do extra work
for you *for free*, after you have already been benefiting for years
from work that we already made available to you free of charge. The
latter part is OK, and entirely the way non-commercial open source
works, but it *does* mean that you *don't* get to tell the core
development team our priorities are wrong when you haven't paid us a
cent. If that's the kind of relationship you want, pay a commercial
vendor to listen to your concerns - mediating between customers and
the community one of the services commercial open source vendors
provide (and the best ones will then contribute upstream development
effort on your behalf).

By contrast,community oriented open source operates on a time-based
economy - the only ways for people to get things are done to dedicate
their own time, inspire others to dedicate their time, or else to pay
developers (either directly or indirectly through a vendor) to spend
*their* time on projects of interest to those providing the funds.

The CPython core development team has made it clear we don't want to
support running Python 2 and 3 code in the same process because doing
so is *hard* (and perhaps even impossible without completely
redesigning the interpreter) and certainly *not fun*. The inspiration
based approach is unlikely to work in this case (because we've already
considered the possibility and dismissed it as impractical), but that
still leaves open the possibility of finding someone else that is
interested in attempting to prove us wrong, as well as people finding
a way to pay someone to make it happen.

However, in both cases, keep in mind that the people *most* familiar
with the CPython code base think it's a bad idea that probably won't
work. Instead, with the aid of many other members of the community,
we've created a source based migration approach that involves taking
advantage of the large common subset of Python 2 & 3.

If commercial operations higher in the stack would like to migrate to
Python 3, but are relying on dependencies that the community has
provided for free, then it's time to think seriously about *investing*
in the community and helping those projects to migrate (either by
contributing developer time directly or by contributing funds to the
existing developers for those projects). Note that the Python Software
Foundation is able to help mediate requests for assistance with
getting dependencies ported.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar  1 08:26:23 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 1 Mar 2014 17:26:23 +1000
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <20140301034611.GD28804@ando>
References: <leqgah$b0o$1@ger.gmane.org>
	<20140301034611.GD28804@ando>
Message-ID: <CADiSq7cqGe9uS5YxjTTAXqVoQdc+MVKWpKJ+YPfu+10CnCyBfg@mail.gmail.com>

On 1 March 2014 13:46, Steven D'Aprano <steve at pearwood.info> wrote:
>
> Something perhaps like a thunk might be appropriate? We can *almost* do
> that now, since Python has a compile function:
>
> thunk = compile("x + 3", "", "eval")
> # much later
> eval(thunk)

FWIW, I think this is why PEP 312 (simple implicit lambda) garnered
any interest at all: sure, you can't pass such a lambda any arguments,
but it works fine as a *closure*.

It's also potentially worth trawling the python-ideas archives for the
various discussions about deferred and delayed expressions - they're
generally all variants of the same basic idea, a less esoteric, easier
to learn way to handle one-shot callbacks and other forms of lazy
expression evaluation. The "lambda" keyword in the current syntax ends
up being a distraction rather than an aid to understanding.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Sat Mar  1 08:44:49 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 28 Feb 2014 23:44:49 -0800
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
Message-ID: <E21B635C-F8C8-4B7D-BD84-6FAF2C6575DD@yahoo.com>

On Feb 28, 2014, at 21:53, ian o <ian.team.python at gmail.com> wrote:

> Agreed.  But in the real world, an example of a major dependency requiring python2, web2py,  states they will be motivated to python3 when the users of their framework move.  

This is an interesting choice for an example.

web2py isn't just a library, it's a framework. Its main purpose is to run your code in the right situations. Which is exactly the kind of thing your bridge cannot handle, for the reasons you already gave. And I don't think this is a coincidence. The same thing is true of many of the most prominent packages that haven't been ported yet--Twisted, Paste, gevent, supervisor, etc.

On top of that, web2py is all about passing Internet strings around. The same thing that makes it difficult to port--the bytes/Unicode distinction--would make it just as hard to use through the bridge.

Anyway, if you're interesting in pursuing this, obviously nobody is going to stop you from doing it and putting it on PyPI. And if there's as much interest in the idea as you think, you should start getting outside contributions as soon as you have a workable beta (or you might even be able to get funded development before that). And if it became popular enough, you could come back to the core dev team and say, "Like it or not, there are tons of major projects relying on my bridge, and the consequences aren't nearly as gloomy as you suspected, so what about putting it in the stdlib?"

From ian.team.python at gmail.com  Sat Mar  1 09:03:05 2014
From: ian.team.python at gmail.com (ian o)
Date: Sat, 1 Mar 2014 00:03:05 -0800 (PST)
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
Message-ID: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>




> Ian, please keep in mind that you are asking people to do extra work 
> for you *for free*, after you have already been benefiting for years 
> from work that we already made available to you free of charge. The 
> latter part is OK, and entirely the way non-commercial open source 
> works, but it *does* mean that you *don't* get to tell the core 
> development team our priorities are wrong when you haven't paid us a 
> cent.


Nic,

Why are you so aggressive?  
No one is trying to 'tell' people to do something.

I was proposing an idea here. I thought it was a relevant useful idea, so i 
have been arguing the case that idea is useful. Others argue the case that 
it is not useful and to me this is a constructive way to share ideas, hear 
other points of view so they can be considered.

For commercial purposes you have managed to convince me I should cease to 
lobby for python to be used in the projects where I assist. Obviously 
commercial use is clearly not desired by the community, that is the first 
lesson i have learned.  As I am not paid to champion python in the work 
environment, having been enlightened that python community does not desire 
commercial usage, I will cease.  

I came this forum hoping to contribute.  I do not get paid to post here, 
nor will python being used in projects where i am involved generate a 
financial outcome for me.  I simply feel a passion for the language and 
have been supporting the use of python in education and it helps push case 
for students learning python when there is also commercial usage.

Is educational usage also to be discouraged?

So now I am being told that not only are commercial users not to be 
encouraged, but also opinions and ideas of those who have been involved in 
commercial projects are not welcome.

Disappointing.  If a welcome as a result of a debate it was agreed that the 
approach I feel will help both library module developers and end users and 
was worth supporting but the community then the decision question would 
come as to who would donate time to implementation.  While i would get no 
commercial benefit and do not have a lot time i would have tried to help.  
I think the idea has merit.  But what is the point?  It seems even debate 
on desirability of the idea in your opinion is not to be encouraged.  From 
others I get the sense of community.  But from you I get the sense of a 
closed club.

Ian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/a3d261fd/attachment.html>

From rosuav at gmail.com  Sat Mar  1 09:05:52 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Mar 2014 19:05:52 +1100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
 <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
Message-ID: <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com>

On Sat, Mar 1, 2014 at 7:03 PM, ian o <ian.team.python at gmail.com> wrote:
> Why are you so aggressive?
> No one is trying to 'tell' people to do something.
>
> I was proposing an idea here.

If your idea is accepted, someone has to implement it :)

That's work, and a lot of it. So unless you're volunteering to
actually write this thunker/bridge/shim/etc, you're asking someone
else to.

ChrisA

From abarnert at yahoo.com  Sat Mar  1 09:29:02 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 1 Mar 2014 00:29:02 -0800
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <20140301034611.GD28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
Message-ID: <807BB590-DE68-4E3C-9DC3-F37335DFBFAD@yahoo.com>

On Feb 28, 2014, at 19:46, Steven D'Aprano <steve at pearwood.info> wrote:

> Something perhaps like a thunk might be appropriate? We can *almost* do 
> that now, since Python has a compile function:

To save Haoyi Li the trouble of saying it: we _can_ do this now, since Python has import hooks, so you can use MacroPy to quote an expression, giving you an AST to evaluate later, or automatically wrapping it up in a function (like quote and function, respectively, in Lisp). There's also an in-between option of compiling it to a code object but not wrapping that in a function object.

There's one other crazy option for representing quoted expressions/thunks: as lazy futures. This is how Alice does it, and it's also implicitly what dataflow languages like Oz are doing (every variable is a lazy future). In Python, this would really just be a wrapper around a function call, so I don't think it would but anything.

Which one of those do people want? I'm not sure. Since you did a great job listing the problems of the string-quoting solution, we can compare the options for each one.

> - You have to write the expression as a string, which means you lose any 
> possibility of syntax highlighting.

Obviously not a problem here.

> - The temptation is to pass some arbitrary untrusted string, which leads 
> to serious security implications. The argument here should be limited 
> to an actual expression, not a string containing an expression which 
> might have come from who knows where.

Again, not a problem.

> - It should be as lightweight as possible. The actual compilation of the 
> expression should occur at compile-time, not run-time. That implies some 
> sort of syntax for making thunks, rather than a function call.

Not a problem for the function or code versions.

For the AST version, you're doing part of the compilation at compile-time, and the rest at runtime.

> - Likewise actually evaluating the thunk should be really lightweight, 
> which may rule out a function call to eval.

A function is obviously no better or worse than using lambda today.

A code object has to be evaluated by passing it to eval, or wrapping it in a function and calling it. I suspect the former may be lighter weight than calling a function, but I really don't know. The latter, or the other hand, is obviously heavier than calling a function, but not by that much.

An AST has to be compiled to a code object, after which you do the same as the above. Obviously this is heavier than not having to compile.

> - How should scoping work? I can see use-cases for flat scoping, static 
> scoping, dynamic scoping, and the ability to optionally provide custom 
> globals and locals, but I have no idea how practical any of them would 
> be or what syntax they should use.

This, I think, is the big question.

A function is clearly a normal, static-scoped closure.

An AST or code object, you could do almost any form of scoping you want _except_ static, either by building a function around it or by calling eval on it. (You can do additional tricks with an AST, but I don't think most programs would want to. For statements, this could be useful for optional hygienic variables, but for expressions that isn't an issue.)

If we want static scoping, we really need functions. At least we need some kind of object that has some form of code plus a closure mechanism--and that's pretty much all functions are. On the other hand, if we want dynamic, flat, or customizable scoping, we need something we can either build a function object out of dynamically, or evaluate dynamically--and that's pretty much what code objects are. 


From ian.team.python at gmail.com  Sat Mar  1 09:40:58 2014
From: ian.team.python at gmail.com (ian o)
Date: Sat, 1 Mar 2014 00:40:58 -0800 (PST)
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
 <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
 <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com>
Message-ID: <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com>

Chris,

First the idea has to be accepted as desirable, then feasible with an 
agreed specification. All to no avail unless someone can then implement it.

I would volunteer to assist but i assume there have to be enough others to 
volunteer as well if it was any help.

But there is no intent to tell people to do something.  There has to be 
sufficient enthusiasm for anyone involved to actually want to do it.

But I have been told that those who would benefit are people that there is 
no desire to help.  My suggestion that it would also be helpful to people 
there is a desire to help seems to have been met with the response 'we do 
not want to hear this'.

I guess I expected a more open community.  I have learned a lot, but not 
what I expected.  I am still not sure what the criteria is to actually be 
welcome here.

I have very much appreciated your comments though.

Ian


On Saturday, March 1, 2014 7:05:52 PM UTC+11, Chris Angelico wrote:
>
> On Sat, Mar 1, 2014 at 7:03 PM, ian o <ian.tea... at gmail.com <javascript:>> 
> wrote: 
> > Why are you so aggressive? 
> > No one is trying to 'tell' people to do something. 
> > 
> > I was proposing an idea here. 
>
> If your idea is accepted, someone has to implement it :) 
>
> That's work, and a lot of it. So unless you're volunteering to 
> actually write this thunker/bridge/shim/etc, you're asking someone 
> else to. 
>
> ChrisA 
> _______________________________________________ 
> Python-ideas mailing list 
> Python... at python.org <javascript:> 
> https://mail.python.org/mailman/listinfo/python-ideas 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/99b65545/attachment-0001.html>

From g.brandl at gmx.net  Sat Mar  1 10:11:59 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 01 Mar 2014 10:11:59 +0100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
 <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
 <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com>
 <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com>
Message-ID: <les86i$37r$1@ger.gmane.org>

Am 01.03.2014 09:40, schrieb ian o:
> Chris,
> 
> First the idea has to be accepted as desirable, then feasible with an agreed
> specification. All to no avail unless someone can then implement it.
> 
> I would volunteer to assist but i assume there have to be enough others to
> volunteer as well if it was any help.
> 
> But there is no intent to tell people to do something.  There has to be
> sufficient enthusiasm for anyone involved to actually want to do it.
> 
> But I have been told that those who would benefit are people that there is no
> desire to help.  My suggestion that it would also be helpful to people there is
> a desire to help seems to have been met with the response 'we do not want to
> hear this'.
> 
> I guess I expected a more open community.  I have learned a lot, but not what I
> expected.  I am still not sure what the criteria is to actually be welcome here.

Hi Ian,

I'm sorry you feel like this.  It's definitely not the intention of this list
to exclude people -- just look at the wild ideas being discussed all the time...

The Python 2/3 topic is simply a "sore point" for many of us, having discussed
it time and again, often times with quite aggressive people (with the "why have
you ruined my Python" mindset).  Therefore we tend to become a little
categorical faster than with other topics, even to people like you who aren't
being aggressive.  I hope you can forgive Nick, who has put tons of work into
the excellent 2 vs 3 FAQ, being a bit thin-skinned :)

In other words: Welcome to the Python community - don't let the 16-ton weight
scare you off!

cheers,
Georg


From ncoghlan at gmail.com  Sat Mar  1 10:46:57 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 1 Mar 2014 19:46:57 +1000
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <les86i$37r$1@ger.gmane.org>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
 <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
 <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com>
 <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com>
 <les86i$37r$1@ger.gmane.org>
Message-ID: <CADiSq7dTeUgv-b48YpEkwEWBC8e+50UyWFD8ucPLW=_dyOtUAA@mail.gmail.com>

On 1 March 2014 19:11, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 01.03.2014 09:40, schrieb ian o:
>> I guess I expected a more open community.  I have learned a lot, but not what I
>> expected.  I am still not sure what the criteria is to actually be welcome here.
>
> Hi Ian,
>
> I'm sorry you feel like this.  It's definitely not the intention of this list
> to exclude people -- just look at the wild ideas being discussed all the time...
>
> The Python 2/3 topic is simply a "sore point" for many of us, having discussed
> it time and again, often times with quite aggressive people (with the "why have
> you ruined my Python" mindset).  Therefore we tend to become a little
> categorical faster than with other topics, even to people like you who aren't
> being aggressive.  I hope you can forgive Nick, who has put tons of work into
> the excellent 2 vs 3 FAQ, being a bit thin-skinned :)

Aye, I hit the "This is holding back Python!" in Ian's original post,
and subsequently interpreted the rest of the post (and thread) in an
extremely negative light.

We get a *lot* of people making such accusations (understandably so,
even though those accusations aren't adequately backed up by the
available data), so I have a very low tolerance at the moment for
anything that even hints at commercial operations complaining about
the quality of open source work they aren't funding (or otherwise
contributing to), but are relying on anyway.

However, Ian, It sounds like that wasn't your intent, so my apologies
(once again) for taking it that way.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stefan_ml at behnel.de  Sat Mar  1 11:51:23 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 01 Mar 2014 11:51:23 +0100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
Message-ID: <lese2v$s63$1@ger.gmane.org>

ian o, 01.03.2014 04:09:
> We have 3 teams working server systems currently with python as the 
> implementation model.  All work is currently in python 2.x and with a 
> solution to this the work could immediately move to 3.x.

You are mixing things up here. If you are developing application code, you
don't really have a problem. Either you stay with 2.x or you switch to 3.x.
You may even decide to stay in Py2 with some of your applications and
migrate the others. Or migrate one after the other, over months or years.
You really don't have to break your running systems, and definitely not all
of them at once.

It's entirely your choice, and doing the switch (if you want to) really
isn't all that hard. Nick pointed you to a couple of resources that can
help you in the progress. Many, many others have done it before, and have
found that breaking all bridges after you have passed over them is a
perfectly reasonable and efficient way to migrate, *specifically* for
application code.

The real problem, and that's what Nick was referring to, is writing and
maintaining libraries that others depend on, because these others may use
Python 2.x (and may not even want to switch to Py3), or they may use Python
3.x (and do not care about Python 2.x). But the library authors do not
control the application code. Therefore, to cater for both user groups,
library maintainers currently have to write code that works on both
platforms, which is annoying and time consuming.

Since you seem to be interested in migrating your application code in order
to take advantage of Python 3, my advice is to (ta-da!) start migrating
your code. And when that's done, stop looking back.

Any "solution" that involves running a mix of Py2 and Py3 inside of one
application is just screaming for endless trouble. Don't forget that
"temporary solutions" tend to be the most durable in practice.

Stefan



From g.rodola at gmail.com  Sat Mar  1 12:33:02 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 1 Mar 2014 12:33:02 +0100
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
 <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com>
Message-ID: <CAFYqXL-FyZ=SYk=oUYJ7_Fj32nF6qD-07Gn46ncuVoWLUAKsQg@mail.gmail.com>

On Sat, Mar 1, 2014 at 6:09 AM, Andrew Barnert <abarnert at yahoo.com> wrote:

> For that matter, have you actually checked whether those libraries really
> are blocking you? I've lost count of the number of people who've claimed
> they can't switch to Python 3 because of some library that's been ported
> since 2010 or that has a 3.x-compatible drop-in replacement.
>

The problem you mention here is absolutely *real*.
It's not only a matter of whether all your deps have currently been ported
already.
Software evolves over time and you can also remain stuck later, when you
suddenly decide you need lib X and lib X hasn't been ported yet (uh-oh!
what do I do now?).
And please, note that *as of now* lib X includes names such as Twisted and
Paramiko, which means hundreds of thousands of users which are stuck as we
speak.
How can a company decide to make the switch starting from such an
assumption? The assumption that lib X or Y might not be ported soon, or
even *ever*?
IMHO as long as we won't get close to a 100% of libs being ported the
current situation is not likely to change.
FWIW here's some stats (inspired by recent Brett Cannon's post about
openstack): http://stackoverflow.com/a/22113627/376587

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/44d44660/attachment.html>

From breamoreboy at yahoo.co.uk  Sat Mar  1 16:48:48 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 01 Mar 2014 15:48:48 +0000
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
Message-ID: <lesvfq$557$1@ger.gmane.org>

On 27/02/2014 23:26, ian o wrote:

I have read every entry so far on this thread and have been forced to 
the conclusion that this is A Bridge Too Far.  Sorry :(

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From Steve.Dower at microsoft.com  Sat Mar  1 16:54:26 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 1 Mar 2014 15:54:26 +0000
Subject: [Python-ideas] Update the required C compiler for Windows to a
 supported version.
In-Reply-To: <CAH-ZgAeEB4+fEwbxHno-cbJeQypMRQX_b9pJMQvNyzYDsoqt2Q@mail.gmail.com>
References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>,
 <CAH-ZgAeEB4+fEwbxHno-cbJeQypMRQX_b9pJMQvNyzYDsoqt2Q@mail.gmail.com>
Message-ID: <682219fab07d40a5acee0eaf50b7992c@BLUPR03MB389.namprd03.prod.outlook.com>

Not at my desk, so I can only get you started (didn't want to keep you waiting until Monday). It'll look something like this:

IF %1 eq x86 CALL bin\vcvars32.bat %*
IF %1 eq amd64 CALL bin\vcvars64.bat %*
(similar for x86_amd64)

The file should go at C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat

If you want to redistribute something that will work for others, it's possible to monkey patch distutils from your setup.py. I can't remember the specifics right now, so I can't help with that until I'm back at work. I think that's the best long term solution for 2.7 packages that want to build on Windows.

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Vernon D. Cole<mailto:vernondcole at gmail.com>
Sent: ?3/?1/?2014 2:24
To: Steve Dower<mailto:Steve.Dower at microsoft.com>
Cc: Stephen J. Turnbull<mailto:stephen at xemacs.org>; python-ideas<mailto:python-ideas at python.org>
Subject: Re: [Python-ideas] Update the required C compiler for Windows to a supported version.

Sorry to resurrect this thread after so long a time, but I am starting to work on a project where I need this again...

Any chance that I can get a copy (or a link to) " the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one" ?
It's only simple to add one if you know what changes to make.
--
VC



On Wed, Nov 27, 2013 at 7:24 PM, Steve Dower <Steve.Dower at microsoft.com<mailto:Steve.Dower at microsoft.com>> wrote:
Stephen J. Turnbull wrote:
> Vernon D. Cole writes:
>
>> I cannot compile a Python extension module with any Microsoft compiler
>> I can obtain.
>
> Your pain is understood, but it's not simple to address it.

FWIW, I'm working on making the compiler easily obtainable. The VS 2008 link that was posted is unofficial, and could theoretically disappear at any time (I'm not in control of that), but the Windows SDK for Windows 7 and .NET 3.5 SP1 (http://www.microsoft.com/en-us/download/details.aspx?id=3138) should be around for as long as Windows 7 is supported. The correct compiler (VC9) is included in this SDK, but unfortunately does not install the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one that will switch on %1 and call the correct vcvars(86|64|...).bat.)

The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.

> The basic problem is that the ABI changes.  Therefore it's going to require
> a complete new set of *all* C extensions for Windows, and the duplication
> of download links for all those extensions from quite a few different vendors
> is likely to confuse a lot of users.

Specifically, the CRT changes. The CRT is an interesting mess of data structures that are exposed in header files, which means while you can have multiple CRTs loaded, they cannot touch each other's data structures at all or things will go bad/crash, and there's no nice way to set it up to avoid this (my colleague who currently owns MSVCRT suggested a not-very-nice way to do it, but I don't think it's going to be reliable enough). Python's stable ABI helps, but does not solve this problem.

The file APIs are the worst culprits. The layout of FILE* objects can and do change between CRT versions, and file descriptors are simply indices into an array of these objects that is exposed through macros rather than function calls. As a result, you cannot mix either FILE pointers or file descriptors between CRTs. The only safe option is to build with the matching CRT, and for MSVCRT, this means with the matching compiler. It's unfortunate, and the responsible teams are well aware of the limitation, but it's history at this point, so we have no choice but to work with it.

Cheers,
Steve

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/a8bc2053/attachment.html>

From oscar.j.benjamin at gmail.com  Sat Mar  1 18:34:34 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 1 Mar 2014 17:34:34 +0000
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
Message-ID: <CAHVvXxQE_N4Nj9Qz=ezhM7Cpws7v+OWJe4uFh=46JUwM1QLF=A@mail.gmail.com>

On 1 March 2014 03:09, ian o <ian.team.python at gmail.com> wrote:
>
>>
>> And we don't actually mind all that much if applications don't migrate
>> in the near term - Python 2 will have commercial support available
>> well past 2020. (The developers of *those applications* might mind,
>> though, just as anyone maintaining Python 2.4 compatibility for the
>> benefit of RHEL/CentOS 5 typically isn't happy about it)
>
> Your response is most dissapointing, and I would hope does not represent the
> community overall.
>
> We have 3 teams working server systems currently with python as the
> implementation model.  All work is currently in python 2.x and with a
> solution to this the work could immediately move to 3.x.
>
> However, you state that our situation, and that of the other 60% of python
> programmers as at end January 2014 who are constrained from using python 3
> by python 2 dependencies:  "we don't mind about those developers"

Hi Ian,

In the past I've sometimes seen Nick being slightly careless with his
words with the result that someone misunderstands him and gets upset.
That's not what's happening here though. I'm not sure if you're
wilfully misunderstanding him or just unable to see this from his
perspective. Nick clearly did not say that he (or anyone else) doesn't
care about any particular group of *developers*.

What he meant is that if you as an application author have decided
that sticking with Python 2 is the right thing to do right now then he
won't try to argue with you. If your application is currently working
and has a dependency on web2py and it's not worth your time/money to
port web2py yourselves then sticking with Python 2 may be the right
decision for you.

OTOH if you were the author of web2py and said "I don't want to port
web2py to Python 3" Nick would mind. He would want to know why you
didn't want to port to Python 3 and whether or not there was something
(reasonable) that the core Python devs could do to help. This is
because if a library is not ported then that has a knock-on effect on
application authors (such as you) who would like to port an existing
application or write a new application for Python 3.


Oscar

From ncoghlan at gmail.com  Sun Mar  2 01:39:17 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 2 Mar 2014 10:39:17 +1000
Subject: [Python-ideas] Update the required C compiler for Windows to a
 supported version.
In-Reply-To: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>
References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>
Message-ID: <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com>

On 28 November 2013 04:24, Steve Dower <Steve.Dower at microsoft.com> wrote:
> The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.

The main challenge we face on that front is the fact that Visual
Studio 2013 is Windows 7+ only, so we'd potentially be causing
problems for people still developing on XP or Vista if we update.

Windows XP goes EOL next month, so that will be dropped for 3.5 next
year in accordance with PEP 11.

However, Vista is still in extended support until 2017 - under the
current PEP 11 guidelines, that means it will be supported for at
least 3.5, and likely 3.6 as well. If we update the required compiler
we'd end up in a situation where CPython 3.5 nominally supports Vista,
but Vista users would still need at least one Windows 7 machine to
install the toolchain to create compatible C extensions. That may be
OK given the number of people that jumped straight from XP to Windows
7, but it's still a consideration that would need to be taken into
account.

Longer term, we'd love to add a build farm to PyPI to take the
drudgery out of creating pre-compiled wheel files (and MSI installers
for Windows), and that would potentially address this problem as well
(since the build farm would include the requisite Windows 7 images).
However, I currently think it would be rather optimistic to assume
that we'll have that in place by the time 3.5 rolls around (there's a
lot of other things that need to be addressed before a build farm will
make it to the top of the distutils-sig todo list).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From shibturn at gmail.com  Sun Mar  2 12:42:52 2014
From: shibturn at gmail.com (Richard Oudkerk)
Date: Sun, 02 Mar 2014 11:42:52 +0000
Subject: [Python-ideas] Update the required C compiler for Windows to a
 supported version.
In-Reply-To: <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com>
References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>
 <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com>
Message-ID: <5313193C.309@gmail.com>

On 02/03/2014 12:39 am, Nick Coghlan wrote:
> Windows XP goes EOL next month, so that will be dropped for 3.5 next
> year in accordance with PEP 11.
>
> However, Vista is still in extended support until 2017 - under the
> current PEP 11 guidelines, that means it will be supported for at
> least 3.5, and likely 3.6 as well.
BTW, Windows Server 2003 has extended support until 14 July 2015. Will 
it be supported in Python 3.5?

-- Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140302/4a6dae32/attachment.html>

From stephen at xemacs.org  Sun Mar  2 15:55:43 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 02 Mar 2014 23:55:43 +0900
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <lese2v$s63$1@ger.gmane.org>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com>
 <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com>
 <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com>
 <lese2v$s63$1@ger.gmane.org>
Message-ID: <87iorwsl8w.fsf@uwakimon.sk.tsukuba.ac.jp>

Stefan Behnel writes:

 > Since you seem to be interested in migrating your application code
 > in order to take advantage of Python 3, my advice is to (ta-da!) 
 > start migrating your code. And when that's done, stop looking back.

I'm sure he's thinking that way for himself.  To be fair, though, as
he describes it, Ian is evidently not the boss in the commercial
projects he's talking about projects.  He needs to convince others
that Python 3 is a practical solution to their problems.

This makes the idea of lobbying for Python a real enthusiasm drainer.
I'm still all for it, but we should recognize the social aspects as
well as the technical issues.

Unfortunately, I'm not sure what, if anything, we can do to help
support developers like Ian who can't impose Python 3 -- but rather
has to expend personal energy and reputation in lobbying.

From stephen at xemacs.org  Sun Mar  2 15:37:48 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 02 Mar 2014 23:37:48 +0900
Subject: [Python-ideas] A python bridge between versions
In-Reply-To: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com>
 <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com>
 <20140301040740.GF28804@ando>
 <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com>
 <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com>
 <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com>
Message-ID: <87k3ccsm2r.fsf@uwakimon.sk.tsukuba.ac.jp>

>>>>> Ian O. writes:
 > Nick Coghlan writes:

 >> Ian, please keep in mind that you are asking people to do extra work 
 >> for you *for free*, after you have already been benefiting for years 

 > Why are you so aggressive?  
 > No one is trying to 'tell' people to do something.

In Nick's defense, (1) he spends a *lot* of his time explaining why
Python development proceeds as it does, (2) "just proposing ideas"
sometimes is indeed a passive-aggressive ploy, (3) your timing is bad
due to the appearance of an occasional poster who does do exactly
that, and (4) Nick's probably harried with a major release in process
and PyCon coming up.

 > For commercial purposes you have managed to convince me I should
 > cease to lobby for python to be used in the projects where I
 > assist.

I think that's *exactly* the wrong lesson to learn, unless your
management doesn't understand that free software is an investment and
requires its users to spend resources to take advantage of it.  It's
just that the form of the resources required is often quite different
from those required to use proprietary software.  And of course it
differs from product to product even within open source.  So all
you've learned from Nick is that Python is unlikely to dramatically
reduce the resources required to migrate from Python 2 to Python 3
using technology like a "bridge".  "Everyone would like a pony, but we
don't have everyone's favorite pony."

OTOH, it remains the case that most new applications can be written in
Python 3 because there are good libraries to support them, although
you may have to give up on some frameworks that depend on libraries
that haven't been ported.  That may be, but is not necessarily, a
show-stopper.

Now you say "lobbying".  "Lobbying" implies effort and marshalling
resources.  Even if you strongly favor a particular framework that is
*apparently* blocked by an unported library, it's surely theoretically
possible that the shortest path to serving your company's software
needs is spend company resources to port that library[1] -- as well as
the personal resources to get the necessary support and resources from
management!

In practice, while I can't offhand give a case where a commercial
company has done that port internally, I know at least one
noncommercial project that is considering paying for a port of one of
its Python3 blockers.  I'm willing to bet you can find companies that
have done 2to3 ports of libraries considering that to be the "shortest
path to a product release", if you look around a bit.

 > Obviously commercial use is clearly not desired by the community,
 > that is the first lesson i have learned.  As I am not paid to
 > champion python in the work environment, having been enlightened
 > that python community does not desire commercial usage, I will
 > cease.

Are you aware how passive-aggressive your phrasing sounds?  I'm sure
you're well aware that not only are many Python developers involved in
commercial development, but all of Python from the license on up is
oriented toward encouragement of commercial use of Python.  And it's
been quite successful in commercial use.

It would be nice if the language improvements in Python 3 came at zero
cost.  They don't, and we all feel that pinch at one time or another.
But remember, innovations succeed not only by inventing completely new
ideas that nobody ever thought of before, but also by hurdling cost
barriers that others considered so high as to make known "interesting
in theory" ideas "commercially impractical".

I am not going to tell you that your cost barrier is lower than you
think -- you know your situation, I don't.  But I assure you that
others, with high but not insuperable cost barriers, have indeed made
that hurdle in porting libraries to Python 3, and I'm sure some are
taking on new challenges today.


Footnotes: 
[1]  Just as if it were a completely different language ("that Perl
module would be perfect if only it were in Python..." :-).


From robert.kern at gmail.com  Sun Mar  2 16:05:59 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 02 Mar 2014 15:05:59 +0000
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com>
Message-ID: <levhc8$h3f$1@ger.gmane.org>

On 2014-02-27 22:41, Guido van Rossum wrote:
> I haven't followed this recommendation for years. :-) Good riddance!

FWIW, PEP 8 still suggests the old form, though the example was fixed to conform 
to the new suggestion:

http://hg.python.org/peps/file/272debef6b77/pep-0008.txt#l548
http://hg.python.org/peps/rev/5efe00002b3e

> (However, the one about two spaces after a period stands. :-)

Alas!

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From guido at python.org  Sun Mar  2 18:29:45 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 2 Mar 2014 09:29:45 -0800
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <levhc8$h3f$1@ger.gmane.org>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com>
 <levhc8$h3f$1@ger.gmane.org>
Message-ID: <CAP7+vJJ52894BAQ02DgkymOwOOL8uHfb==szar_pZqkQMWEdRw@mail.gmail.com>

Fixed. Thanks for pointing this out!


On Sun, Mar 2, 2014 at 7:05 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On 2014-02-27 22:41, Guido van Rossum wrote:
>
>> I haven't followed this recommendation for years. :-) Good riddance!
>>
>
> FWIW, PEP 8 still suggests the old form, though the example was fixed to
> conform to the new suggestion:
>
> http://hg.python.org/peps/file/272debef6b77/pep-0008.txt#l548
> http://hg.python.org/peps/rev/5efe00002b3e
>
>
>  (However, the one about two spaces after a period stands. :-)
>>
>
> Alas!
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>   -- Umberto Eco
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140302/70356447/attachment-0001.html>

From harrismh777 at gmail.com  Mon Mar  3 16:41:40 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Mon, 3 Mar 2014 09:41:40 -0600
Subject: [Python-ideas] Python3.3 Decimal Library Released
Message-ID: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>

Greetings,
Python3.3 Decimal Library v0.3 is Released here:

https://code.google.com/p/pythondecimallibrary/

pdeclib.py is the decimal library, pilib.py is the PI library.

pdeclib.py  provides scientific and transcendental functions
for the C Accelerated Decimal module written by Stefan Krah. The
library is open source, GLPv3, comprised of two py files.

My idea for python is to two things really, 1) make floating point decimal
the default floating point type in python4.x, and 2) make
these functions ( pdeclib.py ) or equiv available in python4.x by
default.

Thank you for your consideration.

-- 
Kind regards,
Mark H. Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/9ba9cb5a/attachment.html>

From solipsis at pitrou.net  Mon Mar  3 16:58:02 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 03 Mar 2014 16:58:02 +0100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
Message-ID: <lf28ps$d7n$1@ger.gmane.org>


Hello Mark,

Le 03/03/2014 16:41, Mark H. Harris a ?crit :
> Greetings,
> Python3.3 Decimal Library v0.3 is Released here:
>
> https://code.google.com/p/pythondecimallibrary/
>
> pdeclib.py is the decimal library, pilib.py is the PI library.
>
> pdeclib.py  provides scientific and transcendental functions
> for the C Accelerated Decimal module written by Stefan Krah. The
> library is open source, GLPv3, comprised of two py files.
>
> My idea for python is to two things really, 1) make floating point
> decimal the default floating point type in python4.x, and 2) make
> these functions ( pdeclib.py ) or equiv available in python4.x by
> default.

If you want to contribute those functions to Python, the first required 
step would be to license them under terms compatible with the 
contributor agreement:
http://www.python.org/psf/contrib/contrib-form/

(i.e. under the Academic Free License v. 2.1 or the Apache License, 
Version 2.0)

Regards

Antoine.



From harrismh777 at gmail.com  Mon Mar  3 17:23:49 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Mon, 3 Mar 2014 08:23:49 -0800 (PST)
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <lf28ps$d7n$1@ger.gmane.org>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <lf28ps$d7n$1@ger.gmane.org>
Message-ID: <aed07157-482b-4fa8-b5be-effc14357ced@googlegroups.com>



On Monday, March 3, 2014 9:58:02 AM UTC-6, Antoine Pitrou wrote:
>
>
> Hello Mark, 
>
> Le 03/03/2014 16:41, Mark H. Harris a ?crit : 
> > Greetings, 
> > Python3.3 Decimal Library v0.3 is Released here: 
> > 
> > https://code.google.com/p/pythondecimallibrary/ 
> > 
> > pdeclib.py is the decimal library, pilib.py is the PI library. 
> > 
> > pdeclib.py  provides scientific and transcendental functions 
> > for the C Accelerated Decimal module written by Stefan Krah. The 
> > library is open source, GLPv3, comprised of two py files. 
> > 
> > My idea for python is to two things really, 1) make floating point 
> > decimal the default floating point type in python4.x, and 2) make 
> > these functions ( pdeclib.py ) or equiv available in python4.x by 
> > default. 
>
> If you want to contribute those functions to Python, the first required 
> step would be to license them under terms compatible with the 
> contributor agreement: 
> http://www.python.org/psf/contrib/contrib-form/ 
>
> (i.e. under the Academic Free License v. 2.1 or the Apache License, 
> Version 2.0) 
>

Antoine, thank you for the response.  I have e-signed the Academic Free 
License, 
and it is filed. 

Mark H Harris
marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/f79c57ec/attachment.html>

From oscar.j.benjamin at gmail.com  Mon Mar  3 17:52:01 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 3 Mar 2014 16:52:01 +0000
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
Message-ID: <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com>

On 3 March 2014 15:41, Mark H. Harris <harrismh777 at gmail.com> wrote:
> Greetings,
> Python3.3 Decimal Library v0.3 is Released here:
>
> https://code.google.com/p/pythondecimallibrary/
>
> pdeclib.py is the decimal library, pilib.py is the PI library.
>
> pdeclib.py  provides scientific and transcendental functions
> for the C Accelerated Decimal module written by Stefan Krah. The
> library is open source, GLPv3, comprised of two py files.
>
> My idea for python is to two things really, 1) make floating point decimal
> the default floating point type in python4.x,

This is an interesting suggestion. It's hard to judge how much code
would be broken by such an explicit change. A preliminary less
controversial step would just be to introduce decimal literals e.g
1.23d.

> and 2) make
> these functions ( pdeclib.py ) or equiv available in python4.x by
> default.

If there was a desire to add these then there's no reason why they
would have to wait until the (hypothetical) release of Python 4.x.
However I doubt that they would be accepted without having first spent
some time maturing on PyPI and certainly not without unit tests.


Oscar

From harrismh777 at gmail.com  Mon Mar  3 18:10:07 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Mon, 3 Mar 2014 09:10:07 -0800 (PST)
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com>
Message-ID: <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com>



On Monday, March 3, 2014 10:52:01 AM UTC-6, Oscar Benjamin wrote:
>
>
> This is an interesting suggestion. It's hard to judge how much code 
> would be broken by such an explicit change. A preliminary less 
> controversial step would just be to introduce decimal literals e.g 
> 1.23d. 
>
> If there was a desire to add these then there's no reason why they 
> would have to wait until the (hypothetical) release of Python 4.x. 
> However I doubt that they would be accepted without having first spent 
> some time maturing on PyPI and certainly not without unit tests. 
>

Oscar, sounds right.  I would like to see it just the other way round. We 
have a very speedy decimal module, its 2014, and we just don't need any
more of this:

>>> from decimal import *
>>> Decimal(.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
>>> 

I'm from back in that day. You know, about 1982. We've embedded floats and 
doubles
in everything from the processor to python and beyond. Memory is cheap, 
technology
has advanced (heck, python is one of the coolest most sophisticated 
languages on
the planet), its time to use decimal floating point.  I'm thinking we ought 
to have decimal
by default, and the binary literal  1.23b  as the option.

As far as how much code breaks; who knows, but it can't be any worse than 
moving 
from 2.7.x to 3.3.4.  I mean, everyone thought that was going to break the 
world, and
people have adapted just fine. Some folks are staying put on 2.7.6 (and 
that's fine) and
some folks like me are out there on the bloody edge downloading 3.3.5 !

Thanks for the discussion Oscar.
marcus
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/cdc8356b/attachment-0001.html>

From stefan_ml at behnel.de  Mon Mar  3 18:24:48 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 03 Mar 2014 18:24:48 +0100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com>
 <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com>
Message-ID: <lf2dsi$i60$1@ger.gmane.org>

Mark H. Harris, 03.03.2014 18:10:
> We 
> have a very speedy decimal module, its 2014, and we just don't need any
> more of this:
> 
> >>> from decimal import *
> >>> Decimal(.1)
> Decimal('0.1000000000000000055511151231257827021181583404541015625')

I assume you are aware that the correct way to do this is

   >>> Decimal('0.1')
   Decimal('0.1')

i.e., if you want exact precision, use exact literals.

That being said, did you read through the previous discussions where people
suggested to add decimal literals to Python?

Stefan



From steve at pearwood.info  Mon Mar  3 18:54:29 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Mar 2014 04:54:29 +1100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
Message-ID: <20140303175429.GL28804@ando>

On Mon, Mar 03, 2014 at 09:41:40AM -0600, Mark H. Harris wrote:

> My idea for python is to two things really, 1) make floating point decimal
> the default floating point type in python4.x, and 

I think it is premature to be talking about what goes into Python 4.x, 
which is why I refer to it as "Python 4000". There's no concrete plans 
for a Python 4 yet, or even whether there will be a Python 4, what the 
last Python 3.x version will be, or what sort of changes will be 
considered. But I would expect that any such Python 4 will probably be 
at least four years away, although given the extended life of 2.7 
possibly more like eight.

(Given the stress of the 2->3 migration, I think *nobody* is exactly in 
a hurry for yet another backwards incompatible version. Perhaps we 
should be delaying such things until Python 5000.)


> 2) make
> these functions ( pdeclib.py ) or equiv available in python4.x by
> default.

If it's worth having a decimal maths library, its probably worth having 
it in 3.5 or 3.6.



-- 
Steven

From zachary.ware+pyideas at gmail.com  Mon Mar  3 19:23:58 2014
From: zachary.ware+pyideas at gmail.com (Zachary Ware)
Date: Mon, 3 Mar 2014 12:23:58 -0600
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140303175429.GL28804@ando>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
Message-ID: <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>

On Mon, Mar 3, 2014 at 11:54 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> If it's worth having a decimal maths library, its probably worth having
> it in 3.5 or 3.6.

Agreed.

I'm -1 on a decimal-specific math library, though.  What I would
rather see is a type-agnostic math library, that does what (IIRC) the
new statistics module does: take in numbers of any type, coerce only
as is strictly necessary, calculate, and return an answer of the same
type as the input with the goal of correctness over performance.  If
all input is float (or complex), calculation could be delegated to the
existing math (or cmath) libraries, and performance wouldn't be too
much worse than using those libraries directly.

-- 
Zach

From ziad.sawalha at rackspace.com  Mon Mar  3 19:55:17 2014
From: ziad.sawalha at rackspace.com (Ziad Sawalha)
Date: Mon, 3 Mar 2014 18:55:17 +0000
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com>
 <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com>
 <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com>
 <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com>
 <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com>
Message-ID: <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com>

Thanks, Guido.

I?ll follow up with updates to common tools as I come across them (ex. pep257: https://github.com/GreenSteam/pep257/pull/64).

Ziad


On Feb 28, 2014, at 11:53 AM, Guido van Rossum <guido at python.org<mailto:guido at python.org>> wrote:

Updated!


On Thu, Feb 27, 2014 at 8:58 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote:
Approved :-)


On Feb 27, 2014, at 9:33 PM, "Guido van Rossum" <guido at python.org<mailto:guido at python.org>> wrote:

How about this patch? http://codereview.appspot.com/69870043


On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote:

On Feb 27, 2014, at 4:42 PM, "Bruce Leban" <bruce at leapyear.org<mailto:bruce at leapyear.org>> wrote:

To clarify are you asking to delete the recommendation of a blank line or the entire recommendation?

That is are you suggesting the recommendation change to

?It is recommended to place the closing quotes on a line by themselves."

That's right. I just want to get rid of the recommendation for the blank line.

I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the code begins, especially when using doctest.

--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban



On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote:
PEP-257 includes this recommendation:

?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line
docstring and its closing quotes, placing the closing quotes on a line by themselves. This way,
Emacs' fill-paragraph command can be used on it.?

I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode
within a docstring, emacs already ignores the closing triple-quote.  In fact, the most recent version
of emacs supports several different docstring formatting styles and gives you the ability to switch
between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am.

I?m considering removing that recommendation and updating some of the examples in PEP-257,
but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to
conversations that may have already been had on this topic?

Regards,
Ziad
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org<mailto:Python-ideas at python.org>
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


_______________________________________________
Python-ideas mailing list
Python-ideas at python.org<mailto:Python-ideas at python.org>
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/



--
--Guido van Rossum (python.org/~guido<http://python.org/~guido>)



--
--Guido van Rossum (python.org/~guido<http://python.org/~guido>)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/377693d1/attachment.html>

From tjreedy at udel.edu  Mon Mar  3 20:25:50 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 03 Mar 2014 14:25:50 -0500
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140303175429.GL28804@ando>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
Message-ID: <lf2kvr$aje$1@ger.gmane.org>

On 3/3/2014 12:54 PM, Steven D'Aprano wrote:

> I think it is premature to be talking about what goes into Python 4.x,
> which is why I refer to it as "Python 4000". There's no concrete plans
> for a Python 4 yet, or even whether there will be a Python 4, what the
> last Python 3.x version will be,

Given that Guido does not want double-digit minor version numbers, 3.9, 
at the latest, will be followed by 4.0.

 > or what sort of changes will be considered.

There will be several deprecated items removed, such as
http://docs.python.org/2/library/unittest.html#deprecated-aliases
I think it would be worth making a consolidated list.

I think a few modules that have replacements will be considered for 
removal. Optparse (if argparse is an adequate replacement)? Asyncore (if 
the new async catches on)?  Changing the meaning of a core feature like 
float literals is a different matter.

 > But I would expect that any such Python 4 will probably be
> at least four years away, although given the extended life of 2.7
> possibly more like eight.

3.5, ..., 3.9, 4.0 is 6 releases, which should be about 10 years.

> (Given the stress of the 2->3 migration, I think *nobody* is exactly in
> a hurry for yet another backwards incompatible version. Perhaps we
> should be delaying such things until Python 5000.)


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Mar  3 20:27:26 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 03 Mar 2014 14:27:26 -0500
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com>
 <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com>
 <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com>
 <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com>
 <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com>
 <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com>
Message-ID: <lf2l2q$aje$2@ger.gmane.org>

On 3/3/2014 1:55 PM, Ziad Sawalha wrote:
> Thanks, Guido.
>
> I?ll follow up with updates to common tools as I come across them (ex.
> pep257: https://github.com/GreenSteam/pep257/pull/64).

Does pep8.py try to enforce the extra blank?


-- 
Terry Jan Reedy



From p.f.moore at gmail.com  Mon Mar  3 21:12:53 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 3 Mar 2014 20:12:53 +0000
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
Message-ID: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>

On 3 March 2014 18:23, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote:
> I'm -1 on a decimal-specific math library, though.  What I would
> rather see is a type-agnostic math library, that does what (IIRC) the
> new statistics module does: take in numbers of any type, coerce only
> as is strictly necessary, calculate, and return an answer of the same
> type as the input with the goal of correctness over performance.

Given that incorrect results aren't acceptable in any situation, fair
enough, but what I'd like to see would be library functions that
provided good performance for decimal first. If they can do so while
remaining type-agnostic, then fine, but I'd favour decimal performance
over the ability to calculate the sine of a rational... Ultimately,
I'd like to feel that I pay a one-off cost for working with decimals
(the cost of a software implementation rather than a hardware one) but
I *don't* have to pay further because the algorithms used for decimals
are suboptimal compared to the floating point ones. I'd say the same
about rationals, but the set of functions involved is different (and
basically wouldn't involve most of what's in the normal math library)

Having said this, I can't actually think of a real-life domain where
math-library functions (by which I assume we basically mean
trancendental functions?) would be used and yet there would be a
genuine need for decimal arithmetic rather than floating point. So
although I'm +1 for a decimal math library in theory, I'm unsure of
its practical value.

Paul

From mertz at gnosis.cx  Mon Mar  3 21:44:12 2014
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 3 Mar 2014 12:44:12 -0800
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
Message-ID: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>

Taking a look at the documentation for cdecimal itself (now part of Python
3.3) at http://www.bytereef.org/mpdecimal/benchmarks.html, it looks like
for basic add/mul operations that don't exceed the precision of floating
point, FP is more than twice as fast as optimized decimals.  Of course,
where the precision needed is more than FP handles at all, it's simply a
choice between calculating a quantity and not doing so.

This suggests to me a rather strong argument against making decimals the
*default* numeric datatype.  However, it *does* still remain overly
cumbersome to create decimals, and a literal notation for them would be
very welcomed.

It is far from clear to me that decimals are generically the *right* answer
to rounding issues (i.e. even if we don't care about the benchmark
question).  Yes, it's easier for humans who use base 10 to understand
numeric results when using them.  But the hardware on CPUs is oriented
around binary floating point representations, and pretty much every other
programming language uses BFP.  It's a nice property to have 'a+b' in
Python produce the same answer as 'a+b' in C, Ruby, Perl, Mathematica,
Haskell, Java, etc.--I'm not saying that's the only nice property one might
want, but it is one of them since data is exchanged between tools and
libraries written in different languages (or even, e.g. Python often uses
libraries written in C).

I do recognize that in Python 4000, we *could* have default literals be
decimal, and require some other literal notation for binary floating point,
and that *would* be available for calculations meant to be compatible with
outside tools.  However, I haven't seen a case presented for why decimals
are generically better as a default.

Of possible note, I happen to work in a research lab where we worry a lot
about bitwise identity of calculations, not merely about numeric stability.
 I know that is a somewhat unusual requirement, even within scientific
computing.  But it is one that exists, and it makes me think of (bitwise)
compatibility with calculations in other programming languages.



On Mon, Mar 3, 2014 at 12:12 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 3 March 2014 18:23, Zachary Ware <zachary.ware+pyideas at gmail.com>
> wrote:
> > I'm -1 on a decimal-specific math library, though.  What I would
> > rather see is a type-agnostic math library, that does what (IIRC) the
> > new statistics module does: take in numbers of any type, coerce only
> > as is strictly necessary, calculate, and return an answer of the same
> > type as the input with the goal of correctness over performance.
>
> Given that incorrect results aren't acceptable in any situation, fair
> enough, but what I'd like to see would be library functions that
> provided good performance for decimal first. If they can do so while
> remaining type-agnostic, then fine, but I'd favour decimal performance
> over the ability to calculate the sine of a rational... Ultimately,
> I'd like to feel that I pay a one-off cost for working with decimals
> (the cost of a software implementation rather than a hardware one) but
> I *don't* have to pay further because the algorithms used for decimals
> are suboptimal compared to the floating point ones. I'd say the same
> about rationals, but the set of functions involved is different (and
> basically wouldn't involve most of what's in the normal math library)
>
> Having said this, I can't actually think of a real-life domain where
> math-library functions (by which I assume we basically mean
> trancendental functions?) would be used and yet there would be a
> genuine need for decimal arithmetic rather than floating point. So
> although I'm +1 for a decimal math library in theory, I'm unsure of
> its practical value.
>
> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/8722d467/attachment.html>

From guido at python.org  Mon Mar  3 21:52:10 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Mar 2014 12:52:10 -0800
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <lf2l2q$aje$2@ger.gmane.org>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com>
 <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com>
 <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com>
 <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com>
 <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com>
 <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com>
 <lf2l2q$aje$2@ger.gmane.org>
Message-ID: <CAP7+vJJu=-AdZY49caDgB7Ot5m0VMNjLz-uC=f4ZviyMU1FdFQ@mail.gmail.com>

On Mon, Mar 3, 2014 at 11:27 AM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/3/2014 1:55 PM, Ziad Sawalha wrote:
>
>> Thanks, Guido.
>>
>> I'll follow up with updates to common tools as I come across them (ex.
>> pep257: https://github.com/GreenSteam/pep257/pull/64).
>>
>
I already did that.

>
> Does pep8.py try to enforce the extra blank?
>

I dunno; it shouldn't. (I use it regularly and it doesn't complain to me.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/a92e609f/attachment-0001.html>

From harrismh777 at gmail.com  Mon Mar  3 23:32:37 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Mon, 3 Mar 2014 14:32:37 -0800 (PST)
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
Message-ID: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>



On Monday, March 3, 2014 2:44:12 PM UTC-6, David Mertz wrote:

  However, I haven't seen a case presented for why decimals are generically 
> better as a default.
>
> hi David,  here is one, right out of the python3.3 docs,

"Decimal ?is based on a floating-point model which was designed 
with people in mind, and necessarily has a paramount guiding 
principle ? computers must provide an arithmetic that works in the 
same way as the arithmetic that people learn at school.? ? excerpt 
from the decimal arithmetic specification.

Business apps require precision (banking, sales, marketing, finance, 
& on and on). 

One big issue that is going to confront everyone sooner than later is
cryptography. Fast bignum support, fast factoring, and fast transcendentals
are going to become more important as firms and individuals move
into their own on crypto; not too far fetched, really. We have got to 
come up with this on our own, because we can not trust others to get
it right for us.  NSA and GCHQ have made that clear. PGP was one thing,
but we have got to invent something better.

But, really the real reason is the first paragraph.
marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/a70f9d1f/attachment.html>

From mertz at gnosis.cx  Mon Mar  3 23:47:06 2014
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 3 Mar 2014 14:47:06 -0800
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
Message-ID: <CAEbHw4aVZXT9wDDum2_DtPv1Mxnb48fJPu+CDjQhAwCh9U2bEA@mail.gmail.com>

On Mon, Mar 3, 2014 at 2:32 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

> Business apps require precision (banking, sales, marketing, finance,
> & on and on).
>

I definitely agree the decimal numbers are much better for financial data
than binary floating point.  That's certainly a very legitimate domain.  On
the other hand, there are a great many other domains that are very
different from this--i.e. scientific areas.  For this, decimal is no
better, and in many cases worse.  At the very least, throwing in a 2x+
slowdown of calculations is notably worse for scientific computing.

In other words, I'm glad we have decimals in Python now.  I'd be more glad
if they were expressed more easily (e.g. as literals), but for more domains
than not, for backward and cross-language compatibility, and for speed,
binary floating point remains better.


> One big issue that is going to confront everyone sooner than later is
> cryptography. Fast bignum support, fast factoring, and fast
> transcendentals are going to become more important as firms and
> individuals move into their own on crypto; not too far fetched, really.
>

Sounds far fetched to me to imagine that amateur cryptography will ever be
of value over well-audited, well studied, crypto-analyzed techniques.

But bracketing that, exceedingly few cryptographic techniques are likely to
use either binary or decimal floating point operations.  This is strictly
the domain of integer math, so mentioning it seems like random hand waving
unrelated to the topic of default or convenient representation of
fractional numeric data.

Yours, David...

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/ea51db41/attachment.html>

From greg.ewing at canterbury.ac.nz  Mon Mar  3 23:54:08 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Mar 2014 11:54:08 +1300
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
Message-ID: <53150810.20507@canterbury.ac.nz>

Mark H. Harris wrote:
> One big issue that is going to confront everyone sooner than later is
> cryptography. Fast bignum support, fast factoring, and fast transcendentals
> are going to become more important

I don't see any reason that these have to be done in
decimal, though. All the bignum arithmetic used in
cryptography is integer, which is exact in any base.
Also, the user never sees the resulting numbers as
numbers. So the primary requirement is speed, which
argues for binary rather than decimal, at least on
current hardware.

-- 
Greg

From solipsis at pitrou.net  Tue Mar  4 00:01:22 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 4 Mar 2014 00:01:22 +0100
Subject: [Python-ideas] Python3.3 Decimal Library Released
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz>
Message-ID: <20140304000122.4f61e287@fsol>

On Tue, 04 Mar 2014 11:54:08 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Mark H. Harris wrote:
> > One big issue that is going to confront everyone sooner than later is
> > cryptography. Fast bignum support, fast factoring, and fast transcendentals
> > are going to become more important
> 
> I don't see any reason that these have to be done in
> decimal, though. All the bignum arithmetic used in
> cryptography is integer, which is exact in any base.
> Also, the user never sees the resulting numbers as
> numbers. So the primary requirement is speed, which
> argues for binary rather than decimal, at least on
> current hardware.

For the record, int doesn't have a sqrt() method while Decimal has, so
if you wanna take the exact square root of a large integer, you'd better
convert it to a Decimal.

Regards

Antoine.



From mertz at gnosis.cx  Tue Mar  4 00:07:57 2014
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 3 Mar 2014 15:07:57 -0800
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140304000122.4f61e287@fsol>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
Message-ID: <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>

On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> For the record, int doesn't have a sqrt() method while Decimal has, so
> if you wanna take the exact square root of a large integer, you'd better
> convert it to a Decimal.
>

Well, actually, if you want to take the square root of a large integer,
most times you'll need an irrational number as a value.  Unfortunately,
neither floats, decimals, nor fractions are able to do that (nor any finite
representation that is numeric; you can only use symbolic ones).

On the other hand, now that you mention it, a floor_sqrt() function that
operated quickly on ints would be nice to have.  But that's a different
thread.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/3851bee6/attachment-0001.html>

From solipsis at pitrou.net  Tue Mar  4 00:10:48 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 4 Mar 2014 00:10:48 +0100
Subject: [Python-ideas] Python3.3 Decimal Library Released
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
Message-ID: <20140304001048.22d6442c@fsol>

On Mon, 3 Mar 2014 15:07:57 -0800
David Mertz <mertz at gnosis.cx> wrote:
> On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> > For the record, int doesn't have a sqrt() method while Decimal has, so
> > if you wanna take the exact square root of a large integer, you'd better
> > convert it to a Decimal.
> >
> 
> Well, actually, if you want to take the square root of a large integer,
> most times you'll need an irrational number as a value.

Well, unless you know by construction that your integer is a perfect
square.

Regards

Antoine.



From rosuav at gmail.com  Tue Mar  4 00:42:57 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Mar 2014 10:42:57 +1100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
Message-ID: <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com>

On Tue, Mar 4, 2014 at 7:44 AM, David Mertz <mertz at gnosis.cx> wrote:
> Taking a look at the documentation for cdecimal itself (now part of Python
> 3.3) at http://www.bytereef.org/mpdecimal/benchmarks.html, it looks like for
> basic add/mul operations that don't exceed the precision of floating point,
> FP is more than twice as fast as optimized decimals.  Of course, where the
> precision needed is more than FP handles at all, it's simply a choice
> between calculating a quantity and not doing so.
>
> This suggests to me a rather strong argument against making decimals the
> *default* numeric datatype.  However, it *does* still remain overly
> cumbersome to create decimals, and a literal notation for them would be very
> welcomed.

You could probably make the same performance argument against making
Unicode the default string datatype. But a stronger argument is that
the default string should be the one that does the right thing with
text. As of Python 3, that's the case. And the default integer type
handles arbitrary sized integers (although Py2 went most of the way
there by having automatic promotion). It's reasonable to suggest that
the default non-integer numeric type should also simply do the right
thing.

It's a trade-off, though, and for most people, float is sufficient. Is
it worth the cost of going decimal everywhere? I want to first see a
decimal literal notation (eg 1.0 == float("1.0") and 1.0d ==
Decimal("1.0"), as has been suggested a few times), and then have a
taggable float marker (1.0f == float("1.0")); then there can be
consideration of a __future__ directive to change the default, which
would let people try it out and see how much performance suffers.Maybe
it'd be worth it for the accuracy. Maybe we'd lose less time
processing than we save answering the "why does this do weird things
sometimes" questions.

ChrisA

From random832 at fastmail.us  Tue Mar  4 02:19:06 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 03 Mar 2014 20:19:06 -0500
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
Message-ID: <1393895946.9103.90200705.2DF58D8B@webmail.messagingengine.com>

On Mon, Mar 3, 2014, at 15:12, Paul Moore wrote:
> Having said this, I can't actually think of a real-life domain where
> math-library functions (by which I assume we basically mean
> trancendental functions?) would be used and yet there would be a
> genuine need for decimal arithmetic rather than floating point. So
> although I'm +1 for a decimal math library in theory, I'm unsure of
> its practical value.

If we count non-integer powers (though, usually they're still rational),
finance.

I suspect though there's a better case for an arbitrary-precision
library and no reason that library shouldn't be (also) able to work in
decimal.

From harrismh777 at gmail.com  Tue Mar  4 02:37:26 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Mon, 3 Mar 2014 17:37:26 -0800 (PST)
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
Message-ID: <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com>



On Monday, March 3, 2014 9:41:40 AM UTC-6, Mark H. Harris wrote:
>
>
> Python3.3 Decimal Library v0.3 is Released here:
>
> https://code.google.com/p/pythondecimallibrary/
>
>  
hi Oscar,

I released my  pdeclib module  on PyPI this afternoon (it only took four
hours) but I'll never forget how to do that again ! So if anyone is 
interested
in the  pdeclib  module, it may be installed  with  pip install pdeclib

The tarball will install both  pdeclib.py (decimal library)  &  pilib.py 
(PI Library).

The library is beta obviously, and is licensed for contribution on PyPI.

Cheers,

marcus


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/2a6d34c2/attachment.html>

From steve at pearwood.info  Tue Mar  4 02:55:16 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Mar 2014 12:55:16 +1100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com>
Message-ID: <20140304015516.GM28804@ando>

On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote:

> You could probably make the same performance argument against making
> Unicode the default string datatype. 

I don't think so -- for ASCII strings the performance cost of Unicode is 
significantly less than the performance hit for Decimal:

[steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()"
100000 loops, best of 3: 8.76 usec per loop
[steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()"
100000 loops, best of 3: 7.05 usec per loop

[steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6"
1000000 loops, best of 3: 0.344 usec per loop
[steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \
> -s "x = Decimal('123.4567')" "x**6"
1000000 loops, best of 3: 1.41 usec per loop


That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal. 
I think that's *fast enough* for all but the most heavy numeric needs, 
but it's not something we can ignore.


> But a stronger argument is that
> the default string should be the one that does the right thing with
> text. As of Python 3, that's the case. And the default integer type
> handles arbitrary sized integers (although Py2 went most of the way
> there by having automatic promotion). It's reasonable to suggest that
> the default non-integer numeric type should also simply do the right
> thing.

Define "the right thing" for numbers.

> It's a trade-off, though, and for most people, float is sufficient.

That's a tricky one. For people doing quote-unquote "serious" numeric 
work, they'll mostly want to stick to binary floats, even if that means 
missing out on all the extra IEEE-754 goodies that the decimal module 
has but floats don't. The momentum of 40+ years of almost entirely 
binary floating point maths does not shift to decimal overnight.

But for everyone else, binary floats are sufficient except when they 
aren't. Decimal, of course, won't solve all you floating point 
difficulties -- it's easy to demonstrate that nearly all the common 
pitfalls of FP maths also occurs with Decimal, with the exception of 
inexact conversion from decimal strings to numbers. But that one issue 
alone is a major cause of confusion.

My personal feeling is that for Python 4000 I'd vote for the default 
floating point format to be decimal, with binary floats available with a 
b suffix.

But since that could be a decade away, it's quite premature to spend too 
much time on this.



-- 
Steven

From python at mrabarnett.plus.com  Tue Mar  4 03:10:29 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 04 Mar 2014 02:10:29 +0000
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140304015516.GM28804@ando>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com>
 <20140304015516.GM28804@ando>
Message-ID: <53153615.4070304@mrabarnett.plus.com>

On 2014-03-04 01:55, Steven D'Aprano wrote:
> On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote:
>
>> You could probably make the same performance argument against making
>> Unicode the default string datatype.
>
> I don't think so -- for ASCII strings the performance cost of Unicode is
> significantly less than the performance hit for Decimal:
>
> [steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()"
> 100000 loops, best of 3: 8.76 usec per loop
> [steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()"
> 100000 loops, best of 3: 7.05 usec per loop
>
> [steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6"
> 1000000 loops, best of 3: 0.344 usec per loop
> [steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \
>> -s "x = Decimal('123.4567')" "x**6"
> 1000000 loops, best of 3: 1.41 usec per loop
>
>
> That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal.
> I think that's *fast enough* for all but the most heavy numeric needs,
> but it's not something we can ignore.
>
>
>> But a stronger argument is that
>> the default string should be the one that does the right thing with
>> text. As of Python 3, that's the case. And the default integer type
>> handles arbitrary sized integers (although Py2 went most of the way
>> there by having automatic promotion). It's reasonable to suggest that
>> the default non-integer numeric type should also simply do the right
>> thing.
>
> Define "the right thing" for numbers.
>
>> It's a trade-off, though, and for most people, float is sufficient.
>
> That's a tricky one. For people doing quote-unquote "serious" numeric
> work, they'll mostly want to stick to binary floats, even if that means
> missing out on all the extra IEEE-754 goodies that the decimal module
> has but floats don't. The momentum of 40+ years of almost entirely
> binary floating point maths does not shift to decimal overnight.
>
[snip]
Won't people doing quote-unquote "serious" numeric work be using numpy?


From mertz at gnosis.cx  Tue Mar  4 03:17:18 2014
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 3 Mar 2014 18:17:18 -0800
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140304001048.22d6442c@fsol>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
 <20140304001048.22d6442c@fsol>
Message-ID: <CAEbHw4aHbAku7_1yR_uMNrLZP-zQaEjedhqvkP7jJH4UzsA-eA@mail.gmail.com>

On Mon, Mar 3, 2014 at 3:10 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 3 Mar 2014 15:07:57 -0800
> David Mertz <mertz at gnosis.cx> wrote:
> > On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >> > For the record, int doesn't have a sqrt() method while Decimal has, so
> > > if you wanna take the exact square root of a large integer, you'd
> better
> > > convert it to a Decimal.
> > Well, actually, if you want to take the square root of a large integer,
> > most times you'll need an irrational number as a value.
>
> Well, unless you know by construction that your integer is a perfect
> square.
>

Umm... if you construct your integer as a perfect square, wouldn't it be
easier just to store the number it is a perfect square of than to work on
optimizing the integer sqrt() function?

It does make me wonder--although this is definitely not actually
python-ideas--whether there is any technique to determine if a number is a
perfect square that takes less work than finding its integral root.  Maybe
so, I don't know very much number theory.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/799961a0/attachment.html>

From musicdenotation at gmail.com  Tue Mar  4 04:39:27 2014
From: musicdenotation at gmail.com (Brian Nguyen)
Date: Tue, 4 Mar 2014 10:39:27 +0700
Subject: [Python-ideas] Local scope for statement blocks
Message-ID: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com>

do:
    nonlocal x
    k = init
    for i in list:
        k = f(i,k)
        n = f2(n,k)

From rosuav at gmail.com  Tue Mar  4 04:40:00 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Mar 2014 14:40:00 +1100
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140304015516.GM28804@ando>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com>
 <20140304015516.GM28804@ando>
Message-ID: <CAPTjJmp2Y8BQ8z9-CgL9jk1iYXzYwUynC+ORwTCJdjSqC6=Hkw@mail.gmail.com>

On Tue, Mar 4, 2014 at 12:55 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote:
>
>> You could probably make the same performance argument against making
>> Unicode the default string datatype.
>
> I don't think so -- for ASCII strings the performance cost of Unicode is
> significantly less than the performance hit for Decimal:
>
> [steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()"
> 100000 loops, best of 3: 8.76 usec per loop
> [steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()"
> 100000 loops, best of 3: 7.05 usec per loop
>
> [steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6"
> 1000000 loops, best of 3: 0.344 usec per loop
> [steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \
>> -s "x = Decimal('123.4567')" "x**6"
> 1000000 loops, best of 3: 1.41 usec per loop
>
>
> That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal.
> I think that's *fast enough* for all but the most heavy numeric needs,
> but it's not something we can ignore.

There is a difference of degree, yes, but Unicode-strings-as-default
has had a few versions to settle in, so the figures mightn't be
perfectly fair. But there's still a difference. My point is that
Python should be choosing what's right over what's fast, so there's a
parallel there.

>> It's reasonable to suggest that
>> the default non-integer numeric type should also simply do the right
>> thing.
>
> Define "the right thing" for numbers.

Yeah, and that's the issue. In this case, since computers don't have
infinite computational power, "the right thing" is going to be fairly
vague, but I'd define it heuristically as "what the average programmer
is most likely to expect". IEEE 754 defines operations on infinity in
a way that makes them do exactly what you'd expect. If that's not
possible, nan.

>>> inf=float("inf")
>>> inf+5
inf
>>> 5-inf
-inf
>>> 5/inf
0.0
>>> inf-inf
nan

A default decimal type would add to the "doing exactly what you
expect" operations the obvious one of constructing an object from a
series of decimal digits. If you say "0.1", you get the real number
1/10, not 3602879701896397/36028797018963968.

> My personal feeling is that for Python 4000 I'd vote for the default
> floating point format to be decimal, with binary floats available with a
> b suffix.

Quite possibly. But changing defaults is a hugely
backward-incompatible change, while adding a decimal literal syntax
isn't. I'd be in favour of adding decimal literals and using
performance and usefulness data from that to guide any discussions
about Py4K changing the default.

And learn from Py3K and keep both the b and d suffixes supported in
the new version :)

ChrisA

From rosuav at gmail.com  Tue Mar  4 04:52:49 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Mar 2014 14:52:49 +1100
Subject: [Python-ideas] Local scope for statement blocks
In-Reply-To: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com>
References: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com>
Message-ID: <CAPTjJmruw7sEXY4prfQKcfRWxp7zBBUiatVoFJCXjWZBUM0piA@mail.gmail.com>

On Tue, Mar 4, 2014 at 2:39 PM, Brian Nguyen <musicdenotation at gmail.com> wrote:
> do:
>     nonlocal x
>     k = init
>     for i in list:
>         k = f(i,k)
>         n = f2(n,k)

You may need to elaborate a bit, especially as you have a declaration
regarding 'x' and never use x anywhere. But if, as I suspect, you
intend for this to have local k/i/n and fetch init/list/f/f2 from its
parent scope, then this is something that's been discussed a few
times. With the current implementation of CPython, the easiest and
cleanest way to do this is with a closure. You could spell 'do' like
this:

def do(f):
    f()

@do
def _():
    nonlocal x
    k = init
    for i in list:
        k = f(i,k)
        n = f2(n,k)

Put that inside a function and it'll do most of what I think you're saying.

ChrisA

From aquavitae69 at gmail.com  Tue Mar  4 07:02:46 2014
From: aquavitae69 at gmail.com (David Townshend)
Date: Tue, 4 Mar 2014 08:02:46 +0200
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <20140301034611.GD28804@ando>
References: <leqgah$b0o$1@ger.gmane.org>
	<20140301034611.GD28804@ando>
Message-ID: <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>

On 1 Mar 2014 05:46, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote:
> >
> > Starting new thread because this bike has a different shape and color.
> >
> > Yesterday I was thinking that just making the keyword lambda assignable
> > like True, False, and None, would be enough.
>
> You can't assign to True, False or None. (You can assign to True and
> False in Python 2, but shouldn't.)
>
>
> [...]
> > This morning I thought we could have in a functions definition
something,
> > like "*", and "**", to take an expression.  Similar to Nicks idea with
=:,
> > but more general.
> >
> > The idea is to have "***" used in def mean to take "any" call expression
> > and not evaluate it until *** is used on it.
> [...]
> > A function call that captures an expression may be tricky to do. Here's
one
> > approach that requires sugar when a function defined with "***" is
called.
>
> I think it would be useful to have a way to delay execution of an
> expression, that is to say, have a way to capture an expression for
> later evaluation, something more lightweight than a function, but I
> don't think that limiting it to inside function calls is the right
> approach.
>
> Something perhaps like a thunk might be appropriate? We can *almost* do
> that now, since Python has a compile function:
>
> thunk = compile("x + 3", "", "eval")
> # much later
> eval(thunk)

What about a new code literal object, e.g.

    thunk = c'x + 3'
    thunk(x=2)

The "c" prefix identifies the literal as an code so highlighters can
recognise it and it can be parsed. Triple quotes could also be used to
support multiline code blocks.  It would also be possible to prohibit any
direct conversion from a string to avoid the security problems. I'm not
sure how positional arguments would be handled though, and scoping still
needs to be thought through.

>
> Some problems with that approach:
>
> - You have to write the expression as a string, which means you lose any
> possibility of syntax highlighting.
>
> - The temptation is to pass some arbitrary untrusted string, which leads
> to serious security implications. The argument here should be limited
> to an actual expression, not a string containing an expression which
> might have come from who knows where.
>
> - It should be as lightweight as possible. The actual compilation of the
> expression should occur at compile-time, not run-time. That implies some
> sort of syntax for making thunks, rather than a function call.
>
> - Likewise actually evaluating the thunk should be really lightweight,
> which may rule out a function call to eval.
>
> - How should scoping work? I can see use-cases for flat scoping, static
> scoping, dynamic scoping, and the ability to optionally provide custom
> globals and locals, but I have no idea how practical any of them would
> be or what syntax they should use.
>
>
> All of this is pie-in-the-sky at the moment, and not a serious proposal
> for Python 3.5.
>
>
> --
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/741cac8a/attachment-0001.html>

From abarnert at yahoo.com  Tue Mar  4 07:51:17 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 3 Mar 2014 22:51:17 -0800
Subject: [Python-ideas] Local scope for statement blocks
In-Reply-To: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com>
References: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com>
Message-ID: <CF05DD08-F2EA-4D76-833E-3F34BE3FDB71@yahoo.com>

On Mar 3, 2014, at 19:39, Brian Nguyen <musicdenotation at gmail.com> wrote:

> do:
>    nonlocal x
>    k = init
>    for i in list:
>        k = f(i,k)
>        n = f2(n,k)

You want _every_ compound statement to be a scope, or you just want to add a new one, a "do" statement, and only that new one is a scope?

If the latter, i believe you can do this pretty easily with a MacroPy macro (which will expand to a def and a function call). It might be worth playing with that to get some real-life examples of using it.


From abarnert at yahoo.com  Tue Mar  4 08:03:03 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 3 Mar 2014 23:03:03 -0800
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
Message-ID: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>

On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote:
> What about a new code literal object, e.g.
> 
>     thunk = c'x + 3'
>     thunk(x=2)
> 
Why does it need to be built from/look like a string? I think it would be just as simple for the parser, and simpler for editors, and less misleading for readers, if it used a different marker.

Not that I'm seriously suggesting backticks here, but...

    thunk = `x + 3`

Most existing editors already treat that as an expression (since in 2.x it means repr(x + 3)). No human reader is going to be misled into thinking this means there's a way to create code objects from string objects without eval (c.f. all those questions on StackOverflow about how to make a raw string from a string...). And as far as the parser is concerned, it's trivial: '`' expr '`' is a Code whose value is whatever expr parses to.

Meanwhile, if this gives you a code object, what do you do with it? Using eval on something that looks like a string but isn't may be correct to anyone who understands Python deeply, but I think it could be very misleading to anyone else. And creating a FunctionType from a code object is pretty ugly for something that deserved literal support...
> Triple quotes could also be used to support multiline code blocks.
> 
That is one advantage of reusing strings. In fact, that gives us a way to embed statements in the middle of expressions. Which I'm not sure is a good thing.
> I'm not sure how positional arguments would be handled though, and scoping still needs to be thought through.
> 
If you have a code object, scoping is up to whoever evaluates it or builds a function out of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/553d0281/attachment.html>

From bruce at leapyear.org  Tue Mar  4 08:26:13 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Mon, 3 Mar 2014 23:26:13 -0800
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
Message-ID: <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com>

On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote:
>
> What about a new code literal object, e.g.
>
>     thunk = c'x + 3'
>     thunk(x=2)
>
> This already exists.

    thunk = lambda: x + 3
    thunk(x=2)

Your original post in this thread wanted to allow functions to magically
capture their parameters as code (~ call by name) and it's been pointed out
that that is incompatible with the way the language is designed and
function calls are implemented. Yes, there are languages where the called
function gets to tell the caller how to package arguments but Python isn't
one of them.

Now you seem to have veered in a different direction. I have no idea what
problem you are trying to solve.

--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/98627761/attachment.html>

From p.f.moore at gmail.com  Tue Mar  4 09:01:44 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 4 Mar 2014 08:01:44 +0000
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com>
Message-ID: <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com>

On 4 March 2014 01:37, Mark H. Harris <harrismh777 at gmail.com> wrote:
> I released my  pdeclib module  on PyPI this afternoon (it only took four
> hours) but I'll never forget how to do that again ! So if anyone is
> interested
> in the  pdeclib  module, it may be installed  with  pip install pdeclib

Regardless of anything else that comes out of this thread, thanks for
doing that.
Paul

From aquavitae69 at gmail.com  Tue Mar  4 10:07:28 2014
From: aquavitae69 at gmail.com (David Townshend)
Date: Tue, 4 Mar 2014 11:07:28 +0200
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com>
Message-ID: <CAEgL-fcWYrGP=NKEziZzS4pT5b+Q-dyKO4W7EcqFjVOkp649Rg@mail.gmail.com>

On Tue, Mar 4, 2014 at 9:26 AM, Bruce Leban <bruce at leapyear.org> wrote:

>
>
> On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote:
>>
>> What about a new code literal object, e.g.
>>
>>     thunk = c'x + 3'
>>     thunk(x=2)
>>
>> This already exists.
>
>     thunk = lambda: x + 3
>     thunk(x=2)
>
> Your original post in this thread wanted to allow functions to magically
> capture their parameters as code (~ call by name) and it's been pointed out
> that that is incompatible with the way the language is designed and
> function calls are implemented. Yes, there are languages where the called
> function gets to tell the caller how to package arguments but Python isn't
> one of them.
>
> Now you seem to have veered in a different direction. I have no idea what
> problem you are trying to solve.
>
>
This is my first post in this thread - I don't recall who the OP was, but
it wasn't me.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/d05a0fe2/attachment-0001.html>

From aquavitae69 at gmail.com  Tue Mar  4 10:03:35 2014
From: aquavitae69 at gmail.com (David Townshend)
Date: Tue, 4 Mar 2014 11:03:35 +0200
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
Message-ID: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com>

On Tue, Mar 4, 2014 at 9:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote:
>
> What about a new code literal object, e.g.
>
>     thunk = c'x + 3'
>     thunk(x=2)
>
> Why does it need to be built from/look like a string? I think it would be
> just as simple for the parser, and simpler for editors, and less misleading
> for readers, if it used a different marker.
>
> Not that I'm seriously suggesting backticks here, but...
>
>     thunk = `x + 3`
>
>
The only real reason for it looking like a string is a shortage of
symbols.  Backticks are off limits, and most other symbols are just plain
ugly (e.g. @x + 3@ or $x + 3$) or already in use in a way that could lead
to ambiguity (e.g. |x + 3|).  String-like quotes seem like a better option
than the alternatives.


> Most existing editors already treat that as an expression (since in 2.x it
> means repr(x + 3)). No human reader is going to be misled into thinking
> this means there's a way to create code objects from string objects without
> eval (c.f. all those questions on StackOverflow about how to make a raw
> string from a string...). And as far as the parser is concerned, it's
> trivial: '`' expr '`' is a Code whose value is whatever expr parses to.
>
> Meanwhile, if this gives you a code object, what do you do with it? Using
> eval on something that looks like a string but isn't may be correct to
> anyone who understands Python deeply, but I think it could be very
> misleading to anyone else. And creating a FunctionType from a code object
> is pretty ugly for something that deserved literal support...
>

My suggestion was to use c'x + 3' as an effective replacement for
compile('x + 3', '', 'eval').  The use of a literal rather than compile
would solve a few of the issues raised in Steven's original post.  I don't
see why it would be any more misleading to eval a code literal than a
compiled string.  I did suggest making it callable (i.e. thunk(x=3), but on
second thoughts I'm not sure that's a good idea.  It adds complexity
without and real benefit over eval.

> Triple quotes could also be used to support multiline code blocks.
>
> That is one advantage of reusing strings. In fact, that gives us a way to
> embed statements in the middle of expressions. Which I'm not sure is a good
> thing.
>
> I'm not sure how positional arguments would be handled though, and scoping
> still needs to be thought through.
>
> If you have a code object, scoping is up to whoever evaluates it or builds
> a function out of it.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/9c2a877f/attachment.html>

From greg.ewing at canterbury.ac.nz  Tue Mar  4 10:18:13 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Mar 2014 22:18:13 +1300
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <20140304001048.22d6442c@fsol>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
 <20140304001048.22d6442c@fsol>
Message-ID: <53159A55.9070905@canterbury.ac.nz>

Antoine Pitrou wrote:
> Well, unless you know by construction that your integer is a perfect
> square.

If you've constructed it that way, you probably already
know its square root.

-- 
Greg

From abarnert at yahoo.com  Tue Mar  4 11:57:57 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 4 Mar 2014 02:57:57 -0800
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <53159A55.9070905@canterbury.ac.nz>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
 <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz>
Message-ID: <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com>

On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Antoine Pitrou wrote:
>> Well, unless you know by construction that your integer is a perfect
>> square.
> 
> If you've constructed it that way, you probably already
> know its square root.

Usually, but not always.

For example, there are algorithms to generate squares of Pythagorean triples without generating the triples themselves. Of course there are simpler and more efficient algorithms to just generate the triples (can't get much simpler than Euclid's formula...), but there might be some reason you're using one of the square algorithms. And then, to test it, you'd need to verify that a2+b2==c2 and that all three of them are perfect squares.

From steve at pearwood.info  Tue Mar  4 12:09:21 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Mar 2014 22:09:21 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
Message-ID: <20140304110921.GP28804@ando>

On Mon, Mar 03, 2014 at 11:03:03PM -0800, Andrew Barnert wrote:
> On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote:
> > What about a new code literal object, e.g.
> > 
> >     thunk = c'x + 3'

Off the top of my head, I think that looks alright.


> >     thunk(x=2)

I'm not entirely sure about that. That basically makes these thunks just 
a function. I'm still not quite sure how this should actually be used in 
practice, so as far as I'm concerned this is just pie in the sky 
thinking aloud. If thunks are just functions, why not make them 
functions? There needs to be something extra (different scoping rules, 
faster/more lightweight, *something*) to make the idea worthwhile.

I think I need to learn more about Algol and other languages that use 
call-by-name and thunks. I may be completely on a wild-goose chase here, 
but I'm surely not the only person who has needed to delay evaluation of 
an expression.

http://en.wikipedia.org/wiki/Thunk


> Why does it need to be built from/look like a string? I think it would 
> be just as simple for the parser, and simpler for editors, and less 
> misleading for readers, if it used a different marker.
> 
> Not that I'm seriously suggesting backticks here, but...
> 
>     thunk = `x + 3`

Actually I'd be happy with backticks, but Guido has said No Backticks 
Ever Again. So until the Glorious Revolution, we're stuck.


> Meanwhile, if this gives you a code object, what do you do with it? 
> Using eval on something that looks like a string but isn't may be 
> correct to anyone who understands Python deeply, but I think it could 
> be very misleading to anyone else. And creating a FunctionType from a 
> code object is pretty ugly for something that deserved literal 
> support...

Agreed. What I have in my head is some vague concept that the Python 
evaluation rules will somehow know when to evaluate the thunk and when 
to treat it as an object, which is (as I understand it) what happens in 
Algol. Again, just thinking aloud, perhaps we do this:

thunk = `some_expression`  # delays evaluation
a = [0, 1 + thunk]  # evaluates thunk in the current scope
b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
                      # equivalent to `1 + some_expression`
c = b[1]  # now evaluates the thunk object
d = f(2, thunk)  # evaluates thunk in f's scope
e = g(3, `thunk`)  # passes the un-evaluated thunk object to g

Consider this just a sketch, and in no way fully thought out.

(This will most definitely need a PEP.)


> > Triple quotes could also be used to support multiline code blocks.
> > 
> That is one advantage of reusing strings. In fact, that gives us a way 
> to embed statements in the middle of expressions. Which I'm not sure 
> is a good thing.

I have little interest in allowing thunks to be statements. If you want 
a delayed statement, use compile and eval. Or def. (But maybe that 
applies to expressions too?)

Did I mention this needs a PEP?



-- 
Steven

From steve at pearwood.info  Tue Mar  4 12:23:20 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Mar 2014 22:23:20 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com>
Message-ID: <20140304112320.GQ28804@ando>

On Tue, Mar 04, 2014 at 11:03:35AM +0200, David Townshend wrote:

> My suggestion was to use c'x + 3' as an effective replacement for
> compile('x + 3', '', 'eval').  The use of a literal rather than compile
> would solve a few of the issues raised in Steven's original post.  I don't
> see why it would be any more misleading to eval a code literal than a
> compiled string.  

Agreed.


> I did suggest making it callable (i.e. thunk(x=3), but on
> second thoughts I'm not sure that's a good idea.  It adds complexity
> without and real benefit over eval.

I don't think calling the thunk is the right API. If we have this sort 
of functionality, I want calling a thunk to be treated like any other 
expression, thunk[0] or thunk+1. I don't want it to mean "evaluate this 
thunk". An example:


thunk = `lambda x: x + y`

def func(y, obj):
    return obj + 1

result = func(50, thunk(100))


(This is not a use-case, just an illustration.)

In this example, the expression "thunk(1000)" is delayed until inside 
the func scope. Inside that scope, we evaluate this expression:

    (lambda x: x + y)(100)

where y has the value 50, and bind the result of this to the name obj. 
The lambda part returns a closure, and calling that closure with 
argument 100 returns 150, so this is equivalent to binding obj=150. Then 
func continues, evaluates "obj + 1", and returns 151.



-- 
Steven

From steve at pearwood.info  Tue Mar  4 12:25:35 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Mar 2014 22:25:35 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <20140304110921.GP28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
Message-ID: <20140304112535.GR28804@ando>

On Tue, Mar 04, 2014 at 10:09:21PM +1100, Steven D'Aprano wrote:

> Agreed. What I have in my head is some vague concept that the Python 
> evaluation rules will somehow know when to evaluate the thunk and when 
> to treat it as an object, which is (as I understand it) what happens in 
> Algol. 

Since Algol is not an OOP language, that would be "value", not 
"object".



-- 
Steven

From rosuav at gmail.com  Tue Mar  4 13:07:46 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Mar 2014 23:07:46 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <20140304110921.GP28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
Message-ID: <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>

On Tue, Mar 4, 2014 at 10:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Agreed. What I have in my head is some vague concept that the Python
> evaluation rules will somehow know when to evaluate the thunk and when
> to treat it as an object, which is (as I understand it) what happens in
> Algol. Again, just thinking aloud, perhaps we do this:
>
> thunk = `some_expression`  # delays evaluation
> a = [0, 1 + thunk]  # evaluates thunk in the current scope
> b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
>                       # equivalent to `1 + some_expression`
> c = b[1]  # now evaluates the thunk object
> d = f(2, thunk)  # evaluates thunk in f's scope
> e = g(3, `thunk`)  # passes the un-evaluated thunk object to g
>
> Consider this just a sketch, and in no way fully thought out.
>
> (This will most definitely need a PEP.)

PEP can come later. First, let's get some solid use-cases, and start
looking at implications. The way it's described here, there's
effectively magic when you try to look at an object of this type,
which will break a lot of assumptions. Most people expect that:

foo = bar
assert foo is bar

to be a safe assumption, but if bar is a thunk, then it's getting
evaluated separately in each of those, and that's potentially going to
create different objects and/or even have side effects. That's going
to surprise people. On the flip side, that's something that could be
dealt with with a naming convention for thunks. We have _private,
__mangled, __magic__, anticollision_, CONSTANT, Class... maybe we
could have thunk__ or something. It's most confusable with magic and
anticollision, but since both of those are connected with specific
keywords, it's reasonably likely there'll be no actual confusion.

Specific downside: There's no way to actually pass an unevaluated
thunk around. Technically, `thunk__` will create a new wrapper thunk
and pass that along. That'll often have the same effect, but it won't
be quite identical (same as there's a difference between f(g) and
f(lambda x: g(x)) in that the second one lazily looks up g), so that
might cause confusion. But you could always special-case it: writing
`thunk__` will be guaranteed to transmit the thunk unchanged, and if
you actually mean to add another wrapper layer, use `(thunk__)` or
something instead.

The biggest thing to figure out is scoping. Does a thunk take a
snapshot of its enclosing scope (a closure), or is it an expression
that gets evaluated in the target namespace? The latter could be
extremely confusing, but the former's just what a nested function
does, so this'd just be a new lambda syntax.

Python's existing lambda syntax has its flaws and its detractors, but
it has one huge benefit over thunking: It exists. :) Thunking has to
get over that difference. I'd like to see some proposals.

ChrisA

From masklinn at masklinn.net  Tue Mar  4 13:50:19 2014
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 4 Mar 2014 13:50:19 +0100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
 <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
Message-ID: <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>

On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Mar 4, 2014 at 10:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Agreed. What I have in my head is some vague concept that the Python
>> evaluation rules will somehow know when to evaluate the thunk and when
>> to treat it as an object, which is (as I understand it) what happens in
>> Algol. Again, just thinking aloud, perhaps we do this:
>> 
>> thunk = `some_expression`  # delays evaluation
>> a = [0, 1 + thunk]  # evaluates thunk in the current scope
>> b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
>>                      # equivalent to `1 + some_expression`
>> c = b[1]  # now evaluates the thunk object
>> d = f(2, thunk)  # evaluates thunk in f's scope
>> e = g(3, `thunk`)  # passes the un-evaluated thunk object to g
>> 
>> Consider this just a sketch, and in no way fully thought out.
>> 
>> (This will most definitely need a PEP.)
> 
> PEP can come later. First, let's get some solid use-cases, and start
> looking at implications. The way it's described here, there's
> effectively magic when you try to look at an object of this type,
> which will break a lot of assumptions. Most people expect that:
> 
> foo = bar
> assert foo is bar
> 
> to be a safe assumption, but if bar is a thunk, then it's getting
> evaluated separately in each of those

Why? Either it's forced during assignment or both names map to
the same thunk, and are both forced when any of them is.

That could be during the identity check, but since both names refer to
the same thunk they can only yield the same value, so the identity check
needs not force the thunk. An equality test would likely force the
thunk.

> so that
> might cause confusion. But you could always special-case it: writing
> `thunk__` will be guaranteed to transmit the thunk unchanged, and if
> you actually mean to add another wrapper layer, use `(thunk__)` or
> something instead.

Is there a use case for actually thunking a thunk?

> The biggest thing to figure out is scoping. Does a thunk take a
> snapshot of its enclosing scope (a closure), or is it an expression
> that gets evaluated in the target namespace? The latter could be
> extremely confusing, but the former's just what a nested function
> does, so this'd just be a new lambda syntax.

That is essentially what a thunk is, at least in my experience: it is
conceptually a nullary memoized function, forced (evaluated/called) if
an actual value/object is ever needed but potentially thrown out during
reduction. The difference, under the proposed semantics, is that the
forcing of the thunk would be implicit where that of a function is
explicit (not sure that's a good idea in a strict language).

From python at mrabarnett.plus.com  Tue Mar  4 14:30:32 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 04 Mar 2014 13:30:32 +0000
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com>
Message-ID: <5315D578.3040803@mrabarnett.plus.com>

On 2014-03-04 09:03, David Townshend wrote:
>
>
>
> On Tue, Mar 4, 2014 at 9:03 AM, Andrew Barnert <abarnert at yahoo.com
> <mailto:abarnert at yahoo.com>> wrote:
>
>     On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com
>     <mailto:aquavitae69 at gmail.com>> wrote:
>>
>>     What about a new code literal object, e.g.
>>
>>         thunk = c'x + 3'
>>         thunk(x=2)
>>
>     Why does it need to be built from/look like a string? I think it
>     would be just as simple for the parser, and simpler for editors, and
>     less misleading for readers, if it used a different marker.
>
>     Not that I'm seriously suggesting backticks here, but...
>
>          thunk = `x + 3`
>
>
> The only real reason for it looking like a string is a shortage of
> symbols.  Backticks are off limits, and most other symbols are just
> plain ugly (e.g. @x + 3@ or $x + 3$) or already in use in a way that
> could lead to ambiguity (e.g. |x + 3|).  String-like quotes seem like a
> better option than the alternatives.
>
[snip]

I'm sure that Unicode could provide some characters/codepoints.

For example:

thunk = ?x + 3?


From rosuav at gmail.com  Tue Mar  4 15:14:09 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 01:14:09 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
 <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
 <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>
Message-ID: <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com>

On Tue, Mar 4, 2014 at 11:50 PM, Masklinn <masklinn at masklinn.net> wrote:
> On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote:
>> Most people expect that:
>>
>> foo = bar
>> assert foo is bar
>>
>> to be a safe assumption, but if bar is a thunk, then it's getting
>> evaluated separately in each of those
>
> Why? Either it's forced during assignment or both names map to
> the same thunk, and are both forced when any of them is.
>
> That could be during the identity check, but since both names refer to
> the same thunk they can only yield the same value, so the identity check
> needs not force the thunk. An equality test would likely force the
> thunk.

Most certainly not. Try this:

bar = `[1,2,3]`
foo = bar
spam = bar
assert foo is spam

Here's the evaluated version:

foo = [1,2,3]
spam = [1,2,3]
assert foo is spam

You can try this one out directly. They're not going to be the same
object - they'll be two separate lists. They will be equal, in this
case, but there's no guarantee of that either:

bar = `random.random()`

Separate evaluation of the same expression isn't guaranteed to have
the same result, AND it might have unexpected side effects:

bar = `whiskey.pop().drink()`

and you might find yourself underneath the bar before you know it.

ChrisA

From masklinn at masklinn.net  Tue Mar  4 15:55:37 2014
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 4 Mar 2014 15:55:37 +0100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
 <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
 <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>
 <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com>
Message-ID: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net>


On 2014-03-04, at 15:14 , Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, Mar 4, 2014 at 11:50 PM, Masklinn <masklinn at masklinn.net> wrote:
>> On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote:
>>> Most people expect that:
>>> 
>>> foo = bar
>>> assert foo is bar
>>> 
>>> to be a safe assumption, but if bar is a thunk, then it's getting
>>> evaluated separately in each of those
>> 
>> Why? Either it's forced during assignment or both names map to
>> the same thunk, and are both forced when any of them is.
>> 
>> That could be during the identity check, but since both names refer to
>> the same thunk they can only yield the same value, so the identity check
>> needs not force the thunk. An equality test would likely force the
>> thunk.
> 
> Most certainly not. Try this:
> 
> bar = `[1,2,3]`
> foo = bar
> spam = bar
> assert foo is spam
> 
> Here's the evaluated version:
> 
> foo = [1,2,3]
> spam = [1,2,3]
> assert foo is spam

I don't agree with this, again why would the thunk be evaluated twice?
If thunks are added to *delay* expression evaluation (which is what I
understood from Steven's messages) surely something akin to Haskell's
semantics is simpler to understand and implement. That is, instead
of thunks being sugar for:

    bar = lambda: expr

they're sugar for

    bar = memoize(lambda: expr)

> You can try this one out directly. They're not going to be the same
> object - they'll be two separate lists. They will be equal, in this
> case, but there's no guarantee of that either:
> 
> bar = `random.random()`
> 
> Separate evaluation of the same expression isn't guaranteed to have
> the same result

Obviously, but why would repeated evaluation of the expression be
desirable?

> AND it might have unexpected side effects:
> 
> bar = `whiskey.pop().drink()`
> 
> and you might find yourself underneath the bar before you know it.


From rosuav at gmail.com  Tue Mar  4 16:11:42 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 02:11:42 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
 <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
 <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>
 <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com>
 <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net>
Message-ID: <CAPTjJmqO7_h7k0qoLtcq59YKR5u6x9g1tWfccWW+OoZimoE=FA@mail.gmail.com>

On Wed, Mar 5, 2014 at 1:55 AM, Masklinn <masklinn at masklinn.net> wrote:
> I don't agree with this, again why would the thunk be evaluated twice?
> If thunks are added to *delay* expression evaluation (which is what I
> understood from Steven's messages) surely something akin to Haskell's
> semantics is simpler to understand and implement. That is, instead
> of thunks being sugar for:
>
>     bar = lambda: expr
>
> they're sugar for
>
>     bar = memoize(lambda: expr)

Okay. That's an interesting point, and a distinction from lambda.
Effectively, once a thunk is evaluated once, it devolves to its value.
That would be *extremely* interesting in the case of lazy evaluation -
you could actually make a ternary-if function:

value = true_expr if cond else false_expr

def if_(cond, true_thunk, false_thunk):
    if cond: return true_thunk
    return false_thunk

value = if_(cond, `true_expr`, `false_expr`)

There's still the questions of scoping, but now you have the
distinction between a thunk and a function. And if it's at all
possible, the thunk could even replace itself in memory with its
result - that would be massively implementation-dependent, but since
you can't look at the identity of the thunk itself anyway, it wouldn't
break anything. Effectively, as soon as you evaluate bar, it assigns
to bar whatever the expression evaluates to. (Which is simpler and
cleaner to explain than the memoization.)

ChrisA

From steve at pearwood.info  Tue Mar  4 16:19:59 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 02:19:59 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando>
 <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com>
 <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net>
 <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com>
 <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net>
Message-ID: <20140304151958.GS28804@ando>

On Tue, Mar 04, 2014 at 03:55:37PM +0100, Masklinn wrote:

> I don't agree with this, again why would the thunk be evaluated twice?
> If thunks are added to *delay* expression evaluation (which is what I
> understood from Steven's messages) surely something akin to Haskell's
> semantics is simpler to understand and implement. That is, instead
> of thunks being sugar for:
> 
>     bar = lambda: expr
> 
> they're sugar for
> 
>     bar = memoize(lambda: expr)

+1


-- 
Steven

From guido at python.org  Tue Mar  4 18:20:20 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 4 Mar 2014 09:20:20 -0800
Subject: [Python-ideas] PEP-257: drop recommendation for extra line at
 end of multi-line docstring
In-Reply-To: <2d6d1d67-bf89-4b33-acfe-4db2bd5e28e3@googlegroups.com>
References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com>
 <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com>
 <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com>
 <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com>
 <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com>
 <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com>
 <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com>
 <2d6d1d67-bf89-4b33-acfe-4db2bd5e28e3@googlegroups.com>
Message-ID: <CAP7+vJLF65FLjJ8j7M-ThDQVOSvKhNrKsY0pvv554io19cuBRA@mail.gmail.com>

Removed.


On Tue, Mar 4, 2014 at 8:51 AM, Simon Kennedy <sffjunkie at gmail.com> wrote:

> On Monday, 3 March 2014 18:55:17 UTC, Ziad Sawalha wrote:
>>
>>  Thanks, Guido.
>>
>>  I'll follow up with updates to common tools as I come across them (ex.
>> pep257: https://github.com/GreenSteam/pep257/pull/64).
>>
>>  Ziad
>>
>>
> The footnote's still in the PEP text.
>
> --Simon
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/44bbb0e4/attachment.html>

From greg.ewing at canterbury.ac.nz  Tue Mar  4 23:17:17 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Mar 2014 11:17:17 +1300
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
 <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz>
 <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com>
Message-ID: <531650ED.1050206@canterbury.ac.nz>

Andrew Barnert wrote:
> On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
>> If you've constructed it that way, you probably already know its square
>> root.
> 
> Usually, but not always.

In any case, the fact remains that any algorithm for
calculating the square root of a perfect square will
work just as well in any base. There's nothing special
about decimal in that regard.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Mar  4 23:31:41 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Mar 2014 11:31:41 +1300
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <20140304110921.GP28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando>
Message-ID: <5316544D.7050302@canterbury.ac.nz>

Steven D'Aprano wrote:
> What I have in my head is some vague concept that the Python 
> evaluation rules will somehow know when to evaluate the thunk and when 
> to treat it as an object, which is (as I understand it) what happens in 
> Algol.

But Algol has the benefit of static typing -- the procedure
being called explicitly declares whether the argument is to
be passed by name or value. Python has no idea about that at
compile time.

> b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
>                       # equivalent to `1 + some_expression`
> c = b[1]  # now evaluates the thunk object
> d = f(2, thunk)  # evaluates thunk in f's scope
> e = g(3, `thunk`)  # passes the un-evaluated thunk object to g

When exactly does implicit evaluation of a thunk object occur?
Does `b[1]` give you an unevaluated thunk object? What if b is
a custom sequence type implemented in Python -- how does its
__getitem__ method avoid evaluating the thunk object prematurely?

None of these problems occur in Algol, because its thunks are
not first-class values (you can't store them in arrays, etc.)
and it uses static type information to tell when to create
and evaluate them.

-- 
Greg

From rosuav at gmail.com  Tue Mar  4 23:40:09 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 09:40:09 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <5316544D.7050302@canterbury.ac.nz>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
Message-ID: <CAPTjJmrvTxA0rhHM=0H78KNr6-gtkP54vvgJpLDX+08w-JiG-Q@mail.gmail.com>

On Wed, Mar 5, 2014 at 9:31 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Does `b[1]` give you an unevaluated thunk object?

I would say that `b[1]` should create a brand new thunk object of that
expression. On evaluation of that, it'll look up b, subscript it, and
touch the thunk. When it does, it Schrodingers itself into a value and
that's that.

In my opinion, __getitem__ isn't a problem, but __setitem__ is.
Somewhere along the way, you have to be able to "hold" a thunk as a
thunk. I've no idea how that works when you pass it around from
function to function.

ChrisA

From solipsis at pitrou.net  Wed Mar  5 00:08:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 5 Mar 2014 00:08:12 +0100
Subject: [Python-ideas] Python3.3 Decimal Library Released
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <20140303175429.GL28804@ando>
 <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com>
 <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com>
 <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com>
 <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com>
 <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol>
 <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com>
 <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz>
 <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com>
 <531650ED.1050206@canterbury.ac.nz>
Message-ID: <20140305000812.7676feca@fsol>

On Wed, 05 Mar 2014 11:17:17 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Andrew Barnert wrote:
> > On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> >
> >> If you've constructed it that way, you probably already know its square
> >> root.
> > 
> > Usually, but not always.
> 
> In any case, the fact remains that any algorithm for
> calculating the square root of a perfect square will
> work just as well in any base. There's nothing special
> about decimal in that regard.

The point was not decimal vs. binary, but arbitrarily high precision
vs. fixed low precision.
(i.e. if you have a 1000-digit integer, just calling N ** 0.5 will give
you a very low-precision result compared to the integer's precision).

Regards

Antoine.



From carl.input at gmail.com  Wed Mar  5 00:09:02 2014
From: carl.input at gmail.com (Carl Smith)
Date: Tue, 4 Mar 2014 23:09:02 +0000
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
Message-ID: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>

The tentatively proposed idea here is using dollar signed expressions to
define 'bills'. A bill object is essentially an expression which can be
evaluated any number of times, potentially in different scopes.

The following expression [a bill literal] would be pointless, but would
define a bill that always evaluates to 1.

    $1

So, eval($1)==1.

Some better examples...

* assign a bill to `a` so that `a` will evaluate to the value of the name
`foo` any time that `a` is evaluated, in the scope of that evaluation

   a = $foo

* as above, but always plus one

   a = $foo + 1

* make `a` a bill that evaluates to the value of the name `foo` at the time
that `a` is evaluated, in that scope, plus the value of `bar` **at the time
and in the scope of the assignment to `a`**

   a = $foo + bar

Note. Similarly to mixing floats with ints, any expression that contains a
bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill
too. Passing a bill to eval should be the obvious way to get the value.

The point? It allows functions to accept bills to use internally. The
function would specify any names the bill can reference in the function's
API, like keywords.

def f(b): # the bill arg `b` can reference `item`
    for item in something:
        if eval(b): return True

f($item < 0)

You could also use a function call, for example `$foo()` would evaluate to
a bill that evaluates to a call to `foo` in the scope and at the time of
any evaluation of the bill.

I've no idea if this is even possible in Python, and have no hope of
implementing it, but thought I'd share :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/f5479771/attachment.html>

From rymg19 at gmail.com  Wed Mar  5 00:57:31 2014
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 4 Mar 2014 17:57:31 -0600
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
Message-ID: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com>

Only problem is that it looks a tad perlish...


On Tue, Mar 4, 2014 at 5:09 PM, Carl Smith <carl.input at gmail.com> wrote:

> The tentatively proposed idea here is using dollar signed expressions to
> define 'bills'. A bill object is essentially an expression which can be
> evaluated any number of times, potentially in different scopes.
>
> The following expression [a bill literal] would be pointless, but would
> define a bill that always evaluates to 1.
>
>     $1
>
> So, eval($1)==1.
>
> Some better examples...
>
> * assign a bill to `a` so that `a` will evaluate to the value of the name
> `foo` any time that `a` is evaluated, in the scope of that evaluation
>
>    a = $foo
>
> * as above, but always plus one
>
>    a = $foo + 1
>
> * make `a` a bill that evaluates to the value of the name `foo` at the
> time that `a` is evaluated, in that scope, plus the value of `bar` **at the
> time and in the scope of the assignment to `a`**
>
>    a = $foo + bar
>
> Note. Similarly to mixing floats with ints, any expression that contains a
> bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill
> too. Passing a bill to eval should be the obvious way to get the value.
>
> The point? It allows functions to accept bills to use internally. The
> function would specify any names the bill can reference in the function's
> API, like keywords.
>
> def f(b): # the bill arg `b` can reference `item`
>     for item in something:
>         if eval(b): return True
>
> f($item < 0)
>
> You could also use a function call, for example `$foo()` would evaluate to
> a bill that evaluates to a call to `foo` in the scope and at the time of
> any evaluation of the bill.
>
> I've no idea if this is even possible in Python, and have no hope of
> implementing it, but thought I'd share :)
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/e7c9b5d9/attachment-0001.html>

From cs at zip.com.au  Wed Mar  5 01:06:10 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 5 Mar 2014 11:06:10 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com>
References: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com>
Message-ID: <20140305000610.GA3326@cskk.homeip.net>

On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Only problem is that it looks a tad perlish...

Of shellish.

But worse, does nothing even remotely loke what that does in perl
or shell (or basic or...).

I was going to remark that in my mind I would always see:

  a = $foo + 1

and (1) mentally bind the "$" to "foo" alone, not treat it as a
prefix to the whole expression and (2) expect "a" to evaluate right
now, regardess.

And then follow up that remark with a strong preference for forming
the who expression inside a clear construct so you know at the
outself that it is something for evaluation later.

And then realised that is already spelt "lambda".

I do take the point that this "bill" idea is meant to grant access
to the local scope when used.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Take a perfectly parallel uniform plank of constant density. Balance it exactly
at its centre on a frictionnless pivot. Place the hog at a known distance
from this pivot. Now add stones on the other end at exactly the same distance
until the plank is perfectly horizontal as measured by a precision tiltmeter.

Now guess the weight of the stones.
        - Bob Newhart on weighing a hog

From jbvsmo at gmail.com  Wed Mar  5 02:05:57 2014
From: jbvsmo at gmail.com (=?ISO-8859-1?Q?Jo=E3o_Bernardo?=)
Date: Tue, 4 Mar 2014 22:05:57 -0300
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
Message-ID: <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com>

>
> * assign a bill to `a` so that `a` will evaluate to the value of the name
> `foo` any time that `a` is evaluated, in the scope of that evaluation
>
>    a = $foo
>
> * as above, but always plus one
>
>    a = $foo + 1
>
> * make `a` a bill that evaluates to the value of the name `foo` at the
> time that `a` is evaluated, in that scope, plus the value of `bar` **at the
> time and in the scope of the assignment to `a`**
>
>    a = $foo + bar
>
>

You can do all that programmatically. No need for magic operators.
Check this module as a proof of concept:
https://github.com/jbvsmo/funcbuilder

foo = FuncBuilder()
a = foo + 1
# Then evaluate a with a function call

For it to do exactly what you want, you just need override the evaluation
function (Funcbuilder.__call__) to load names from current frame locals or
something.
BTW, I don't recommend using that on real code because it is extremely
experimental (unless you're a hipster... In this case, carry on).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/48623a83/attachment.html>

From abarnert at yahoo.com  Wed Mar  5 02:05:39 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 4 Mar 2014 17:05:39 -0800
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <5316544D.7050302@canterbury.ac.nz>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
Message-ID: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>

On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Steven D'Aprano wrote:
>> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol.
> 
> But Algol has the benefit of static typing -- the procedure
> being called explicitly declares whether the argument is to
> be passed by name or value. Python has no idea about that at
> compile time.

This is the main reason I think it's more productive to think of this in terms of Lisp-style quoting than Algol-style thunks. The fact that quoting/thunking gives you a code object instead of an AST (sexpr) is not significant here* (unless we're also considering adding macros). The fact that it gives you a first-class value, and that we can't use the "implicit casting" syntax that comes with static typing to magically evaluate the object at the right time, is critical.

This is basically the same problem I described trying to implement Boost-style auto lambdas in Python without C++-style implicit cast from lambda to function.

* It is significant _elsewhere_. In an AST, "b" is just a reference to a name; in a code object, we have to have already determined whether it's a reference to a fast, closure, or global name--unless you want to do something like automatically compile `b[1]` as if it were something like vars('b')[1], or to do something equivalent like add a LOAD_DYNAMIC opcode.

>> b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
>>                      # equivalent to `1 + some_expression`
>> c = b[1]  # now evaluates the thunk object
>> d = f(2, thunk)  # evaluates thunk in f's scope
>> e = g(3, `thunk`)  # passes the un-evaluated thunk object to g

There's really no way to cleanly distinguish the c, d, and e cases. There are really only two possibilities: Magic evaluation everywhere (so d is passing in eval(thunk) to f), or explicit evaluation (whether via eval or otherwise) only (so c is just binding the unevaluated thunk). I think either of those is potentially viable, but I don't think there's anything in between.

Note that you can always quote the implicit evaluation (if you go the first way) or explicitly eval anywhere you want (the second), so neither one really limits what you can do.
 
> When exactly does implicit evaluation of a thunk object occur?
> Does `b[1]` give you an unevaluated thunk object? What if b is
> a custom sequence type implemented in Python -- how does its
> __getitem__ method avoid evaluating the thunk object prematurely?

You're creating a new thunk/quoting a new expression inside the backticks, so the issue doesn't arise. You're creating a new code object that looks up b and indexes it. The __getitem__ call doesn't happen until that new code object is evaluated.

> None of these problems occur in Algol, because its thunks are
> not first-class values (you can't store them in arrays, etc.)
> and it uses static type information to tell when to create
> and evaluate them.



From abarnert at yahoo.com  Wed Mar  5 02:16:38 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 4 Mar 2014 17:16:38 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com>
References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
 <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com>
Message-ID: <8B8381FA-AC52-4D5A-A01C-285B21E2B5E9@yahoo.com>

On Mar 4, 2014, at 17:05, Jo?o Bernardo <jbvsmo at gmail.com> wrote:

>> * assign a bill to `a` so that `a` will evaluate to the value of the name `foo` any time that `a` is evaluated, in the scope of that evaluation
>> 
>>    a = $foo
>> 
>> * as above, but always plus one
>> 
>>    a = $foo + 1
>> 
>> * make `a` a bill that evaluates to the value of the name `foo` at the time that `a` is evaluated, in that scope, plus the value of `bar` **at the time and in the scope of the assignment to `a`**
>> 
>>    a = $foo + bar
> 
> 
> You can do all that programmatically. No need for magic operators.
> Check this module as a proof of concept: https://github.com/jbvsmo/funcbuilder
> 
> foo = FuncBuilder()
> a = foo + 1
> # Then evaluate a with a function call
> 
> For it to do exactly what you want, you just need override the evaluation function (Funcbuilder.__call__) to load names from current frame locals or something.
> BTW, I don't recommend using that on real code because it is extremely experimental (unless you're a hipster... In this case, carry on).

I posted something similar in the first of the many improving-lambda threads this month, but it had a lot of little problems and one big one.

The big one is that not every expression can be caught by operator overloading (foo[i] is fine, but lst[foo] isn't). And call expressions are one of the most important kinds of expression, but you can't catch that--or, rather, if you _do_ catch it, then you have no way to call the resulting object.

The same idea in C++ doesn't have that last problem because C++ is statically typed, and you can use an implicit cast from autolambda object to function object to get from an object whose call operator builds a call expression to one whose call operator evaluates the expression. But that doesn't work in Python. 

I wrote a blog post that gets into this further, but I can't find the link from my phone. I included it in an earlier message in the thread.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/07f6bb2f/attachment-0001.html>

From mertz at gnosis.cx  Wed Mar  5 02:23:13 2014
From: mertz at gnosis.cx (David Mertz)
Date: Tue, 4 Mar 2014 17:23:13 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305000610.GA3326@cskk.homeip.net>
References: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com>
 <20140305000610.GA3326@cskk.homeip.net>
Message-ID: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>

On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote:

> On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > Only problem is that it looks a tad perlish...Of shellish.
>
> But worse, does nothing even remotely loke what that does in perl
> or shell (or basic or...).
>
> I was going to remark that in my mind I would always see:
>
>   a = $foo + 1
>
> and (1) mentally bind the "$" to "foo" alone, not treat it as a
> prefix to the whole expression and (2) expect "a" to evaluate right
> now, regardess.
>

I think I could be +0 for a bit different spelling that *is* actually
shell-ish.  I.e. as a way to handle snippets, this doesn't seem so bad:

  foo = 1
  a = $(foo + 1)
  b = eval(a) * 6
  c = $(foo + a/2)
  print(c, eval(c))
  # <thunk object at NNNN> 2
  print(a + 1)
  # TypeError, incompatible types for '+' operation, thunk and int

That is, the whole "thunk" or really just "code-snippet" could be a
shorthand way to refer to a segment of (dynamically scoped, basically) code
we might use in different places.  It's a lot like Ruby blocks, of course,
but the difference is that it's not only in the final position of a call,
but anywhere anything might be used or passed.

On the other hand, this isn't *really* different from just putting snippets
in strings like we can do now.  The one difference I envision is that maybe
these "thunks" would be completely resolved by an eval(), unlike a string.
 That is, 'c' is a thunk, but it's a thunk that contains another thunk.  So
the eval() function becomes a recursive-unthunk.

 Well, maybe I'm -0 rather than +0.  I'm definitely -1 on the "thunks taint
all expressions" approach suggested initially, which seems far too magical
and implicit.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/a8d49d75/attachment.html>

From cs at zip.com.au  Wed Mar  5 02:46:11 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 5 Mar 2014 12:46:11 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
Message-ID: <20140305014611.GA4396@cskk.homeip.net>

On 04Mar2014 17:23, David Mertz <mertz at gnosis.cx> wrote:
> On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote:
> > On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > > Only problem is that it looks a tad perlish...Of shellish.
> >
> > But worse, does nothing even remotely loke what that does in perl
> > or shell (or basic or...).
[...]
> I think I could be +0 for a bit different spelling that *is* actually
> shell-ish.  I.e. as a way to handle snippets, this doesn't seem so bad:
> 
>   foo = 1
>   a = $(foo + 1)

Definitely nicer. Still irrationally uncomfortable about the "$" though.

A thought, though it could break existing code (and nested tuples, alas):

    a = (( foo + 1 ))

Still, what does this mean?

    a = 3 + (( foo + 1 ))

I think that would need to be a syntax error, because I can't see it being
anything except nonsense otherwise.

That makes me think the (()) or $() is in the wrong place. Maybe:

    a := foo + 1

Hmm. hasn't that one already come up?

>   b = eval(a) * 6

This makes me unhappy. Eval parses a string and runs it, currently.
I would _want_ to read that as "compute a, then eval() the result".

I would prefer the occurence of "a" in an expression to cause it
to be evaluated then, much as happens already. It does mean that
there would be "silent" evaluation bombs lurking in expressions.
For example, given a doubly linked list (to make the "reference"
stuff glaring):

    node.right = node2
    node2.left = node

If node or node2 were thunks, this would be surprising to the reader.
Of course, we can surprise the reader already with __getattr__ etc.

But I think it would be better if:

    b = a

just made "b" a reference to the same thunk as "a".

Since I'm cavilling about "eval", how about just treating thunks
like other functions (which is how we spell code to be run later
presently) and have them be callables. So your "b = eval(a) * 6"
would just be:

    b = a() * 6

That way it is glaringly obvious that "a" happens now, rather than
earlier.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Disease and deprivation stalk the land like two giant....   stalking things.
        - Rowan Atkinson, _Black Adder the Third_

From mertz at gnosis.cx  Wed Mar  5 03:36:40 2014
From: mertz at gnosis.cx (David Mertz)
Date: Tue, 4 Mar 2014 18:36:40 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305014611.GA4396@cskk.homeip.net>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
 <20140305014611.GA4396@cskk.homeip.net>
Message-ID: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>

On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote:

> >   foo = 1
> >   a = $(foo + 1)
> Definitely nicer. Still irrationally uncomfortable about the "$" though.
> A thought, though it could break existing code (and nested tuples, alas):
>
>     a = (( foo + 1 ))
>

That looks like unresolvable ambiguity to me.  I confess that I am more
comfortable with '$(...)' because I'm one of those folks who actually likes
bash, and uses that spelling often over there (where the meaning isn't the
*same* as this, but is enough similar for the meaning to carry over)


> Still, what does this mean?
>     a = 3 + (( foo + 1 ))
> I think that would need to be a syntax error, because I can't see it being
> anything except nonsense otherwise.
>

In my example I made it a TypeError on the grounds that a thunk and and int
aren't things that can be added together.  That makes more sense to me than
a SyntaxError, since a thunk is a perfectly good expression (or would be if
the feature is added).

Besides, you really need to allow thunks in expressions, since sometimes
they make sense as values to operate on, e.g. (as I wrote before):

  print(foo, 'foo', $(foo))  # Using my bash-like syntax
  # 1 foo <thunk object ...>

You could also, for example, have a type of object that knew how to operate
on a thunk with operators:

  class Thunker(object):
      def __radd__(self, other):
          if isinstance(other, thunk):
              ... return something sensible ...

  thunker = Thunker()
  a = thunker + $(foo + bar)

>   b = eval(a) * 6
> This makes me unhappy. Eval parses a string and runs it, currently.
> I would _want_ to read that as "compute a, then eval() the result".



> b = a() * 6


I don't think I would want a thunk to be *exactly* a callable. That feels
wrong to me.  But I can see that overloading the meaning of eval() to
operate on either a string or a thunk might feel odd. *Except* that eval()
is *already* overloaded in a similar manner:

  >>> exp = compile("1+2", "<string>", "eval")
  >>> exp, eval(exp)
  (<code object <module> at 0x100730ae0, file "<string>", line 1>, 3)

However, what if it was spelled differently from eval(), e.g.:

  b = unthunk(a) * 6

OK, I don't actually like that name, but you see what I mean that other
names are perfectly possible.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/46581697/attachment.html>

From harrismh777 at gmail.com  Wed Mar  5 03:36:17 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Tue, 4 Mar 2014 18:36:17 -0800 (PST)
Subject: [Python-ideas] Python3.3 Decimal Library Released
In-Reply-To: <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com>
References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com>
 <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com>
 <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com>
Message-ID: <9858a1ce-2062-4460-9b8e-b13ee9f1c388@googlegroups.com>



On Tuesday, March 4, 2014 2:01:44 AM UTC-6, Paul Moore wrote:
>
> Regardless of anything else that comes out of this thread, thanks for 
> doing that. 
> Paul 
>
>
Thank you, you are most welcome.
marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/196a4a13/attachment-0001.html>

From cs at zip.com.au  Wed Mar  5 04:49:45 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 5 Mar 2014 14:49:45 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
Message-ID: <20140305034945.GA67087@cskk.homeip.net>

On 04Mar2014 18:36, David Mertz <mertz at gnosis.cx> wrote:
> On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote:
> > >   foo = 1
> > >   a = $(foo + 1)
> > Definitely nicer. Still irrationally uncomfortable about the "$" though.
> > A thought, though it could break existing code (and nested tuples, alas):
> >
> >     a = (( foo + 1 ))
> 
> That looks like unresolvable ambiguity to me.

Me too.

> I confess that I am more
> comfortable with '$(...)' because I'm one of those folks who actually likes
> bash, and uses that spelling often over there (where the meaning isn't the
> *same* as this, but is enough similar for the meaning to carry over)

I write a lot of shell scripts too.

> > Still, what does this mean?
> >     a = 3 + (( foo + 1 ))
> > I think that would need to be a syntax error, because I can't see it being
> > anything except nonsense otherwise.
> 
> In my example I made it a TypeError on the grounds that a thunk and and int
> aren't things that can be added together.  That makes more sense to me than
> a SyntaxError, since a thunk is a perfectly good expression (or would be if
> the feature is added).
> 
> Besides, you really need to allow thunks in expressions, since sometimes
> they make sense as values to operate on, e.g. (as I wrote before):
> 
>   print(foo, 'foo', $(foo))  # Using my bash-like syntax
>   # 1 foo <thunk object ...>
> 
> You could also, for example, have a type of object that knew how to operate
> on a thunk with operators:
> 
>   class Thunker(object):
>       def __radd__(self, other):
>           if isinstance(other, thunk):
>               ... return something sensible ...
> 
>   thunker = Thunker()
>   a = thunker + $(foo + bar)

Hmm, yes. Ok. We should get TypeErrors for free except where some
operator has been designed.

> >   b = eval(a) * 6
> > This makes me unhappy. Eval parses a string and runs it, currently.
> > I would _want_ to read that as "compute a, then eval() the result".
> 
> > b = a() * 6
> 
> I don't think I would want a thunk to be *exactly* a callable. That feels
> wrong to me.

Can you be more precise? It seems like exactly what's going on, semanticly.
Except that there's no notion of parameters.

> But I can see that overloading the meaning of eval() to
> operate on either a string or a thunk might feel odd. *Except* that eval()
> is *already* overloaded in a similar manner:
> 
>   >>> exp = compile("1+2", "<string>", "eval")
>   >>> exp, eval(exp)
>   (<code object <module> at 0x100730ae0, file "<string>", line 1>, 3)

Hmm. The lack of params makes eval a better match then.

> However, what if it was spelled differently from eval(), e.g.:
> 
>   b = unthunk(a) * 6
> 
> OK, I don't actually like that name, but you see what I mean that other
> names are perfectly possible.

How about:

  a.eval()

Don't make a new public function, give thunks a method.

Are we still intending thunks to be effectively a calling-scope
closure? That seems subject to being fragile: you can define a thunk
far from where it is called/evaled and therefore keeping it in sync
with the user's scope is less solid.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Steve is going for the pink ball - and for those of you who are
watching in black and white, the pink is next to the green.
- Snooker commentator 'Whispering' Ted Lowe

From harrismh777 at gmail.com  Wed Mar  5 04:42:28 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Tue, 4 Mar 2014 19:42:28 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
Message-ID: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>

Greetings,

   The purpose of this idea is to expand a bit on the decimal default idea
which I submitted previously. In this idea I want to suggest the human 
idea of a *python number.* The concept is very simple, yet may have far
reaching implications not only for future python(s) but also for the wider
adaptation of python in the greater academic and professional communities.

   The idea of *python number* means that there are no types, no limits, no 
constraints, and that all *python numbers *are dynamic. The upper level
syntax is also very simple; all* python numbers *are simply human.

   My influence for this preference is rooted in the Rexx programming 
language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx
programming language is dynamic, and has no types. To put it more 
succinctly for those of you who have not used Rexx, the only data type
is a string of characters (that's it).  *Rexx numbers* are simply those 
strings of characters that may be interpreted as a valid *Rexx number.*

http://books.google.com/books?id=cNiVqFmPs8AC&pg=PA100&lpg=PA100&dq=rexx+numbers&source=bl&ots=SNv00ARBqU&sig=cbRb2pqCsZpYhIXtGUXTDxmtqkw&hl=en&sa=X&ei=OpQWU-XuPMT70gHhooG4BA&ved=0CDcQ6AEwAQ#v=onepage&q=rexx%20numbers&f=false
 
   The Python language might be changed to adopt the *python number*
concept for *all math processing*, unless explicitly modified. This goes 
somewhat beyond using decimal floating point as a default numerical 
type.  It means using human numeric expressions that meet human 
expectation for numeric processing by default.

   Under the covers (whatever we mean by that) processing is handled
by decimal.Decimal, unless explicitly modified. What does this mean for
python users in general?  Well, no more worrying about types at all... no 
ints,
no floats, no rationals, no irrationals, no fractions, and certainly no 
binaries.
In short, if its a human number, its a *python number.*

   I am expecting that (just as in Rexx numbers) defining very clearly what
is a *python number*  will be key for wide adaptation of the concept. But 
there
should be no surprises for users, particularly average users. Anyone with 
a middle school expectation of a numeric format should be able to use 
*python numbers *without surprises.  However, for advanced users the full
interface should be available (as is the context for Decimal) through coding
based on knowledge and experience, yet the default context for Decimal 
should be based on average users in most environments and use cases.

   It is my belief that Python should lead the way in the twenty-first 
century 
for advanced computation for academic, professional, business, and 
scientific 
communities. There is a 40 year momentum for embedded binary floats &
doubles, also numeric types generally,  but it is time to move forward. The
technology is ready, and the need is great.  Let's do it.

{shameless plug}
pdeclib  https://pypi.python.org/pypi/pdeclib

Thank you for your consideration.   Good evening.

Mark H Harris
marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/635e25ea/attachment.html>

From python at mrabarnett.plus.com  Wed Mar  5 05:06:38 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 05 Mar 2014 04:06:38 +0000
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305014611.GA4396@cskk.homeip.net>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
 <20140305014611.GA4396@cskk.homeip.net>
Message-ID: <5316A2CE.2040509@mrabarnett.plus.com>

On 2014-03-05 01:46, Cameron Simpson wrote:
> On 04Mar2014 17:23, David Mertz <mertz at gnosis.cx> wrote:
>> On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote:
>> > On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > > Only problem is that it looks a tad perlish...Of shellish.
>> >
>> > But worse, does nothing even remotely loke what that does in perl
>> > or shell (or basic or...).
> [...]
>> I think I could be +0 for a bit different spelling that *is* actually
>> shell-ish.  I.e. as a way to handle snippets, this doesn't seem so bad:
>>
>>   foo = 1
>>   a = $(foo + 1)
>
> Definitely nicer. Still irrationally uncomfortable about the "$" though.
>
> A thought, though it could break existing code (and nested tuples, alas):
>
>      a = (( foo + 1 ))
>
That's not a tuple. It's equivalent to:

      a = foo + 1

[snip]


From cs at zip.com.au  Wed Mar  5 06:15:34 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 5 Mar 2014 16:15:34 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <5316A2CE.2040509@mrabarnett.plus.com>
References: <5316A2CE.2040509@mrabarnett.plus.com>
Message-ID: <20140305051534.GA75905@cskk.homeip.net>

On 05Mar2014 04:06, MRAB <python at mrabarnett.plus.com> wrote:
> On 2014-03-05 01:46, Cameron Simpson wrote:
> >>  a = $(foo + 1)
> >
> >Definitely nicer. Still irrationally uncomfortable about the "$" though.
> >A thought, though it could break existing code (and nested tuples, alas):
> >     a = (( foo + 1 ))
> >
> That's not a tuple. It's equivalent to:
>      a = foo + 1

I know that. I should have said: though the below looks nice, in more complicated forms it fights with tuples, eg:

  ((1,2,3),(4,5,6))
  (((1,2,3),(4,5,6)))

:-(
-- 
Cameron Simpson <cs at zip.com.au>

Theoretical Physicist,N.:A physicist whose existence is postulated, to make
the numbers balance but who is never actually observed in the laboratory.

From ron3200 at gmail.com  Wed Mar  5 06:25:42 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Tue, 04 Mar 2014 23:25:42 -0600
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>
Message-ID: <lf6cg9$rpp$1@ger.gmane.org>



On 03/04/2014 07:05 PM, Andrew Barnert wrote:
> On Mar 4, 2014, at 14:31, Greg
> Ewing<greg.ewing at canterbury.ac.nz>  wrote:

>>> Steven D'Aprano wrote:

>>>>> What I have in my head is some vague concept that the Python
>>>>> evaluation rules will somehow know when to evaluate the thunk
>>>>> and when to treat it as an object, which is (as I understand it)
>>>>> what happens in Algol.

>>> But Algol has the benefit of static typing -- the procedure being
>>> called explicitly declares whether the argument is to be passed by
>>> name or value. Python has no idea about that at compile time.

> This is the main reason I think it's more productive to think of this in
> terms of Lisp-style quoting than Algol-style thunks. The fact that
> quoting/thunking gives you a code object instead of an AST (sexpr) is
> not significant here* (unless we're also considering adding macros). The
> fact that it gives you a first-class value, and that we can't use the
> "implicit casting" syntax that comes with static typing to magically
> evaluate the object at the right time, is critical.

> This is basically the same problem I described trying to implement
> Boost-style auto lambdas in Python without C++-style implicit cast from
> lambda to function.

In a experimental language I'm writing, I use a concept I call "Context 
Resolution" to resolve objects to expected kinds of objects.  The expected 
type/kind is determined in the context of how things are used together 
rather than by how they are defined.  (That allows everything to be 
objects).  Keywords, Names, Expressions, and CodeBlocks, etc...


In python it would probably depend on AttributeError instead of the type.

If an object doesn't have the needed attribute, then it could try calling a 
different method, possibly __resolve__.  Then retry the attribute lookup 
again on the result.

If there's no __resolve__ attribute, then the AttributeError would be 
raised as usual.  The chained __resolve__ resolution attempts would also 
give a useful exception backtrace.

Cheers,
    Ron


From mertz at gnosis.cx  Wed Mar  5 07:54:30 2014
From: mertz at gnosis.cx (David Mertz)
Date: Tue, 4 Mar 2014 22:54:30 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305034945.GA67087@cskk.homeip.net>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <20140305034945.GA67087@cskk.homeip.net>
Message-ID: <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>

On Tue, Mar 4, 2014 at 7:49 PM, Cameron Simpson <cs at zip.com.au> wrote:

> > >   b = eval(a) * 6
> > > This makes me unhappy. Eval parses a string and runs it, currently.
> > > I would _want_ to read that as "compute a, then eval() the result".
> >
> > > b = a() * 6
> >
> > I don't think I would want a thunk to be *exactly* a callable. That feels
> > wrong to me.
>
> Can you be more precise? It seems like exactly what's going on, semanticly.
> Except that there's no notion of parameters.
>

As *I* am thinking of it, a "thunk" is really just like a C macro.  Not
even really like a Lisp macro.  So it's not like a function in that it
doesn't define a scope, doesn't have a call stack, etc.  It's just "pretend
I typed these other literal symbols here." Maybe I should stop calling the
idea a thunk, and just call it a macro.

However, I'm pulled in several directions here.  On the one hand, if it
really is essentially the same thing as a code object, I'm not convinced we
actually need syntax for it.  That is, what's really the point of having

  a = $(expr)  # or `expr`, or `(expr), or c"expr"

If it's simply a slightly shorter way of spelling:

  a = compile(expr, "<string>", "eval")

One thing about a code object is that it is NOT callable.  Which is to say,
that it's not usable as a callback.  If this other thing (thunk, macro,
whatever) is callable, it's usable as a callback; albeit a callback with
zero arguments, which again is of limited purpose, and not really that much
shorter than lambda, e.g.:

  b = Button(text="click me", command=lambda: print("Clicked!"))

Under the callable-thunk spelling, we might have:

  b = Button(text="click me", command=$(print("Clicked!")))

All of this is moving me towards the -0, or even -0.5 on my own idea (even
though I like my spelling in principle, but I'm having trouble seeing why I
actually need it).

  a.eval()
> Don't make a new public function, give thunks a method.
>

Sure, that's a nice enough spelling, but what do we really get over code
objects there?

Actually, here's some short and trivial code in existing Python that seems
to already get *everything* we've discussed in this thread:

>>> class Thunk(object):
...     def __init__(self, s):
...         self.code = compile(s, "<thunk>", "eval")
...     def __call__(self, **kws):
...         return eval(self.code, globals(), kws)
...     eval = __call__
...
>>> a = Thunk("foo + bar")
>>> for foo, bar in enumerate(range(5,8)):
...     a()
...
5
7
9
>>> for foo, bar in enumerate(range(5,8)):
...     a.eval()
...
5
7
9
>>> a(foo=10, bar=20)
30

Other than using the '$' which I'm mixed about anyway, and saving two quote
symbols, my Thunk class seems to be exactly what I had proposed (with
Cameron's variations).

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/8e680b50/attachment.html>

From rosuav at gmail.com  Wed Mar  5 07:57:10 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 17:57:10 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
Message-ID: <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com>

On Wed, Mar 5, 2014 at 2:42 PM, Mark H. Harris <harrismh777 at gmail.com> wrote:
>    The idea of python number means that there are no types, no limits, no
> constraints, and that all python numbers are dynamic. The upper level
> syntax is also very simple; all python numbers are simply human.
>
>    My influence for this preference is rooted in the Rexx programming
> language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx
> programming language is dynamic, and has no types. To put it more
> succinctly for those of you who have not used Rexx, the only data type
> is a string of characters (that's it).  Rexx numbers are simply those
> strings of characters that may be interpreted as a valid Rexx number.

I've used REXX extensively. (It's the native scripting language of
OS/2, which I first met in the early 1990s and am still using - albeit
usually in a VM under Linux these days - as it's still a good thing,
just a little left-behind.) There are two huge downsides to that kind
of proposal.

1) Performance. Huge huge performance hit. I can take Pike to OS/2, an
unoptimized build on an unsupported platform, and absolutely cream a
REXX program at any sort of computational work. By specifically
working with integers rather than "numbers", I can run code blazingly
fast.

2) Accuracy. With REXX, all arithmetic is governed by a single setting
of NUMERIC DIGITS, which specifies how many digits of accuracy you
want to preserve. (Python can improve granularity with separate
contexts, but the same issue will still apply - contexts just let you
use different NUMERIC DIGITS settings simultaneously in one program.)
Set it too high and performance suffers because the system has to
calculate more than you actually care about. Set it too low and
accuracy suffers. Even when you're working with integers, going past
DIGITS means it goes exponential.

3) Not so huge a downside, but also worth considering: REXX doesn't
have complex number support at all. You have to simulate it with two
variables. There's no way for the "Python number" system to
intrinsically handle complexes.

So really, what you're looking at is unifying int/float/Decimal into a
single data type, which is basically Decimal. I'd say merging int into
that is a bad idea, but you can get most of what you want to achieve
simply by encouraging the use of Decimal everywhere.

Maybe in some distant future version, the Python literal 1.234 will
represent a Decimal rather than a float. (And then there'd be other
consequences, like what you get when you divide one integer by
another.) But until then, all you can really do is encourage people to
explicitly use Decimal.

ChrisA

From harrismh777 at gmail.com  Wed Mar  5 08:34:15 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Tue, 4 Mar 2014 23:34:15 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com>
Message-ID: <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com>



On Wednesday, March 5, 2014 12:57:10 AM UTC-6, Chris Angelico wrote:
 

> 1) Performance. Huge huge performance hit. I can take Pike to OS/2, an 
> unoptimized build on an unsupported platform, and absolutely cream a 
> REXX program at any sort of computational work. By specifically 
> working with integers rather than "numbers", I can run code blazingly 
> fast. 
>
> 2) Accuracy. With REXX, all arithmetic is governed by a single setting 
> of NUMERIC DIGITS, which specifies how many digits of accuracy you 
> want to preserve. (Python can improve granularity with separate 
> contexts, but the same issue will still apply - contexts just let you 
> use different NUMERIC DIGITS settings simultaneously in one program.) 
>

hi Chris,  good to hear from another Rexx programmer! Back in the day 
(during 
my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370 
systems, and OS/2.  I too still have OS/2 running.  whoohoo!

Thanks for your comments. I would normally have agreed with you on this
but not in this case.  I do not think you are wrong, I just think you may 
have
underestimated the power of decimal.Decimal /its phenomenal--really--!

Don't take the Rexx reference too much to heart, I just wanted folks to know
where I got my influence on this thinking. Rexx was too slow... so was 
Decimal
until 3.3, and then, boy has it taken off so-to-speak!

Implementation details are way off in the future of course, but I am 
thinking 
AI processing on this idea. The Python system will sense and optimize the 
decimal contexts (local and otherwise) so that things are balanced between
speed and accuracy.  And I would note, that speed is not as important for
most python related aspects as user clarity and modern policy and account-
ability.  But, I'm not the least concerned for performance being a problem, 
because I have thoroughly tested decimal.Decimal and it flat screams, 
unlike 
our old friend Rexx. 

Thanks again for your feedback on the idea. Please give it some more 
thought, 
and let me know what you think of AI for load/speed/accuracy balancing.

PS  Have you considered the way things might have been if Palmisano
had played ball with Bill Gates and Steve Ballmer back in the day... OS/2 
would rule, gnu/linux might never have been invented period, and you and
I might not be having this conversation today!  ha!  Go Big Blue.

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/f2801e41/attachment-0001.html>

From abarnert at yahoo.com  Wed Mar  5 09:10:18 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 00:10:18 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
Message-ID: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>

My previous reply got google-grouped, so let me try again.


From: Mark H. Harris <harrismh777 at gmail.com>
Sent: Tuesday, March 4, 2014 7:42 PM


>? ?The Python language might be changed to adopt the python number

>concept for all math?processing, unless explicitly modified. This goes?
>somewhat beyond using decimal floating point as a default numerical?
>type. ?It means using human numeric expressions that meet human?
>expectation for numeric processing by default.


Different humans have different expectations in different situations. Trying to pick a single type that can handle all such expectations is an impossible dream.?

>? ?Under the covers (whatever we mean by that) processing is handled
>by decimal.Decimal, unless explicitly modified. What does this mean for
>python users in general? ?Well, no more worrying about types at all... no ints,
>no floats, no rationals, no irrationals, no fractions, and certainly no binaries.
>In short, if its a human number, its a python number.


This is not really a meaningful concept. The types you dislike are not programming concepts that get in the way of human mathematics, they are human mathematical concepts. The integers, the rationals, and the reals all behave differently. And they're not the only types of numbers?Python handles complex numbers natively, and it's very easy to extend it with, say, quaternions or even arbitrary matrices that act like numbers.

The types that _are_ programming concepts?decimal and binary floats?are necessary, because you simply can't store real numbers in finite storage. And the very fact that they are inexact approximations means you can't ignore the types. For some uses, IEEE binary floats are best; for others, decimal floats are best; for others, fractions are best; for others, you even want to handle symbolic numbers like pi/2 exactly.

>? ?I am expecting that (just as in Rexx numbers) defining very clearly what

>is a python number? will be key for wide adaptation of the concept. But there
>should be no surprises for users, particularly average users. Anyone with?
>a middle school expectation of a numeric format should be able to use?
>python numbers without surprises.

Anyone with a middle school expectation will expect 1/3 to be a fraction?or, at least, something they can multiply by 3 to get exactly 1. Using an?inexact decimal float instead of an inexact binary float is no improvement at all. Sure, it's an improvement in some _other_ cases, like 0.123, but if you want to deal with 1/3, today you can do so explicitly by using, say, Fraction(1, 3), while in your world it will no longer be possible.

And if you take the obvious way around that, you run into the same problem with 2 ** 0.5. Normal humans who write that would expect to be able to square it and get back 2, not an approximation to 2.

>However, for advanced users the full
>interface should be available (as is the context for Decimal) through coding
>based on knowledge and experience, yet the default context for Decimal?
>should be based on average users in most environments and use cases.


Is there a problem with the current default context?

I think just using Decimal as it is in Python today gets you everything you're asking for. Sure, you might want a nicer way to type them and repr them, which goes back to the previous thread, but why do you need to get rid of all of the other types as well?


From masklinn at masklinn.net  Wed Mar  5 10:18:56 2014
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 5 Mar 2014 10:18:56 +0100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <5316544D.7050302@canterbury.ac.nz>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
Message-ID: <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>

On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Steven D'Aprano wrote:
>> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol.
> 
> But Algol has the benefit of static typing -- the procedure
> being called explicitly declares whether the argument is to
> be passed by name or value. Python has no idea about that at
> compile time.

That's not really a blocker though, Haskell thunks are implicit and not
type-encoded. A name may correspond to a (unforced) thunk or to a strict
value (an already forced thunk, whether through a previous implicit
forcing, through an explicit forcing ? a strict annotation ? or through
a decision of the strictness analyser).

>> b = [0, `1 + thunk`]  # delays evaluation and creates a thunk object
>>                      # equivalent to `1 + some_expression`
>> c = b[1]  # now evaluates the thunk object
>> d = f(2, thunk)  # evaluates thunk in f's scope
>> e = g(3, `thunk`)  # passes the un-evaluated thunk object to g
> 
> When exactly does implicit evaluation of a thunk object occur?
> Does `b[1]` give you an unevaluated thunk object? What if b is
> a custom sequence type implemented in Python -- how does its
> __getitem__ method avoid evaluating the thunk object prematurely?
> 
> None of these problems occur in Algol, because its thunks are
> not first-class values (you can't store them in arrays, etc.)
> and it uses static type information to tell when to create
> and evaluate them.

There are definitely difficulties in deciding how the decision
to force a thunk comes about.

From rosuav at gmail.com  Wed Mar  5 10:36:42 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 20:36:42 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com>
 <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com>
Message-ID: <CAPTjJmrV_b1Ez33sKONrso3N53psNd7=MjKsPMsu4vMSxQS49w@mail.gmail.com>

On Wed, Mar 5, 2014 at 6:34 PM, Mark H. Harris <harrismh777 at gmail.com> wrote:
> hi Chris,  good to hear from another Rexx programmer! Back in the day
> (during
> my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370
> systems, and OS/2.  I too still have OS/2 running.  whoohoo!
>
> Don't take the Rexx reference too much to heart, I just wanted folks to know
> where I got my influence on this thinking. Rexx was too slow... so was
> Decimal
> until 3.3, and then, boy has it taken off so-to-speak!

Yes, that's true. But I've just done some performance testing. REXX on
our OS/2 VM accounting server "Stanley" calculates 10000! in 14
seconds; Pike on the same hardware calculates 100000! in 4 seconds. (I
didn't bother calculating 100000! in REXX, didn't feel like waiting.)
In each case it was by this naive algorithm:

REXX: n=1; do i=1 to 10000; n=n*i; end
Pike: int n=1; for (int i=1;i<=100000;++i) n*=i;

Those were running on the same hardware, and going to a target an
order of magnitude higher took a fraction of the time. That's what
integer performance is like. (BTW, I set 'numeric digits' to something
just a bit more than the target. REXX was pretty fast if I left digits
low, and I could then look at the exponent to see what digits setting
I needed. In Pike's case, the int type supports arbitrary precision
anyway, so the net result is a fair comparison; I could write the
digits out to a file and get the exact same result.)

>From here on, I've switched computers to Sikorsky, who has more
languages and more power than Stanley has. Timings here aren't
technically comparable to the above ones, but they seem similar. I
guess VirtualBox is doing a decent job of staying out of the way. Pike
on Sikorsky:

> gauge {int n=1; for (int i=1;i<=100000;++i) n*=i;};
(3) Result: 3.394805453

Okay. Now, Python.

Python 3.4.0rc1+ (default:a124b981a7a3, Mar  3 2014, 01:30:30)
[GCC 4.7.2] on linux
>>> def fac(top):
    t=time.time()
    n=1
    for i in range(1,top+1):
        n*=i
    return time.time()-t
>>> fac(100000)
7.814447641372681

Roughly double Pike's time. (I suspect this may be largely because
Pike has an optimization for machine-word integers. The difference is
far starker if the algorithm used is less naive.) Now watch the result
of a switch of data type.

def fac_d_maxprec(top):
    from decimal import Decimal, getcontext, MAX_PREC
    getcontext().prec=MAX_PREC
    t=time.time()
    n=Decimal("1")
    for i in range(1,top+1):
        n*=i
    return time.time()-t

(Note that the multiplication is still with ints. Doing all the
multiplication with Decimals would mean a whole lot more calling of
the constructor, which wouldn't be a fair comparison. If all of Python
used Decimal, then range() would be returning Decimal.)

>>> fac_d_maxprec(100000)
20.71300506591797

I also wrote an alternative version of the above which pre-calculates
at low precision, then sets the precision to just enough, and
recalculates. The timings were statistically identical to the above.

So there you are: A three to one difference. That's pretty amazing,
actually - the fact that it's not *far far* worse is a tribute to the
folks who gave us this highly optimized decimal.Decimal
implementation! But it's still dramatically slower than integer
calculations, and as you can see from the Pike version, the integer
figures can be cut a long way too.

I'm not sure that I want to give that up.

> But, I'm not the least concerned for performance being a problem,
> because I have thoroughly tested decimal.Decimal and it flat screams, unlike
> our old friend Rexx.

Yes, that's true (although frankly, when I first met REXX on OS/2, it
screamed compared to doing the job myself in C - sure, I could do
integer calculations quicker in C, but if I wanted to go above the
size of a long - 1<<32 - that was pretty hard), but the screaming
isn't so impressive compared to a modern integer engine.

> PS  Have you considered the way things might have been if Palmisano
> had played ball with Bill Gates and Steve Ballmer back in the day... OS/2
> would rule, gnu/linux might never have been invented period, and you and
> I might not be having this conversation today!  ha!  Go Big Blue.

Hmm, I don't think so. Linux had a rather different purpose, back
then. But yeah, OS/2 was brilliant tech built in a hostile
environment... I'd like to see some of its best parts lifted onto a
Linux kernel, though (like the WorkPlace Shell - should be possible to
run that on top of X11). Alas, most of OS/2 now is in the "was great
in the 90s, but other things do the same job now" basket, and a
closed-source system that boasts little above a GNU/Linux stack isn't
really going to go anywhere much.

Still, it's hanging around. It's the best way I have for running
16-bit Windows programs, believe it or not. (And yes, I did try
running Pastel Accounting under Wine. Win-OS/2 still beats Wine
hands-down, probably because Wine developers don't see any reason to
bother with supporting anything so ancient.)

ChrisA

From shai at platonix.com  Wed Mar  5 11:16:12 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 12:16:12 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight
Message-ID: <201403051216.12392.shai@platonix.com>

Hi all,

This is my first post here, following a recommendation from Alexander 
Belopolsky to use this list, to try to convince the Python developers to 
reopen a ticket. I am a long-time Python user, and a Django committer.

http://bugs.python.org/issue13936 is a complaint about the fact that midnight 
-- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was 
closed as invalid, under the claim that """It is odd, but really no odder than 
"zero values" of other types evaluating to false in Boolean contexts""".

I would like to ask for this to be reconsidered; since the ticket was closed, 
two main arguments were given for this:

1) The practical argument (paraphrasing Danilo Bergen and Andreas Pelme): The 
current semantics is surprising and useless; users do not expect valid times 
to be falsey, and do not normally want code to have special behavior on 
midnight. Users who ask for Boolean evaluation of a variable that's supposed 
to hold a time value, usually write "if var" as shorthand for "if var is not 
None".

2) The principled argument (which I think is at the root of the practical 
argument); quoting myself from the ticket: """Midnight is not a "zero value", 
it is just a value. It does not have any special qualities analogous to those 
of 0, "", or the empty set. ... Midnight evaluting to false makes as much 
sense as date(1,1,1) -- the minimal valid date value -- evaluating to 
false""".

Thanks,
	Shai.

From p.f.moore at gmail.com  Wed Mar  5 11:23:31 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 10:23:31 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051216.12392.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
Message-ID: <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>

On 5 March 2014 10:16, Shai Berger <shai at platonix.com> wrote:
> http://bugs.python.org/issue13936 is a complaint about the fact that midnight
> -- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was
> closed as invalid, under the claim that """It is odd, but really no odder than
> "zero values" of other types evaluating to false in Boolean contexts""".
>
> I would like to ask for this to be reconsidered; since the ticket was closed,
> two main arguments were given for this:

Why on earth would anyone check the boolean value of a time???

Before looking at whether midnight should be false or true, I'd like
to be clear on why any real life code would care one way or the other.
It seems to me that the correct answer would be to change the code to
simply not rely on the boolean value of a time.

Paul

From masklinn at masklinn.net  Wed Mar  5 11:31:08 2014
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 5 Mar 2014 11:31:08 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
Message-ID: <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>

On 2014-03-05, at 11:23 , Paul Moore <p.f.moore at gmail.com> wrote:
> On 5 March 2014 10:16, Shai Berger <shai at platonix.com> wrote:
>> http://bugs.python.org/issue13936 is a complaint about the fact that midnight
>> -- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was
>> closed as invalid, under the claim that """It is odd, but really no odder than
>> "zero values" of other types evaluating to false in Boolean contexts""".
>> 
>> I would like to ask for this to be reconsidered; since the ticket was closed,
>> two main arguments were given for this:
> 
> Why on earth would anyone check the boolean value of a time???

Side-effect of e.g. an object with an optional (nullable) time field,
the developer didn't realise a time could be false-ish (because
seriously?) and then it turns out code along the lines of

    if event.start_time:
        # stuff

unexpectedly fails if start_time is midnight.

I've actually done something close recently, didn't expect
more_itertool's peekable iterator could be false-ish[0] because I'd
expect empty sequences to be falsy but not empty iterators.

It turned out to be useful and to simplify my code, but I can see
developers not considering such edge cases unless there's a clear
warning (as in ElementTree which IIRC clearly warns that an element
with no children is false-ish) especially for objects for which
false-ish-ness makes so little sense as a time.

[0] even though the behavior is documented.

From p.f.moore at gmail.com  Wed Mar  5 11:44:07 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 10:44:07 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
Message-ID: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>

On 5 March 2014 10:31, Masklinn <masklinn at masklinn.net> wrote:
> Side-effect of e.g. an object with an optional (nullable) time field,
> the developer didn't realise a time could be false-ish (because
> seriously?) and then it turns out code along the lines of
>
>     if event.start_time:
>         # stuff
>
> unexpectedly fails if start_time is midnight.

Yeah, I was classing that as "application bug" and it's easy to fix
with an "is not None". Agreed that the odd behaviour of time in a
boolean context is why this whole class of bugs exists, but it's only
a subclass if the wider problem that people shouldn't truth-test
values without thinking - an explicit test is always better (explicit
is better than implicit and all that).

I actually find the idea of truth-testing a value that's expected to
have a type that is always true *more* unintuitive than an explicit
"is not None", so I'd fix the above code regardless of what the truth
value of midnight is.

Paul

From shai at platonix.com  Wed Mar  5 11:59:40 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 12:59:40 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
Message-ID: <201403051259.40289.shai@platonix.com>

On Wednesday 05 March 2014 12:44:07 Paul Moore wrote:
> 
> I actually find the idea of truth-testing a value that's expected to
> have a type that is always true *more* unintuitive than an explicit
> "is not None", so I'd fix the above code regardless of what the truth
> value of midnight is.
> 

I can appreciate this point of view, but I think it is not the position 
generally taken by Python (if it were, Boolean evaluation of an object of a 
type that is always true would have raised an exception).

Shai.

From steve at pearwood.info  Wed Mar  5 12:18:23 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 22:18:23 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com>
Message-ID: <20140305111823.GU28804@ando>

On Tue, Mar 04, 2014 at 11:09:02PM +0000, Carl Smith wrote:
> The tentatively proposed idea here is using dollar signed expressions to
> define 'bills'. A bill object is essentially an expression which can be
> evaluated any number of times, potentially in different scopes.

Please don't invent "cute" names for things which already have names. 
What you are describing could be considered a thunk, or a macro, or a 
lazily-evaluated expression, depending on the precise details of how it 
works. But calling it a "bill" just because it starts with a $ just 
makes me cringe.


> The following expression [a bill literal] would be pointless, but would
> define a bill that always evaluates to 1.
> 
>     $1
> 
> So, eval($1)==1.

My personal feeling here is that if you have to explicitly call eval on 
the "bill" to evaluate it, it's not worth doing. If we were happy with 
that, we've already got compile(), or we have functions. Or just eval a 
string directly. My feeling is that evaluating the expression needs to 
be implicit, performed at need rather than up front, rather in the same 
way that short-circuiting operators don't actually evaluate the 
expression unless needed:

    x or expr  # expr is only evaluated if x is a falsey value


Yes, I know, the Zen of Python says that "explicit is better than 
implicit", but the Zen is intended as guidelines, not thought-blockers. 
We have perfectly good explicit idioms for delaying computations (eval 
and functions), to make the thunk/macro/whatever worth doing it has to 
offer something different.


> Some better examples...
> 
> * assign a bill to `a` so that `a` will evaluate to the value of the name
> `foo` any time that `a` is evaluated, in the scope of that evaluation
> 
>    a = $foo
> 
> * as above, but always plus one
> 
>    a = $foo + 1
>
> * make `a` a bill that evaluates to the value of the name `foo` at the time
> that `a` is evaluated, in that scope, plus the value of `bar` **at the time
> and in the scope of the assignment to `a`**
> 
>    a = $foo + bar

Hmmm. That implies that if you want an entire expression to have 
delayed evaluation, you have to tag everything with a sigil:

    a = $spam + $eggs - $spam*$eggs + $cheese*$spam

I think it is better to have syntax with delimiters, to turn delayed 
evaluation ON and OFF, rather than having to tag each and every 
sub-expression:

    a = `spam + eggs - spam*eggs + cheese*spam`

(disclaimer: using ` ` is just for the sake of illustration.)


> Note. Similarly to mixing floats with ints, any expression that contains a
> bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill
> too. Passing a bill to eval should be the obvious way to get the value.
> 
> The point? It allows functions to accept bills to use internally. The
> function would specify any names the bill can reference in the function's
> API, like keywords.

Well, this contradicts your previous point. If any expression that 
contains a "bill" is a "bill", then so is func(bill). So given that, 
your example here:

> def f(b): # the bill arg `b` can reference `item`
>     for item in something:
>         if eval(b): return True
> 
> f($item < 0)

would actually need to be written as:

eval(f($item < 0))



-- 
Steven

From steve at pearwood.info  Wed Mar  5 12:25:32 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 22:25:32 +1100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>
Message-ID: <20140305112532.GV28804@ando>

On Tue, Mar 04, 2014 at 05:05:39PM -0800, Andrew Barnert wrote:
> On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> > Steven D'Aprano wrote:
> >> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol.
> > 
> > But Algol has the benefit of static typing -- the procedure
> > being called explicitly declares whether the argument is to
> > be passed by name or value. Python has no idea about that at
> > compile time.

I'm not convinced that this really matters, but for the sake of the 
argument let's say it does.


> This is the main reason I think it's more productive to think of this 
> in terms of Lisp-style quoting than Algol-style thunks.

Can you give more detail please?


-- 
Steven

From oscar.j.benjamin at gmail.com  Wed Mar  5 12:29:10 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Mar 2014 11:29:10 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
Message-ID: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>

On 5 March 2014 08:10, Andrew Barnert <abarnert at yahoo.com> wrote:
> From: Mark H. Harris <harrismh777 at gmail.com>
>>
>>   I am expecting that (just as in Rexx numbers) defining very clearly what
>>is a python number  will be key for wide adaptation of the concept. But there
>>should be no surprises for users, particularly average users. Anyone with
>>a middle school expectation of a numeric format should be able to use
>>python numbers without surprises.
>
> Anyone with a middle school expectation will expect 1/3 to be a fraction--or, at least, something they can multiply by 3 to get exactly 1. Using an inexact decimal float instead of an inexact binary float is no improvement at all.

I actually think that it is an improvement. Most people are surprised
by the fact that just writing x = 0.1 causes a rounding error. Python
only has dedicated syntax for specifying binary floats in terms of
decimal digits meaning that there is no syntax for exactly specifying
non integer numbers. I would say that that is clearly a sub-optimal
situation.

I don't agree with Mark's proposal in this thread but I would like to
have decimal literals e.g. 1.23d, and I would also use Fraction
literals if available e.g. 1/3F.


Oscar

From steve at pearwood.info  Wed Mar  5 12:32:33 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 22:32:33 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <20140305014611.GA4396@cskk.homeip.net>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
 <20140305014611.GA4396@cskk.homeip.net>
Message-ID: <20140305113233.GW28804@ando>

On Wed, Mar 05, 2014 at 12:46:11PM +1100, Cameron Simpson wrote:

> That makes me think the (()) or $() is in the wrong place. Maybe:
> 
>     a := foo + 1

No good. That implies that there is no way to have an anonymous thunk. 
You can do this:

my_list = [x + 1, y + 2,   # eager evaluation
           $(z + 3),  # lazy evaluation
            ... ]

but would have to write:

temp := z + 3
my_list = [x + 1, y + 2, temp, ...]


Similarly for passing a thunk into a function.



> Since I'm cavilling about "eval", how about just treating thunks
> like other functions (which is how we spell code to be run later
> presently) and have them be callables. 

In which case, what's the point? What makes them different from regular 
functions?



-- 
Steven

From steve at pearwood.info  Wed Mar  5 12:43:44 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 22:43:44 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <20140305034945.GA67087@cskk.homeip.net>
 <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>
Message-ID: <20140305114344.GX28804@ando>

On Tue, Mar 04, 2014 at 10:54:30PM -0800, David Mertz wrote:

> As *I* am thinking of it, a "thunk" is really just like a C macro.  Not
> even really like a Lisp macro.  So it's not like a function in that it
> doesn't define a scope, doesn't have a call stack, etc.  It's just "pretend
> I typed these other literal symbols here." Maybe I should stop calling the
> idea a thunk, and just call it a macro.

I'm intrigued by this suggestion. I think that what I'm really after is 
two things: lazy evaluation, and dynamic scoping. I think a macro would 
give us both.

But hasn't Guido said No Macros at some point?


> However, I'm pulled in several directions here.  On the one hand, if it
> really is essentially the same thing as a code object, I'm not convinced we
> actually need syntax for it.  That is, what's really the point of having
> 
>   a = $(expr)  # or `expr`, or `(expr), or c"expr"
> 
> If it's simply a slightly shorter way of spelling:
> 
>   a = compile(expr, "<string>", "eval")

You can't write it like that. You have to wrap the expression is quotes 
and turn it into a string:

    a = compile("expr", "<string>", "eval")

which means you lose syntax highlighting. Perhaps the ability to get 
syntax highlighting is not sufficient to justify this idea.

Another disadvantage: the ever-present temptation to pass a 
user-generated string to compile:

    a = compile(template % some_string, "<string>", "eval")

If some_string came from an untrusted source, you are now the proud 
owner of a brand new code injection vulnerability. But with syntax, you 
cannot generate a thunk/macro except from code you write yourself. It's 
still code written by you, it's just that evaluation is delayed.



-- 
Steven

From cfkaran2 at gmail.com  Wed Mar  5 12:53:10 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Wed, 5 Mar 2014 06:53:10 -0500
Subject: [Python-ideas] Suggestion for standardized annotations
Message-ID: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>

Hi all, I wanted to make a suggestion for a convention on how to write annotations.  I know that PEP 8 says that the standard library won't use them in order to avoid setting a convention, but that could lead to confusion and conflicts in how they are used in non-standard libraries.  The reason I'm thinking about this is because of how docstrings are currently used.  If you use PLY (http://www.dabeaz.com/ply/) and also want to use something like Sphinx (http://sphinx-doc.org/), you're going to have a problem; PLY stores its rules in the docstring, while Sphinx parses it for documentation.  I want to prevent this problem for annotations.

My thought is that all annotations should be dictionaries.  The keys should all be unicode strings that are uppercase UUIDs (basically, what 'uuid.uuid1().hex.upper()' returns), and the values can be anything the programmer wants.  Each project can generate one (or more) UUIDs to put into the dictionary, and publicly document what the UUID's meaning is (preferably somewhere where a search engine can find it).  The advantage is that since UUIDs are unique, the number of false positives you'll get while searching for it should be low; I've tested this on a different mailing list I'm on, and the UUID I generated for it has 0 false positives, while picking up the complete discussion involving it.  

As an example, if project A had chosen the UUID B3D2AFE8A45A11E3AE24D49A20C52EF2 and project B chose the UUID C02D7C64A45A11E39DAFD49A20C52EF2, we might annotate a function as follows:

def foo(first : {
                    B3D2AFE8A45A11E3AE24D49A20C52EF2 : {"doc" : "English documentation"},
                    C02D7C64A45A11E39DAFD49A20C52EF2 : {"doc" : "expression : MINUS expression %prec UMINUS"}
                },
        second) -> {B3D2AFE8A45A11E3AE24D49A20C52EF2: {"type" : int, "doc" : "Typechecking for a linter"}}:

    pass

You can already see the downside of this approach; it's really, really verbose.  However, at least it avoids outright conflicts in usage that prevent usage of certain tools/projects together.

Note that I did consider using names as keys directly (e.g. 'doc').  However, that requires a strong, universal convention on what each key means.  Since we can't seem to figure that out for the docstring, I don't see why we should expect to be able to figure it out for any of proposed keys.  Moreover, the set of keys would need to be documented somewhere, the documentation kept up to date, etc.  It becomes a management nightmare.  UUIDs have the advantage the we just tell everyone how to generate their own, and let them go at it.  If someone wants to use a given project's docstring for their own purposes, it is up to them to keep the meaning the same.

Thoughts/suggestions?

Thanks,
Cem Karan

From steve at pearwood.info  Wed Mar  5 13:04:10 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 5 Mar 2014 23:04:10 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
Message-ID: <20140305120410.GY28804@ando>

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

> I don't agree with Mark's proposal in this thread but I would like to
> have decimal literals e.g. 1.23d, 

+1 on that. Although that will depend on how big a slowdown it causes to 
Python's startup time.


> and I would also use Fraction
> literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?


(By the way, I think it is somewhat amusing that Python not only has a 
built-in complex type, but also *syntax* for creating complex numbers, 
but no built-in support for exact rationals.)


-- 
Steven

From oscar.j.benjamin at gmail.com  Wed Mar  5 13:30:19 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Mar 2014 12:30:19 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305120410.GY28804@ando>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
Message-ID: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>

On 5 March 2014 12:04, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:
>
>> I don't agree with Mark's proposal in this thread but I would like to
>> have decimal literals e.g. 1.23d,
>
> +1 on that. Although that will depend on how big a slowdown it causes to
> Python's startup time.

If it is a significant slowdown then it can be a delayed import that
only occurs when a decimal literal is first encountered. Applications
that don't need decimal won't be slowed down. Applications that do
would have had to import it anyway.

>> and I would also use Fraction
>> literals if available e.g. 1/3F.
>
> Out of curiosity, why F rather than f?

In C and some other languages the f suffix indicates a numeric literal
that has type "float" e.g. "6.0f". You can use upper case there as
well but the convention is lower case and to include the .0 when it's
an integer. I just though that 3F looks sufficiently distinct from the
way it's typically done in C.


Oscar

From rosuav at gmail.com  Wed Mar  5 13:33:11 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 23:33:11 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305051534.GA75905@cskk.homeip.net>
References: <5316A2CE.2040509@mrabarnett.plus.com>
 <20140305051534.GA75905@cskk.homeip.net>
Message-ID: <CAPTjJmpf1XQA+PCi3=5KiT_=DE+TAVxMNqmHV_+keUxS+CLeHg@mail.gmail.com>

On Wed, Mar 5, 2014 at 4:15 PM, Cameron Simpson <cs at zip.com.au> wrote:
> On 05Mar2014 04:06, MRAB <python at mrabarnett.plus.com> wrote:
>> On 2014-03-05 01:46, Cameron Simpson wrote:
>> >>  a = $(foo + 1)
>> >
>> >Definitely nicer. Still irrationally uncomfortable about the "$" though.
>> >A thought, though it could break existing code (and nested tuples, alas):
>> >     a = (( foo + 1 ))
>> >
>> That's not a tuple. It's equivalent to:
>>      a = foo + 1
>
> I know that. I should have said: though the below looks nice, in more complicated forms it fights with tuples, eg:
>
>   ((1,2,3),(4,5,6))
>   (((1,2,3),(4,5,6)))

Bikeshedding the syntax without expressing an opinion on the feature:
Using {{ }} would be safer. The inner one might be interpreted as a
dict or a set, but neither of those can go into a set, so it'd be
illogical. Whether the similarity with C-style block delimiters is a
good thing or a bad thing remains to be seen :)

ChrisA

From oscar.j.benjamin at gmail.com  Wed Mar  5 13:34:09 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Mar 2014 12:34:09 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051259.40289.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <201403051259.40289.shai@platonix.com>
Message-ID: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>

On 5 March 2014 10:59, Shai Berger <shai at platonix.com> wrote:
> On Wednesday 05 March 2014 12:44:07 Paul Moore wrote:
>>
>> I actually find the idea of truth-testing a value that's expected to
>> have a type that is always true *more* unintuitive than an explicit
>> "is not None", so I'd fix the above code regardless of what the truth
>> value of midnight is.
>
> I can appreciate this point of view, but I think it is not the position
> generally taken by Python (if it were, Boolean evaluation of an object of a
> type that is always true would have raised an exception).

I think it's clear that if this were new code being added to the
stdlib then the consensus would be that having midnight evaluate as
False is ridiculous.

The question is surely whether the issue is worth a backwards
compatibility break not whether the current behaviour is a good idea
(it clearly isn't).


Oscar

From carl.input at gmail.com  Wed Mar  5 13:41:10 2014
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 5 Mar 2014 12:41:10 +0000
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
Message-ID: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>

Just for the record, I never called them bills just because they contain a
dollar sign! I was thinking of a short word which describes an expression
who's evaluation changes depending on the local circumstances. Like a legal
bill, not a dollar bill.

The name macro doesn't really work, as it's only an expression, it's not a
code block.

---

The idea was never to start passing complex expressions around the place.
It was just to allow a function to take an expression containing about one
or two names, and evaluate the expression internally, so this...

func(lambda foo: foo > 0)

...can become...

func($foo > 0)

Other uses are there, and might be helpful from time to time, but it's
mainly about cleaning up APIs. I personally work with user facing Python
APIs all day, and users are generally programming interactively, so perhaps
my take is atypical.

---

What the hell is a thunk anyway? It's a horrible name.

---

On the jQuery like syntax, $(foo), that might work and might do away with
the issue of expressions containing bills always evaluating to bills. I
took that too far. Good point.

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5fa7acf1/attachment.html>

From rosuav at gmail.com  Wed Mar  5 13:42:14 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Mar 2014 23:42:14 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
Message-ID: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> I just though that 3F looks sufficiently distinct from the
> way it's typically done in C.

3F would be about -16C wouldn't it?

(Not painting any bike sheds when it's that cold, thanks!)

I'm not sure I like the idea of tagging the end of the expression.
Currently, the nearest Python has to that is the complex notation:

>>> 1+2j
(1+2j)

And that works just fine when considered to be addition:

>>> a=1
>>> b=2j
>>> a+b
(1+2j)

So really, what Python has is a notation for a j-suffixed float,
meaning a complex number with no real part. You can't do that with
Fraction:

>>> a=2
>>> b=3F
>>> a/b

What I'd prefer would be some modification of the division operator.
We currently have two ways to divide an integer by an integer:

>>> 1234/10
123.4
>>> 1234//10
123

Adding a third operator, defined only between two integers, would be
cleaner than tagging one of the integer literals. Exactly what that
operator would be is hard to say, but I'm sure there's a combination
that would look right and not be ambiguous.

>>> 1234-/-10
Fraction(617, 5)

I can't think of anything good, though.

ChrisA

From p.f.moore at gmail.com  Wed Mar  5 13:50:14 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 12:50:14 +0000
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
Message-ID: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>

On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote:
> Thoughts/suggestions?

I think the core/stdlib position is that agreeing conventions would be
better done once some real world experience of the practical issues
and benefits of annotations has been established. So while a proposal
like this is not without merit, it needs to be considered in the light
of how projects actually use annotations. Personally, I'm not aware of
any libraries that make significant use of annotations, so a good
first step would be to survey existing use, and summarise it here.
That would allow you to clarify your proposal in terms of exactly how
existing projects would need to modify their current code.

Of course, there's likely a chicken and egg problem here - projects
may be holding off using annotations through fear of issues caused by
clashes. But I'm not sure that a UUID-based proposal like the above
(which as you admit is very verbose, and not particularly user
friendly) would be more likely to encourage use.

If I were developing a library that would benefit from annotations, at
this point in time I'd probably just choose whatever conventions
suited me and go with those - likely marking the feature as "subject
to change" initially. Then, when people raised bug reports or feature
requests that asked for better interoperability, I'd look at how to
achieve that in conjunction with the other project(s) that clashed
with me.

Paul

From p.f.moore at gmail.com  Wed Mar  5 13:54:27 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 12:54:27 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <201403051259.40289.shai@platonix.com>
 <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
Message-ID: <CACac1F8oCRHWSo777p6NvkXv4i4wE8C+Nj2cSFWBdhZSnNNuBw@mail.gmail.com>

On 5 March 2014 12:34, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 5 March 2014 10:59, Shai Berger <shai at platonix.com> wrote:
>> On Wednesday 05 March 2014 12:44:07 Paul Moore wrote:
>>>
>>> I actually find the idea of truth-testing a value that's expected to
>>> have a type that is always true *more* unintuitive than an explicit
>>> "is not None", so I'd fix the above code regardless of what the truth
>>> value of midnight is.
>>
>> I can appreciate this point of view, but I think it is not the position
>> generally taken by Python (if it were, Boolean evaluation of an object of a
>> type that is always true would have raised an exception).
>
> I think it's clear that if this were new code being added to the
> stdlib then the consensus would be that having midnight evaluate as
> False is ridiculous.
>
> The question is surely whether the issue is worth a backwards
> compatibility break not whether the current behaviour is a good idea
> (it clearly isn't).

Precisely. Sorry I wasn't clear enough.

And FWIW, my opinion is that the problem is not worth a compatibility
break, because it's so easily solved (and the fixed code is probably
an improvement in any case).
Paul

From oscar.j.benjamin at gmail.com  Wed Mar  5 13:56:46 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Mar 2014 12:56:46 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
 <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>
Message-ID: <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com>

On 5 March 2014 12:42, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin
> <oscar.j.benjamin at gmail.com> wrote:
>> I just though that 3F looks sufficiently distinct from the
>> way it's typically done in C.
>
> 3F would be about -16C wouldn't it?
>
> (Not painting any bike sheds when it's that cold, thanks!)
>
> I'm not sure I like the idea of tagging the end of the expression.
> Currently, the nearest Python has to that is the complex notation:
>
>>>> 1+2j
> (1+2j)
>
> And that works just fine when considered to be addition:
>
>>>> a=1
>>>> b=2j
>>>> a+b
> (1+2j)
>
> So really, what Python has is a notation for a j-suffixed float,
> meaning a complex number with no real part. You can't do that with
> Fraction:
>
>>>> a=2
>>>> b=3F
>>>> a/b

>From a syntactic perspective Python doesn't have syntax for general
complex literals. There is only syntax for creating imaginary literals
and it is trivial to create a complex number by adding a real and an
imaginary one.

3F could create a Fraction with the integer value 3 so that a/b gives
a rational number:

>>> from fractions import Fraction as F
>>> a = 2
>>> b = F(3)
>>> a/b
Fraction(2, 3)

I don't understand why you say that can't be done.


Oscar

From kaiser.yann at gmail.com  Wed Mar  5 14:09:58 2014
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Wed, 5 Mar 2014 14:09:58 +0100
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
Message-ID: <CANUJvPXt7gbWt0btRe6hGesO2A7LC5_fnAqVkR771teLF_PZ6g@mail.gmail.com>

You could use a slight modification of sigtools.modifiers.annotate [1]
to create different objects with different function annotations. That
way programmers who only use one library have no change to do, and
those who use more only have to add a few lines devoid of the
verbosity of UUIDs.


[1] http://sigtools.readthedocs.org/en/latest/#sigtools.modifiers.annotate

On 5 March 2014 13:50, Paul Moore <p.f.moore at gmail.com> wrote:
> On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote:
>> Thoughts/suggestions?
>
> I think the core/stdlib position is that agreeing conventions would be
> better done once some real world experience of the practical issues
> and benefits of annotations has been established. So while a proposal
> like this is not without merit, it needs to be considered in the light
> of how projects actually use annotations. Personally, I'm not aware of
> any libraries that make significant use of annotations, so a good
> first step would be to survey existing use, and summarise it here.
> That would allow you to clarify your proposal in terms of exactly how
> existing projects would need to modify their current code.
>
> Of course, there's likely a chicken and egg problem here - projects
> may be holding off using annotations through fear of issues caused by
> clashes. But I'm not sure that a UUID-based proposal like the above
> (which as you admit is very verbose, and not particularly user
> friendly) would be more likely to encourage use.
>
> If I were developing a library that would benefit from annotations, at
> this point in time I'd probably just choose whatever conventions
> suited me and go with those - likely marking the feature as "subject
> to change" initially. Then, when people raised bug reports or feature
> requests that asked for better interoperability, I'd look at how to
> achieve that in conjunction with the other project(s) that clashed
> with me.
>
> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From shai at platonix.com  Wed Mar  5 14:10:54 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 15:10:54 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051259.40289.shai@platonix.com>
 <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
Message-ID: <201403051510.54769.shai@platonix.com>

On Wednesday 05 March 2014 14:34:09 Oscar Benjamin wrote:
> 
> I think it's clear that if this were new code being added to the
> stdlib then the consensus would be that having midnight evaluate as
> False is ridiculous.
> 
> The question is surely whether the issue is worth a backwards
> compatibility break not whether the current behaviour is a good idea
> (it clearly isn't).
> 

If this is the case, then we should be asking a variant of Paul's question:

Why would anyone check if a time is equal to midnight, by Boolean evaluation 
rather than comparison to midnight? Or, rather, how many people do this?

We have several reports of the current behavior causing surprising, hard-to-
detect bugs (a co-worker spent a couple of hours on one such bug yesterday; 
frankly, I'm impressed he found it that fast). Unless we have reason to think 
people are using this on purpose, the effect of the change will probably be to 
fix many more bugs than it causes.

I suggest to handle the backwards-compatibility issue with a deprecation 
cycle: Python 3.4 is already feature frozen, so make 3.5 raise a 
PendingDeprecationWarning when datetime.time.__nonzero__ returns False; 3.6 a 
DeprecationWarning; and 3.7 return True. That's what we would do in Django.

On Wednesday 05 March 2014 14:54:27 Paul Moore wrote:
> 
> And FWIW, my opinion is that the problem is not worth a compatibility
> break, because it's so easily solved (and the fixed code is probably
> an improvement in any case).

Ah, but it is only easily solved once it is detected; the behavior causes bugs 
that surface once in a blue moon (in production, and then they cannot be 
reproduced by developers who, usually, do not work at midnight).

So, on a second thought -- an alternative solution is to make any Boolean 
evaluation of time objects warn the user about midnight.
 
Thanks for your attention,

Shai.
	

From steve at pearwood.info  Wed Mar  5 14:21:03 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 00:21:03 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
Message-ID: <20140305132103.GZ28804@ando>

On Wed, Mar 05, 2014 at 12:10:18AM -0800, Andrew Barnert wrote:

> Anyone with a middle school expectation will expect 1/3 to be a 
> fraction?or, at least, something they can multiply by 3 to get exactly 
> 1. 

Not a very good example -- that happens to work for Python floats (which 
are C doubles under the hood):

py> (1/3)*3 == 1
True
py> (1/3 + 1/3 + 1/3) == 1
True


But it *does not work* with Decimal, at least not with the default 
precision:

py> from decimal import Decimal
py> (Decimal(1)/3)*3
Decimal('0.9999999999999999999999999999')


Decimal is not a panacea! It does not eliminate floating point issues. 
Between 1 and 100, there are 32 Decimal numbers that fail the test that 
(1/n)*n == 1, and only two floats. 

Between 1 and 100, there are only four floats where 1/(1/n) does not 
equal n: 49 93 98 and 99. In comparison, there are 46 such failing 
Decimals, including 6 7 and 9.



-- 
Steven

From steve at pearwood.info  Wed Mar  5 14:23:33 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 00:23:33 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
 <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>
Message-ID: <20140305132333.GA28804@ando>

On Wed, Mar 05, 2014 at 11:42:14PM +1100, Chris Angelico wrote:

> I'm not sure I like the idea of tagging the end of the expression.
> Currently, the nearest Python has to that is the complex notation:
> 
> >>> 1+2j
> (1+2j)
> 
> And that works just fine when considered to be addition:
> 
> >>> a=1
> >>> b=2j
> >>> a+b
> (1+2j)
> 
> So really, what Python has is a notation for a j-suffixed float,
> meaning a complex number with no real part. You can't do that with
> Fraction:
> 
> >>> a=2
> >>> b=3F
> >>> a/b

Why not? If 3d is a Decimal with the value of 3, why couldn't 3F be a 
Fraction with the value of 3?


-- 
Steven

From p.f.moore at gmail.com  Wed Mar  5 14:26:38 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 13:26:38 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051510.54769.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051259.40289.shai@platonix.com>
 <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
 <201403051510.54769.shai@platonix.com>
Message-ID: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>

On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote:
> On Wednesday 05 March 2014 14:54:27 Paul Moore wrote:
>>
>> And FWIW, my opinion is that the problem is not worth a compatibility
>> break, because it's so easily solved (and the fixed code is probably
>> an improvement in any case).
>
> Ah, but it is only easily solved once it is detected; the behavior causes bugs
> that surface once in a blue moon (in production, and then they cannot be
> reproduced by developers who, usually, do not work at midnight).

That's a fair point. But your deprecation suggestion doesn't seem to
help this, unless you expect (in a realistic timescale) to be able to
drop support for Python <3.7. Your users (on say 2.7 or 3.3) will
still get bugs, and now your developers (using 3.7) won't even be able
to reproduce the issue by staying up till midnight. And once they do
find the bug they will still have to fix it by an explicit test.

Again, I'm not saying the current behaviour is sensible, but I doubt
the work to fix it will benefit anyone in practice.

Paul.

From jsbueno at python.org.br  Wed Mar  5 14:40:45 2014
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Wed, 5 Mar 2014 10:40:45 -0300
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051259.40289.shai@platonix.com>
 <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
 <201403051510.54769.shai@platonix.com>
 <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
Message-ID: <CAH0mxTRVn5-RnuS9N+kYfWQnyeyyP-JdZwR2EKKraCx=yv3+uA@mail.gmail.com>

I agree this change would cause far more good than harm -
I see that it implies an incompatible change, but lets
think on "Although practicality beats purity" terms here:
the only instance I see this could possibly be used by purpose
is in a code to that would increase  a "day counter" if the time.now()
would evaluate to false. Such a piece of code would be broken in many
other levels
to start with.

A critical session in any piece of code that has to check if "at this
second now we are at midnight" should be using one of the specialized
timers to take care of borderline cases, such as start and ending of
D.S.T. and such. It would have to be orders of magnitude better
written than "if not current_time: ... "

And even in the improbable case of such a piece of code exists, the
changing of behavior would make it break in the first 24 hours, and
the problem would not be that hard to find out.

In this light, I think having to wait for 3 major release cycles until
that is gone is a lot of overkill.

On 5 March 2014 10:26, Paul Moore <p.f.moore at gmail.com> wrote:
> On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote:
>> On Wednesday 05 March 2014 14:54:27 Paul Moore wrote:
>>>
>>> And FWIW, my opinion is that the problem is not worth a compatibility
>>> break, because it's so easily solved (and the fixed code is probably
>>> an improvement in any case).
>>
>> Ah, but it is only easily solved once it is detected; the behavior causes bugs
>> that surface once in a blue moon (in production, and then they cannot be
>> reproduced by developers who, usually, do not work at midnight).
>
> That's a fair point. But your deprecation suggestion doesn't seem to
> help this, unless you expect (in a realistic timescale) to be able to
> drop support for Python <3.7. Your users (on say 2.7 or 3.3) will
> still get bugs, and now your developers (using 3.7) won't even be able
> to reproduce the issue by staying up till midnight. And once they do
> find the bug they will still have to fix it by an explicit test.
>
> Again, I'm not saying the current behaviour is sensible, but I doubt
> the work to fix it will benefit anyone in practice.
>
> Paul.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From stefan at bytereef.org  Wed Mar  5 14:39:42 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 5 Mar 2014 13:39:42 +0000 (UTC)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
Message-ID: <loom.20140305T143713-529@post.gmane.org>

Steven D'Aprano:
> > I don't agree with Mark's proposal in this thread but I would like to
> > have decimal literals e.g. 1.23d, 
> 
> +1 on that. Although that will depend on how big a slowdown it causes to 
> Python's startup time.

Startup time should not be a problem once http://bugs.python.org/issue19232
is dealt with.


Stefan Krah




From ncoghlan at gmail.com  Wed Mar  5 14:45:33 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 5 Mar 2014 23:45:33 +1000
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
Message-ID: <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>

On 5 Mar 2014 22:51, "Paul Moore" <p.f.moore at gmail.com> wrote:
>
> On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote:
> > Thoughts/suggestions?
>
> I think the core/stdlib position is that agreeing conventions would be
> better done once some real world experience of the practical issues
> and benefits of annotations has been established.

MyPy uses function annotations for optional static typing, which is pretty
much the use case Guido originally had in mind and the main reason that PEP
8 doesn't make combining annotations with an associated decorator
mandatory: http://www.mypy-lang.org/

You do still have to import a particular module to indicate that all
annotations in the importing module are to be interpreted as type
annotations.

Beyond that, the guidance in PEP 8 stands:

"""It is recommended that third party experiments with annotations use an
associated decorator to indicate how the annotation should be
interpreted."""

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/50045387/attachment.html>

From steve at pearwood.info  Wed Mar  5 14:56:46 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 00:56:46 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
Message-ID: <20140305135646.GB28804@ando>

On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote:

>    The idea of *python number* means that there are no types, no limits, no 
> constraints, and that all *python numbers *are dynamic. The upper level
> syntax is also very simple; all* python numbers *are simply human.

What makes this a "python number"? In what way are they "dynamic"?

 
>    My influence for this preference is rooted in the Rexx programming 
> language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx
> programming language is dynamic, and has no types. To put it more 
> succinctly for those of you who have not used Rexx, the only data type
> is a string of characters (that's it).  *Rexx numbers* are simply those 
> strings of characters that may be interpreted as a valid *Rexx number.*

I haven't used Rexx, but I have used Hypertalk, which worked the same 
way. If you don't care about performance, it can work quite well.


>    The Python language might be changed to adopt the *python number*
> concept for *all math processing*, unless explicitly modified.

Well, there's a bit of a problem here. Numbers in Python are not just 
used for maths processing. They're also used for indexing into lists, as 
keys in dicts, for bitwise operations, for compatibility with external 
libraries that have to interface with other languages, as flags, etc. 
For some of these purposes, we *really do* want to distinguish between 
ints and floats that happen to have the same value:

mylist[3]  # allowed
mylist[3.0]  # not allowed

Now, you might argue that this distinction is unnecessary, but it runs 
quite deep in Python. You'd need to change that philosophy for this idea 
to work.


> This goes 
> somewhat beyond using decimal floating point as a default numerical 
> type.  It means using human numeric expressions that meet human 
> expectation for numeric processing by default.

I don't understand what that means, unless it means that you want Python 
to somehow, magically, make all the unintuitive issues with floating 
point to disappear. Good luck with that one.

If you want that, Decimal is not the answer. It would have to be a 
Rational type, like Fraction, although even that doesn't support surds. 
Fractions have their own problems too. Compare the status quo:

py> 3**0.5
1.7320508075688772

with a hypothetical version that treats all numbers as exact fractions:

py> 3**0.5
Fraction(3900231685776981, 2251799813685248)

Which do you think the average person using Python as a calculator would 
prefer to see?

And another issue: before he invented Python, Guido spent a lot of time 
working with ABC, which used Fractions as the native number type. The 
experience soured him on the idea for nearly two decades. Although Guido 
has softened his stance enough to allow the fractions module into the 
standard library, I doubt he would allow Fractions to become the 
underlying implementation of numbers in Python.

The problem is that fractions can be unbounded in memory, and some 
simple operations become extremely inefficient. For example, without 
converting to float, which is bigger?

Fraction(296, 701)
Fraction(355, 594)

For many purposes, the fact that floats (whether binary or decimal) have 
finite precision and hence introduce rounding error is actually a good 
thing. Compare:

py> 1e-300 + 1e300
1e+300

versus fractions:

py> from fractions import Fraction as F
py> F(10)**-300 + F(10)**300
Fraction(100000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000001, 100000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000)


There are *very few* applications where such precision is needed or 
wanted. The performance hit in calculating such excessively precise 
numbers when the user doesn't need it will be painful.

The same applies to Decimal, although to a lesser extent since Decimals 
do have a finite precision. Unlike fractions, they cannot grow without 
limit:

py> Decimal("1e-300") + Decimal("1e300")
Decimal('1.000000000000000000000000000E+300')



-- 
Steven

From shai at platonix.com  Wed Mar  5 14:53:19 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 15:53:19 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051510.54769.shai@platonix.com>
 <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
Message-ID: <201403051553.19311.shai@platonix.com>

On Wednesday 05 March 2014 15:26:38 Paul Moore wrote:
> On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote:
> > On Wednesday 05 March 2014 14:54:27 Paul Moore wrote:
> >> And FWIW, my opinion is that the problem is not worth a compatibility
> >> break, because it's so easily solved (and the fixed code is probably
> >> an improvement in any case).
> > 
> > Ah, but it is only easily solved once it is detected; the behavior causes
> > bugs that surface once in a blue moon (in production, and then they
> > cannot be reproduced by developers who, usually, do not work at
> > midnight).
> 
> That's a fair point. But your deprecation suggestion doesn't seem to
> help this, unless you expect (in a realistic timescale) to be able to
> drop support for Python <3.7.

I expect to drop support for Python<3.7 sometime...

> Your users (on say 2.7 or 3.3) will
> still get bugs, and now your developers (using 3.7) won't even be able
> to reproduce the issue by staying up till midnight.

If my developers are using 3.7 to debug problems that user encounter with 3.3, 
they deserve to stay up till midnight and beyond.

> And once they do
> find the bug they will still have to fix it by an explicit test.

The process I want to see is, that Boolean evaluations of time objects is 
slowly, but more-or-less completely removed from code that targets Python<3.7; 
then, in a few years, people can go back to writing "if event.start_time" like 
they do with dates and datetimes.

> Again, I'm not saying the current behaviour is sensible, but I doubt
> the work to fix it will benefit anyone in practice.
> 

So, you support my other suggestion -- a warning on every use of bool(time)?

Also, the work to fix it is probably less than the work invested in this 
discussion so far...

In any case, if we all agree that http://bugs.python.org/issue13936 is a valid 
problem -- can we re-open it, and then discuss how (or if) we solve it?

Thanks,
	Shai.

From skip at pobox.com  Wed Mar  5 15:34:16 2014
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 5 Mar 2014 08:34:16 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
Message-ID: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 4:44 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>     if event.start_time:
>>         # stuff
>>
>> unexpectedly fails if start_time is midnight.
>
> Yeah, I was classing that as "application bug" and it's easy to fix
> with an "is not None".

+1, +1, +1. This is nothing more than an application bug. Issue 13936
should not be reopened.

This is a very common mistake when None is used as a sentinel value
(though people normally get away with it), especially by people coming
from languages having NULL pointers which shall not be named. After
all, "None" and "NULL" both start with the letter "N". Surely, they
must represent the same fundamental concept, right? <wink>

Let's change the problem slightly. Instead of time objects, let's
consider the domain of possible values to be integers. Clearly, None
is not a member of that set. You could thus use it as a sentinel. So,
tell me, OP. Would this usage be correct?

    if event.some_id:
        # stuff

I would argue, "no." I would not argue that 0 should not compare as
False, however. I would fix my code:

    if event.some_id is not None:
        # stuff

In contexts where None is an element of the set of possible values
(and thus can't be used as a sentinel value itself), it's common to
explicitly create such a (unique) value, e.g.:

    SENTINEL = ["xyz"]                # or [None] or {} or set([sys])
or ..., but not () or 1 or True!

When you check for the sentinel you have to be explicit (and you MUST
check for it using the "is" operator, to drag in another recent
thread):

    if something is SENTINEL:
        mix_the_special_sauce()

Using None as a sentinel value is no different, it just happens to a)
already exist, and b) be unique, saving you an exceedingly small
amount of typing. If you were trying to be terribly clever, there are
lots of other values in Python (at least in CPython) which happen to
be unique (the empty tuple, small integers, True and False, ...), and
could thus theoretically be used as a sentinel. Someone would
eventually slap your wrist for that sort of thing though (one would
hope).

Two final points.

1. If you wanted to modify the language (or the standard library) in
some way, perhaps the more theoretically reasonable request would be
to ask that None not be usable in a boolean context. Making that
change would certainly break a lot of code though, so it's too late to
do that.

2. The more I think about it, the more I think None should not be used
as a sentinel.

Skip

From p.f.moore at gmail.com  Wed Mar  5 16:01:33 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 15:01:33 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051553.19311.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051510.54769.shai@platonix.com>
 <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
 <201403051553.19311.shai@platonix.com>
Message-ID: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>

On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote:
> then, in a few years, people can go back to writing "if event.start_time" like
> they do with dates and datetimes.

Why on earth would they do that? It's still bad practice. If you're
using None as a sentinel, you should test for it explicitly. Nobody
has yet suggested any other use case where this matters.

>> Again, I'm not saying the current behaviour is sensible, but I doubt
>> the work to fix it will benefit anyone in practice.
>>
>
> So, you support my other suggestion -- a warning on every use of bool(time)?

No. Good programming practice should cover that. We don't warn in
other cases where programmers make silly coding errors. Look at Skip's
message - should we also warn on uses of bool(int) because people can
write bad code that fails to work properly with zero, as well?

> Also, the work to fix it is probably less than the work invested in this
> discussion so far...

That is not obvious (given documentation, release management, etc, costs).

> In any case, if we all agree that http://bugs.python.org/issue13936 is a valid
> problem -- can we re-open it, and then discuss how (or if) we solve it?

We don't. Can we agree that it's not a bug and abandon this fruitless
discussion?

Paul

From kaiser.yann at gmail.com  Wed Mar  5 16:08:02 2014
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Wed, 5 Mar 2014 16:08:02 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
Message-ID: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>

On 5 March 2014 15:34, Skip Montanaro <skip at pobox.com> wrote:
> On Wed, Mar 5, 2014 at 4:44 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>>     if event.start_time:
>>>         # stuff
>>>
>>> unexpectedly fails if start_time is midnight.
>>
>> Yeah, I was classing that as "application bug" and it's easy to fix
>> with an "is not None".
>
> +1, +1, +1. This is nothing more than an application bug. Issue 13936
> should not be reopened.
>
> This is a very common mistake when None is used as a sentinel value
> (though people normally get away with it), especially by people coming
> from languages having NULL pointers which shall not be named. After
> all, "None" and "NULL" both start with the letter "N". Surely, they
> must represent the same fundamental concept, right? <wink>
>
> Let's change the problem slightly. Instead of time objects, let's
> consider the domain of possible values to be integers. Clearly, None
> is not a member of that set. You could thus use it as a sentinel. So,
> tell me, OP. Would this usage be correct?
>
>     if event.some_id:
>         # stuff
>
> I would argue, "no." I would not argue that 0 should not compare as
> False, however.

You come up with an instance where 0 clearly does not apply as
something to incur a different branch, which is very apparent in just
reading the line, so very easy to spot as a mistake.  But this 0 isn't
a real zero.  It's just a unique identifier that happens to be
implemented as an integer(because it's convenient for your database,
iterators and whatnot.)  Let me show you an actual zero:

    if event.num_attendants:
        prepare_cake()
    else:
        cancel_event()

When no one is coming to your party, is is clearly a different
condition than if any number of people are coming to your event.  When
you read "if num_attendants", you can clearly tell this is going to do
something depending on if people are coming or not.

Let's read this line:

    if not event.date:

How would you read it, knowing nothing about date objects?  I would
read it to mean "has no date been set?" and would naturally continue
the code as such:

    if event.date:
        schedule_event()

I cannot fathom one example where it could read as "does the party
start at midnight?".  Can you?

From breamoreboy at yahoo.co.uk  Wed Mar  5 16:10:28 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 15:10:28 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051510.54769.shai@platonix.com>
 <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
Message-ID: <lf7eou$bf6$1@ger.gmane.org>

On 05/03/2014 15:01, Paul Moore wrote:
> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote:
>> then, in a few years, people can go back to writing "if event.start_time" like
>> they do with dates and datetimes.
>
> Why on earth would they do that? It's still bad practice. If you're
> using None as a sentinel, you should test for it explicitly. Nobody
> has yet suggested any other use case where this matters.
>
>>> Again, I'm not saying the current behaviour is sensible, but I doubt
>>> the work to fix it will benefit anyone in practice.
>>>
>>
>> So, you support my other suggestion -- a warning on every use of bool(time)?
>
> No. Good programming practice should cover that. We don't warn in
> other cases where programmers make silly coding errors. Look at Skip's
> message - should we also warn on uses of bool(int) because people can
> write bad code that fails to work properly with zero, as well?
>
>> Also, the work to fix it is probably less than the work invested in this
>> discussion so far...
>
> That is not obvious (given documentation, release management, etc, costs).
>
>> In any case, if we all agree that http://bugs.python.org/issue13936 is a valid
>> problem -- can we re-open it, and then discuss how (or if) we solve it?
>
> We don't. Can we agree that it's not a bug and abandon this fruitless
> discussion?
>
> Paul

I wouldn't agree that this has been a fruitless discussion.  I would 
agree that this is not a bug, that the code needs changing, and that at 
least one new test needs to be added to the unit test suite(s) for the 
buggy application(s).

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From shai at platonix.com  Wed Mar  5 16:19:09 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 17:19:09 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
Message-ID: <201403051719.09276.shai@platonix.com>

On Wednesday 05 March 2014 17:01:33 you wrote:
> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote:
> 
> >> Again, I'm not saying the current behaviour is sensible, but I doubt
> >> the work to fix it will benefit anyone in practice.
> > 
[...]
> > In any case, if we all agree that http://bugs.python.org/issue13936 is a
> > valid problem 
> 
> We don't. Can we agree that it's not a bug and abandon this fruitless
> discussion?
> 

What, other than "bug", do you call behavior that isn't sensible?

From shai at platonix.com  Wed Mar  5 16:15:29 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 17:15:29 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
Message-ID: <201403051715.29773.shai@platonix.com>

On Wednesday 05 March 2014 16:34:16 Skip Montanaro wrote:
> 
> Let's change the problem slightly. Instead of time objects, let's
> consider the domain of possible values to be integers. Clearly, None
> is not a member of that set. You could thus use it as a sentinel. So,
> tell me, OP. Would this usage be correct?
> 
>     if event.some_id:
>         # stuff
> 
> I would argue, "no." I would not argue that 0 should not compare as
> False, however.

What Yann said. Also, when opening this thread, I argued that 0 is special, 
but midnight is not -- which you completely ignore.

Thanks,
	Shai.

From rosuav at gmail.com  Wed Mar  5 16:23:19 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Mar 2014 02:23:19 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
 <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com>
 <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com>
Message-ID: <CAPTjJmqywp2NgeKA7MXQD=cCinm1W+qjzAfafdYieGLwx2m5WQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:56 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> 3F could create a Fraction with the integer value 3 so that a/b gives
> a rational number:
>
>>>> from fractions import Fraction as F
>>>> a = 2
>>>> b = F(3)
>>>> a/b
> Fraction(2, 3)
>
> I don't understand why you say that can't be done.

(Also Steven who said the same thing.)

Uhh... brown-paper-bag moment. When I wrote up that post, I somehow
blanked out the obvious fact that Fraction can happily represent an
integer. Whoops...

As Julia Jellicoe said, my objection falls to the ground. Very well!

ChrisA

From steve at pearwood.info  Wed Mar  5 16:35:29 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 02:35:29 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
Message-ID: <20140305153529.GC28804@ando>

On Wed, Mar 05, 2014 at 12:41:10PM +0000, Carl Smith wrote:
> Just for the record, I never called them bills just because they contain a
> dollar sign! I was thinking of a short word which describes an expression
> who's evaluation changes depending on the local circumstances. Like a legal
> bill, not a dollar bill.

Hmmm. Well, I don't really get the connection between legal bills and 
delayed evaluation, and I still really dislike the name.


> The name macro doesn't really work, as it's only an expression, it's not a
> code block.

There is nothing about macros that *require* them to accept a full block 
of code. In fact, there are multiple different meanings for macro, 
although they are all related they do have significant differences:

http://en.wikipedia.org/wiki/Macro_%28computer_science%29

Macros in C are not the same kind of thing as macros in Lisp.


> ---
> 
> The idea was never to start passing complex expressions around the place.

What you consider a complex expression somebody else may consider a 
simple expression. Unless you want to demand some arbitrary hard limit 
on complexity (and how do you measure complexity?) this "bill" system 
would have to allow the exact same types of expressions that are legal 
elsewhere in Python.


[...]
> What the hell is a thunk anyway? It's a horrible name.

Wikipedia is your friend:
http://en.wikipedia.org/wiki/Thunk

Also, more here: http://c2.com/cgi/wiki?CallByName

The meaning of thunk I'm referring to comes from the call-by-name 
argument passing model. Suppose we pass an expression to a function:

    function(arg=x-1)

and the body of the function looks like this:

    if arg > 1: 
        y = arg + 1
    else: 
        y = arg - 1
    return y*arg

In the "call-by-name" model, the function body executes like this:

    if (x+1) > 1: 
        y = (x+1) + 1
    else: 
        y = (x+1) - 1
    return y*(x+1)

which not only duplicates the "x+1" part four times, but may require 
evaluating it three times. To avoid this wasteful duplication, the 
compiler creates a special thunk value, which is something like a 
function with no arguments:

    def thunk():
        if cache is None:
            cache = x-1
        return cache

(This should be considered pseudo-code, not the exact way Algol or 
Haskell work -- real thunks also allow you to assign a value back to the 
"name". Also, technically the presence of a cache makes it call-by-need 
rather than call-by-name).

The body of the function then becomes:

    if thunk() > 1: 
        y = thunk() + 1
    else: 
        y = thunk() - 1
    return y*thunk()


So as you can see, what I've been calling a thunk is not precisely like 
Algol thunks. One thing which is missing is the dynamic scoping: in the 
thunk evaluation, the x comes from the caller's scope. But the delayed 
evaluation part is quite similar, enough that I think the name may be 
appropriate. Another name for this might be a "promise", as in the 
promise to execute a computation later.


-- 
Steven

From p.f.moore at gmail.com  Wed Mar  5 16:37:14 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 15:37:14 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051719.09276.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
Message-ID: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>

On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote:
> What, other than "bug", do you call behavior that isn't sensible?

"Unfortunate". I'm not arguing that the behaviour is useful. Or
sensible. What I'm arguing, and you're missing, is that the behaviour
is as documented, and so can't be changed without backward
compatibility implications. Specifically, see
http://docs.python.org/3.4/library/datetime.html#time-objects:

"""in Boolean contexts, a time object is considered to be true if and
only if, after converting it to minutes and subtracting utcoffset()
(or 0 if that's None), the result is non-zero."""

Your code is ignoring documented behaviour. Worse, your code can
clearly be improved by being explicit in your test, and yet you'd
rather argue that the Python developers implement a backward
compatibility break which you won't even be able to take advantage of
until you drop support of versions of Python before 3.7.

Paul.

From ron3200 at gmail.com  Wed Mar  5 16:38:27 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Wed, 05 Mar 2014 09:38:27 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305120410.GY28804@ando>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
Message-ID: <lf7gd6$1eg$1@ger.gmane.org>



On 03/05/2014 06:04 AM, Steven D'Aprano wrote:
> (By the way, I think it is somewhat amusing that Python not only has a
> built-in complex type, but also*syntax*  for creating complex numbers,
> but no built-in support for exact rationals.)

That is interesting.

I think Mark is correct in unifying numbers.  And also adding in decimal 
features.  The way it should actually be done is another thing.  But having 
an up to date package that can be used is a very good start.  (Thanks Mark!)

Mark describes an AI approach,, which I think he means having a internal 
representation that may change as needed depending on how a number can best 
be stored and calculated while still keeping it's accuracy.

Weather or not that approach is called Decimals is another thing. It might 
be called "Unified Numbers".. or just Numbers.

The point is for the internal representation to be an implementation detail 
the user doesn't need to worry about. And have Decimal features available 
by default.

If things are decided by use case, then I can't even think of a good enough 
argument against having decimal features available by default. The 
financial use cases are that overwhelming.

-Ron


From p.f.moore at gmail.com  Wed Mar  5 16:42:32 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 15:42:32 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
Message-ID: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>

On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote:
>  Let me show you an actual zero:
>
>     if event.num_attendants:
>         prepare_cake()
>     else:
>         cancel_event()
>
> When no one is coming to your party, is is clearly a different
> condition than if any number of people are coming to your event.  When
> you read "if num_attendants", you can clearly tell this is going to do
> something depending on if people are coming or not.

I'm sorry, are you really trying to say that the above code is better than

    if event.num_attendants != 0:
        prepare_cake()
    else:
        cancel_event()

?

(Personally, I'd actually prefer something like "if
event.num_attendants > 0" or switch the order of clauses and test for
being equal to 0, but that's a minor issue. The major point is I'd
prefer any of these to your version).

Paul

From shai at platonix.com  Wed Mar  5 17:05:15 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 18:05:15 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
Message-ID: <201403051805.15363.shai@platonix.com>

On Wednesday 05 March 2014 17:37:14 Paul Moore wrote:
> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote:
> > What, other than "bug", do you call behavior that isn't sensible?
> 
> "Unfortunate".

That's unfortunate. You miss a lot of bugs.

> I'm not arguing that the behaviour is useful. Or
> sensible. What I'm arguing, and you're missing, is that the behaviour
> is as documented, and so can't be changed without backward
> compatibility implications.

I think I have given ample room to backwards-compatibility considerations.

> Specifically, see
> http://docs.python.org/3.4/library/datetime.html#time-objects:
> 
> """in Boolean contexts, a time object is considered to be true if and
> only if, after converting it to minutes and subtracting utcoffset()
> (or 0 if that's None), the result is non-zero."""
> 
Yes, I know. It's mentioned in the bug.

> Your code is ignoring documented behaviour. Worse, your code can
> clearly be improved by being explicit in your test,

We disagree. I don't see adding "is not None" as an improvement at all. I see 
it as adding unnecessary ceremony. If you really like that much explicitness, 
use Java.

> and yet you'd
> rather argue that the Python developers implement a backward
> compatibility break which you won't even be able to take advantage of
> until you drop support of versions of Python before 3.7.
> 

What Ryan said: The code in question will not benefit from this change; it is 
Python 2.7, and will probably be rewritten from scratch before anything comes 
out of this. This is about fixing my projects a decade from now.

Shai.

From ryan at ryanhiebert.com  Wed Mar  5 16:50:04 2014
From: ryan at ryanhiebert.com (Ryan Hiebert)
Date: Wed, 5 Mar 2014 09:50:04 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
Message-ID: <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com>

On Wed, Mar 5, 2014 at 9:37 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote:
> > What, other than "bug", do you call behavior that isn't sensible?
>
> "Unfortunate". I'm not arguing that the behaviour is useful. Or
> sensible. What I'm arguing, and you're missing, is that the behaviour
> is as documented, and so can't be changed without backward
> compatibility implications. Specifically, see
> http://docs.python.org/3.4/library/datetime.html#time-objects:
>
> """in Boolean contexts, a time object is considered to be true if and
> only if, after converting it to minutes and subtracting utcoffset()
> (or 0 if that's None), the result is non-zero."""
>
> Your code is ignoring documented behaviour. Worse, your code can
> clearly be improved by being explicit in your test, and yet you'd
> rather argue that the Python developers implement a backward
> compatibility break which you won't even be able to take advantage of
> until you drop support of versions of Python before 3.7.
>
> Or, perhaps, even though he wouldn't be able to use this change right
away, he'd just like to see Python be better. I can't see that there's
going to come a better time to discuss this issue than the present, thus it
seems to me that you're suggesting that this odd and illogical behavior
continue forever, simply because it's been documented.

The backward compatibility and migration path still need to be addressed,
but it seems that we are disagreeing on whether this strange behavior
should ever change. I think that, at some point, it should. If we ever
agree on that, then we can start thinking about when.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f158fa02/attachment.html>

From ryan at ryanhiebert.com  Wed Mar  5 16:55:13 2014
From: ryan at ryanhiebert.com (Ryan Hiebert)
Date: Wed, 5 Mar 2014 09:55:13 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
Message-ID: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>

On Wed, Mar 5, 2014 at 9:42 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote:
> >  Let me show you an actual zero:
> >
> >     if event.num_attendants:
> >         prepare_cake()
> >     else:
> >         cancel_event()
> >
> > When no one is coming to your party, is is clearly a different
> > condition than if any number of people are coming to your event.  When
> > you read "if num_attendants", you can clearly tell this is going to do
> > something depending on if people are coming or not.
>
> I'm sorry, are you really trying to say that the above code is better than
>
>     if event.num_attendants != 0:
>         prepare_cake()
>     else:
>         cancel_event()
>
> ?
>
> (Personally, I'd actually prefer something like "if
> event.num_attendants > 0" or switch the order of clauses and test for
> being equal to 0, but that's a minor issue. The major point is I'd
> prefer any of these to your version).


I suspect you'd like the whole idea of empty collections and zeroes
evaluating false to go away, to force this kind of style (one right way to
do it and all). Some languages do that. Python isn't one of them.

I can see both sides, but I like how Python can use empty lists and zeros
as false values, and I often (but not always) write code that takes
advantage of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/3f02e5cf/attachment.html>

From breamoreboy at yahoo.co.uk  Wed Mar  5 17:19:28 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 16:19:28 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051719.09276.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
Message-ID: <lf7iqa$224$1@ger.gmane.org>

On 05/03/2014 15:19, Shai Berger wrote:
> On Wednesday 05 March 2014 17:01:33 you wrote:
>> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote:
>>
>>>> Again, I'm not saying the current behaviour is sensible, but I doubt
>>>> the work to fix it will benefit anyone in practice.
>>>
> [...]
>>> In any case, if we all agree that http://bugs.python.org/issue13936 is a
>>> valid problem
>>
>> We don't. Can we agree that it's not a bug and abandon this fruitless
>> discussion?
>>
>
> What, other than "bug", do you call behavior that isn't sensible?

Documented behaviour.  As opposed to insanity: doing the same thing over 
and over again and expecting different results.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From rosuav at gmail.com  Wed Mar  5 17:20:08 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Mar 2014 03:20:08 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051805.15363.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
 <201403051805.15363.shai@platonix.com>
Message-ID: <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>

On Thu, Mar 6, 2014 at 3:05 AM, Shai Berger <shai at platonix.com> wrote:
> On Wednesday 05 March 2014 17:37:14 Paul Moore wrote:
>> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote:
>> > What, other than "bug", do you call behavior that isn't sensible?
>>
>> "Unfortunate".
>
> That's unfortunate. You miss a lot of bugs.

A bug is something where behaviour differs from documentation or
intent. Differing from expectation isn't necessarily a bug (although
in the absence of clear docs or statement of intent, it could be
called one), partly because expectations vary. When the bulk of
programmers expect one thing and the code does another, it's probably
a wart or misfeature, but if it's properly documented, it can't really
be called a bug.

In this case, I would say that it's a wart. The fact that one
timestamp happens to be treated as zero is odd, and I wouldn't
advocate it, but now that it's there and documented, it's not a bug.

ChrisA

From p.f.moore at gmail.com  Wed Mar  5 17:24:16 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 16:24:16 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
Message-ID: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com>

On 5 March 2014 15:55, Ryan Hiebert <ryan at ryanhiebert.com> wrote:
> I suspect you'd like the whole idea of empty collections and zeroes
> evaluating false to go away, to force this kind of style (one right way to
> do it and all). Some languages do that. Python isn't one of them.

Not really. I just found the particular example given to be confusing.
I tend to use boolean tests when I think of something as a yes/no type
of thing, and the variable name implies it. If the example had
referred to "event.attendees" I would probably have been fine with the
check as written. The attribute in that case could be any one of a
boolean flag, a count, or a list of attendees. That's actually a
reasonably plausible way of writing the test to allow for later
changes to the event class API. The point here is that there's more to
whether code is good than just a black and white "does it work"
question.

But we're way off topic now - I don't even have any assurance that the
OP's code is trying to test for a None sentinel and hitting a bug
because not all time values evaluate as True (but if that's *not* his
code, he really should have pointed that out by now!) All I can say is
that *if* that is what his code is, then I think that rewriting it
with an explicit test is clearer, and I'm not convinced by his
assertion that "fixing" Python is a better solution.

> I can see both sides, but I like how Python can use empty lists and zeros as
> false values, and I often (but not always) write code that takes advantage
> of it.

Agreed, but that doesn't mean that code taking advantage of this is
never obfuscated or hard to maintain.

Paul.

PS One final point, and I'll say this one last time and then shut up.
I have no objection to making Python better. Nor do I insist that
backward compatibility should never be broken. Nor do I wish Python
were Java. Nor do I discount the issue of hard to find bugs. But
equally, I don't think Python is perfect, nor do I think that this
behaviour is ideal. I also don't think that the core developers have
infinite time, nor do I think that every wart in Python needs to be
fixed. And I trust the core developers to make the right judgements
most of the time. All anyone should infer from this discussion is that
I don't think that reopening the bug report referred to by the OP is
productive. (You may also assume from the fact that I will drop out of
this discussion now, that I no longer think that this thread is
particularly productive :-()

From ryan at ryanhiebert.com  Wed Mar  5 16:20:57 2014
From: ryan at ryanhiebert.com (Ryan Hiebert)
Date: Wed, 5 Mar 2014 09:20:57 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051510.54769.shai@platonix.com>
 <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
Message-ID: <CABpHFHS4gTBAsQYkj7MCANmgTJ8ftNsa4gR_cDqRcTceBDVaaA@mail.gmail.com>

On Wed, Mar 5, 2014 at 9:01 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote:
>
> We don't. Can we agree that it's not a bug and abandon this fruitless
> discussion.


Please don't abandon this discussion. The behavior we're discussing is
clearly illogical and useless.

If we simply removed the __nonzero__ behavior, and it did what user-created
types do, then it would perform as I would expect it to.

Consider also that this time has no special significance, unlike the
special cases of 0, (), {}, etc. This is true also for any arbitrarily
zeroed unit, such as temperature (C or F), and including datetime, date,
and time. Even zero in those cases (however that is defined) is simply an
arbitrary point, and not the same as a nonzero value.

This is completely illogical behavior, and, IMO, is a bug.

Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bb7db7b8/attachment.html>

From rosuav at gmail.com  Wed Mar  5 17:26:09 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Mar 2014 03:26:09 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
Message-ID: <CAPTjJmoAn9zpQi==RxW+S2iaiCdqymmgBTLq566nBmkM4+Fngw@mail.gmail.com>

On Thu, Mar 6, 2014 at 2:55 AM, Ryan Hiebert <ryan at ryanhiebert.com> wrote:
> I suspect you'd like the whole idea of empty collections and zeroes
> evaluating false to go away, to force this kind of style (one right way to
> do it and all). Some languages do that. Python isn't one of them.
>
> I can see both sides, but I like how Python can use empty lists and zeros as
> false values, and I often (but not always) write code that takes advantage
> of it.

The broad concept is: A thing is true, a non-thing is false. The
question is, though, which things are actually non-things? Since
Python doesn't have a concept of not storing anything into a variable,
there has to be a thing that represents the state of not being a
thing, so None should be false. Python's made the choice that an empty
string is a non-thing, but other languages choose instead to have a
separate "Nil" value and all strings are true. Same with lists,
tuples, etc. Pike's choice is that the integer 0 is false, and
anything else is true, so you distinguish between having a string and
not-having a string, rather than between having a non-empty string and
having an empty string. Simple design choice, and one that each
language follows consistently.

But if you truly want to abolish the confusion, you want REXX. In
REXX, 0 is false and 1 is true, and everything else is error 34,
"Logical value not 0 or 1". Is that really an improvement? Not in my
opinion.

ChrisA

From shai at platonix.com  Wed Mar  5 17:38:25 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 18:38:25 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
Message-ID: <201403051838.25342.shai@platonix.com>

On Wednesday 05 March 2014 18:20:08 Chris Angelico wrote:
> On Thu, Mar 6, 2014 at 3:05 AM, Shai Berger <shai at platonix.com> wrote:
> > On Wednesday 05 March 2014 17:37:14 Paul Moore wrote:
> >> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote:
> >> > What, other than "bug", do you call behavior that isn't sensible?
> >> 
> >> "Unfortunate".
> > 
> > That's unfortunate. You miss a lot of bugs.
> 
> A bug is something where behaviour differs from documentation or
> intent. Differing from expectation isn't necessarily a bug (although
> in the absence of clear docs or statement of intent, it could be
> called one), partly because expectations vary. When the bulk of
> programmers expect one thing and the code does another, it's probably
> a wart or misfeature, but if it's properly documented, it can't really
> be called a bug.
> 
> In this case, I would say that it's a wart.

Fine. That is, I'm not too interested in the semantics of English, but rather 
in the semantics of Python.

So, I rephrase my request: Would anybody object to a silent warning issued 
whenever a time-object is evaluated as Boolean? Something along the lines of,

"Evaluating time object as Boolean: Note that time(0,0,0) evaluates to False"

A silent warning, IIUC, is only printed when requested -- so it won't get in 
anybody's way, but still provide a tool for avoiding the bug. It can be added 
into Python 3.5. It could probably be added to earlier versions, but for that 
we'd need to consider this a bug :)

Shai.

From rob.cliffe at btinternet.com  Wed Mar  5 18:00:40 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 05 Mar 2014 17:00:40 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
 <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com>
Message-ID: <53175838.6020100@btinternet.com>


On 05/03/2014 16:24, Paul Moore wrote:
>> I can see both sides, but I like how Python can use empty lists and zeros as
>> false values, and I often (but not always) write code that takes advantage
>> of it.
> Agreed, but that doesn't mean that code taking advantage of this is
> never obfuscated or hard to maintain.
>
> Paul.
I'm with Shai and Ryan on this one.  There's no such thing as a perfect 
language, but that doesn't mean we shouldn't strive towards it.
>
> PS One final point, and I'll say this one last time and then shut up.
> I have no objection to making Python better. Nor do I insist that
> backward compatibility should never be broken. Nor do I wish Python
> were Java. Nor do I discount the issue of hard to find bugs. But
> equally, I don't think Python is perfect, nor do I think that this
> behaviour is ideal. I also don't think that the core developers have
> infinite time, nor do I think that every wart in Python needs to be
> fixed.
Good.  In other words, each wart should be considered on its merits, and 
some may be worth fixing.
Changes have been made to Python where the backward compatibility issue 
is far greater.
In this case any code that would be broken by the change is already 
flawed, if not broken.
And I find it hard to imagine that the development effort in making this 
change is anything other than minimal,
_compared with other comparable changes_ (I'm not trying to minimise the 
undoubted effort required to make _any_ change no matter how small).

Rob Cliffe
>   And I trust the core developers to make the right judgements
> most of the time. All anyone should infer from this discussion is that
> I don't think that reopening the bug report referred to by the OP is
> productive. (You may also assume from the fact that I will drop out of
> this discussion now, that I no longer think that this thread is
> particularly productive :-()
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.2247 / Virus Database: 3705/6652 - Release Date: 03/04/14
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a822a83c/attachment-0001.html>

From skip at pobox.com  Wed Mar  5 18:09:11 2014
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 5 Mar 2014 11:09:11 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051715.29773.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
Message-ID: <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>

On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote:
> Also, when opening this thread, I argued that 0 is special,
> but midnight is not -- which you completely ignore.

Zero is not different, at least when considered as an element of a set
(of integers, in my example). What's important is that we have a set
of values (either time objects or integers in these two examples) and
need some way to specify a sentinel value. That sentinel must not be
in the set of valid values. In both of these cases, None fills the
bill. Note that the only reasonable relationship between None and
either the set of all time objects or the set of all integers is that
it is not a member of either set. Testing that membership by appealing
to their value cast to a boolean is simply wrong.

I'll push the argument just a bit further. Suppose you have an
attribute which corresponds to the setting of a two-valued sensor
external to your system. You choose to represent those two values in
your program as True and False. You need a way to tell if any sensor
inputs have been detected yet, or not, so you initialize the attribute
with a sentinel value. I don't care what value you choose out of all
the possible sentinel values available to you (None, [], ["xyz"], {},
7, 0.0, etc), treating any of them as a boolean will be wrong. If you
do, you will erroneously always believe that your sensor has produced
a value.

I have made this sort of mistake many times over the years, and seen
other people do it as well. If what you really want to test is
membership of a value in a set, be explicit about it.

Skip

From alexander.belopolsky at gmail.com  Wed Mar  5 18:27:21 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 12:27:21 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051838.25342.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
Message-ID: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com> wrote:

> So, I rephrase my request: Would anybody object to a silent warning issued
> whenever a time-object is evaluated as Boolean?
>

I would.  Users should not be penalized for using a documented behavior.
 There are legitimate uses for bool(midnight) being False.  Midnight is
special in many contexts.  For example, it is uncertain whether midnight
belongs to the previous or next day.  If your application wants to group
midnight differently from other times - it is perfectly fine to use "if
dt.time()" instead of a more verbose "if dt.time() != datetime.time(0, 0)".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/3f930cd1/attachment.html>

From amber.yust at gmail.com  Wed Mar  5 18:27:27 2014
From: amber.yust at gmail.com (Amber Yust)
Date: Wed, 05 Mar 2014 17:27:27 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
Message-ID: <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>

Let's turn this around. Aside from backwards compatibility, what benefit is
there from having midnight evaluate as false?

On Wednesday, March 5, 2014 9:09:50 AM, Skip Montanaro <skip at pobox.com>
wrote:

> On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote:
> > Also, when opening this thread, I argued that 0 is special,
> > but midnight is not -- which you completely ignore.
>
> Zero is not different, at least when considered as an element of a set
> (of integers, in my example). What's important is that we have a set
> of values (either time objects or integers in these two examples) and
> need some way to specify a sentinel value. That sentinel must not be
> in the set of valid values. In both of these cases, None fills the
> bill. Note that the only reasonable relationship between None and
> either the set of all time objects or the set of all integers is that
> it is not a member of either set. Testing that membership by appealing
> to their value cast to a boolean is simply wrong.
>
> I'll push the argument just a bit further. Suppose you have an
> attribute which corresponds to the setting of a two-valued sensor
> external to your system. You choose to represent those two values in
> your program as True and False. You need a way to tell if any sensor
> inputs have been detected yet, or not, so you initialize the attribute
> with a sentinel value. I don't care what value you choose out of all
> the possible sentinel values available to you (None, [], ["xyz"], {},
> 7, 0.0, etc), treating any of them as a boolean will be wrong. If you
> do, you will erroneously always believe that your sensor has produced
> a value.
>
> I have made this sort of mistake many times over the years, and seen
> other people do it as well. If what you really want to test is
> membership of a value in a set, be explicit about it.
>
> Skip
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/2bb9f055/attachment.html>

From shai at platonix.com  Wed Mar  5 18:24:04 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 19:24:04 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
Message-ID: <201403051924.05252.shai@platonix.com>

On Wednesday 05 March 2014 19:09:11 Skip Montanaro wrote:
> On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote:
> > Also, when opening this thread, I argued that 0 is special,
> > but midnight is not -- which you completely ignore.
> 
> Zero is not different, at least when considered as an element of a set
> (of integers, in my example). What's important is that we have a set
> of values (either time objects or integers in these two examples) and
> need some way to specify a sentinel value.

Your argument has merit, but is besides the point. As noted by many in this 
thread, the idiom "if obj:" for deciding on the existence of some value is 
quite common. We all know it doesn't always fit, because Python chose to make 
some objects falsey; however, when it does fit, it makes for succinct, clear, 
unceremonious code.

If your argument is not that "if obj:" should never be used, except when obj 
actually is Boolean, please elaborate.

At issue here is the fact that the idiom doesn't fit where one would expect 
that it should.

Thanks,
	Shai.


From ryan at ryanhiebert.com  Wed Mar  5 17:38:56 2014
From: ryan at ryanhiebert.com (Ryan Hiebert)
Date: Wed, 5 Mar 2014 10:38:56 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com>
 <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com>
Message-ID: <CABpHFHTPBRGgBFj3vfNxmZD3wOBXKrzZGhXBXPsj_oeju=SGpA@mail.gmail.com>

On Wed, Mar 5, 2014 at 10:24 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> PS One final point, and I'll say this one last time and then shut up.
> I have no objection to making Python better. Nor do I insist that
> backward compatibility should never be broken. Nor do I wish Python
> were Java. Nor do I discount the issue of hard to find bugs. But
> equally, I don't think Python is perfect, nor do I think that this
> behaviour is ideal. I also don't think that the core developers have
> infinite time, nor do I think that every wart in Python needs to be
> fixed. And I trust the core developers to make the right judgements
> most of the time. All anyone should infer from this discussion is that
> I don't think that reopening the bug report referred to by the OP is
> productive. (You may also assume from the fact that I will drop out of
> this discussion now, that I no longer think that this thread is
> particularly productive :-()
>

Thanks for bringing this up. Even assuming that this was accepted as a bug,
it's an entirely separate discussion to decide whether worth fixing or if
it's worth the dev's time. I make no assertion on that, as that's their
time and not mine. I hold the developers with highest regard, even in my
disagreement. Python is my language of choice because of the effort of
these folks, and I thank them.

This seems like it could be a simple enough bug that were it allowed, it
would be a fix that even someone as inexperienced with Python's internals
as myself could fix. If that's not true and the issue is that it's simply
too much work to change, then it should be marked "wontfix" rather than
"invalid", and that's a decision that I would not contest (at least not
without a working patch).

I, like you, suspect that I will be abandoning further involvement, as
simply arguing won't change people's minds.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5cc69176/attachment-0001.html>

From p.f.moore at gmail.com  Wed Mar  5 18:58:12 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Mar 2014 17:58:12 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
Message-ID: <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>

On 5 March 2014 17:27, Amber Yust <amber.yust at gmail.com> wrote:
> Let's turn this around. Aside from backwards compatibility, what benefit is
> there from having midnight evaluate as false?

Zero cost. Any alternative must justify the cost of making the change.
Simple as that. Quantify the cost of making the change (properly!) and
describe the benefits in such a way as to justify those costs.

Paul

From shai at platonix.com  Wed Mar  5 18:56:23 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 19:56:23 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
Message-ID: <201403051956.24081.shai@platonix.com>

On Wednesday 05 March 2014 19:27:21 Alexander Belopolsky 
<alexander.belopolsky at gmail.com> wrote:
> On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com> wrote:
> > So, I rephrase my request: Would anybody object to a silent warning
> > issued whenever a time-object is evaluated as Boolean?
> 
> I would.  Users should not be penalized for using a documented behavior.

Fair enough. I would also add the warning to the documentation, and call the 
behavior "deprecated" (even if no steps will be taken to actually remove it).

>  There are legitimate uses for bool(midnight) being False.  Midnight is
> special in many contexts.  For example, it is uncertain whether midnight
> belongs to the previous or next day.  If your application wants to group
> midnight differently from other times - it is perfectly fine to use "if
> dt.time()" instead of a more verbose "if dt.time() != datetime.time(0, 0)".

The question is: Of all the Python lines in existence, where a time is 
evaluated as Boolean, what percentage intends to capture midnight, and what 
percentage intends to capture an empty field; the latter is a (user) bug, and I 
suspect it is the much larger part.

So even though there are legitimate uses for midnight being false, keeping 
things as they are is setting a trap for users. It is documented, but highly 
unintuitive (as evidenced by many messages on this thread). I think it best if 
the gotcha is removed, but if not, I ask at least to have a way to mitigate 
the problem.

Thanks,
	Shai.

From amber.yust at gmail.com  Wed Mar  5 19:12:46 2014
From: amber.yust at gmail.com (Amber Yust)
Date: Wed, 5 Mar 2014 10:12:46 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
Message-ID: <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>

I think this is the wrong stage to evaluate cost. After all, one of the
tenets of open source is that the core devs don't have to be the source of
all change.

I see a difference between "this could be better, but the core devs aren't
going to spend time on it" and "this should not change."

It would be nice if those two were not conflated so much.
On Mar 5, 2014 9:58 AM, "Paul Moore" <p.f.moore at gmail.com> wrote:

> On 5 March 2014 17:27, Amber Yust <amber.yust at gmail.com> wrote:
> > Let's turn this around. Aside from backwards compatibility, what benefit
> is
> > there from having midnight evaluate as false?
>
> Zero cost. Any alternative must justify the cost of making the change.
> Simple as that. Quantify the cost of making the change (properly!) and
> describe the benefits in such a way as to justify those costs.
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/9ecbb54b/attachment.html>

From breamoreboy at yahoo.co.uk  Wed Mar  5 19:23:12 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 18:23:12 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
Message-ID: <lf7q2a$vpl$1@ger.gmane.org>

On 05/03/2014 18:12, Amber Yust wrote:
> I think this is the wrong stage to evaluate cost. After all, one of the
> tenets of open source is that the core devs don't have to be the source
> of all change.
>

But the core devs have to review the change and ultimately decide 
whether or not to commit it.  In this case it looks as if a patch would 
not be accepted, so why keep going on about it, especially when there's 
a known work around?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From amber.yust at gmail.com  Wed Mar  5 19:26:59 2014
From: amber.yust at gmail.com (Amber Yust)
Date: Wed, 5 Mar 2014 10:26:59 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
Message-ID: <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com>

Where did you see an indication that a patch would not be accepted?
On Mar 5, 2014 10:24 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:

> On 05/03/2014 18:12, Amber Yust wrote:
>
>> I think this is the wrong stage to evaluate cost. After all, one of the
>> tenets of open source is that the core devs don't have to be the source
>> of all change.
>>
>>
> But the core devs have to review the change and ultimately decide whether
> or not to commit it.  In this case it looks as if a patch would not be
> accepted, so why keep going on about it, especially when there's a known
> work around?
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask what
> you can do for our language.
>
> Mark Lawrence
>
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/abaa2db8/attachment.html>

From skip at pobox.com  Wed Mar  5 19:33:08 2014
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 5 Mar 2014 12:33:08 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403051924.05252.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <201403051924.05252.shai@platonix.com>
Message-ID: <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:24 AM, Shai Berger <shai at platonix.com> wrote:
> If your argument is not that "if obj:" should never be used, except
> when obj actually is Boolean, please elaborate.

My approach continues to be to consider the overall set of objects
from which you draw values to assign to obj. Call that set V. If this
expression is true:

    len([bool(x) for x in V]) == 1

then using "if obj:" might be valid (if the one "false" value of that
set is some sort of distinguishing characteristic, as it often
is). If, however, the length of that set is greater than one, you must
be explicit in your conditional expressions. It is unfortunate for
your purposes that bool(datetime.time(0, 0, 0)) is False, but it's a
documented property of that object, and is highly unlikely to
change. In your case V is None plus the set of all time objects. That
has two values which evaluate to False, so "if obj:" is not
appropriate. If, on the other hand, the set of all values is the set
of all strings or the set of all integers or the set of all lists,
then maybe the falsity of the zero values in those sets ("", 0, [])
has some use.

My last comment on this topic. I will invoke the Uncle Timmy rule
(which I just made up). Never contradict a decision Uncle Timmy
made. He had his reasons. They were very good reasons. They were
almost certainly better than any reasons you can come up with as
counterarguments. Deal with it. :-)

Skip

From shai at platonix.com  Wed Mar  5 19:33:43 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 20:33:43 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
Message-ID: <201403052033.44162.shai@platonix.com>

On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote:
> On 05/03/2014 18:12, Amber Yust wrote:
> > I think this is the wrong stage to evaluate cost. After all, one of the
> > tenets of open source is that the core devs don't have to be the source
> > of all change.
> 
> But the core devs have to review the change and ultimately decide
> whether or not to commit it.  In this case it looks as if a patch would
> not be accepted, so why keep going on about it, especially when there's
> a known work around?

As noted before, the workaround is trivial once you detect the problem. 
Detecting the problem is not easy.

Shai.

From shai at platonix.com  Wed Mar  5 19:44:55 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 20:44:55 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
Message-ID: <201403052044.55720.shai@platonix.com>

On Wednesday 05 March 2014 20:33:08 Skip Montanaro wrote:
> On Wed, Mar 5, 2014 at 11:24 AM, Shai Berger <shai at platonix.com> wrote:
> > If your argument is not that "if obj:" should never be used, except
> > when obj actually is Boolean, please elaborate.
> 
> My approach continues to be to consider the overall set of objects
> from which you draw values to assign to obj. Call that set V. If this
> expression is true:
> 
>     len([bool(x) for x in V]) == 1
> 
> then using "if obj:" might be valid (if the one "false" value of that
> set is some sort of distinguishing characteristic, as it often
> is). If, however, the length of that set is greater than one, you must
> be explicit in your conditional expressions. 

No argument so far.

> It is unfortunate for your purposes that bool(datetime.time(0, 0, 0)) is
> False, but it's a documented property of that object, and is highly unlikely
> to change.

I think this is what's called 'begging the question' -- this discussion was 
not about the validity of boolean testing in general, but about the validity 
of this property of time objects. Yes, it's documented. I'm trying to make it 
change. There is a lot of support for that, and the opposition seems to be 
centered on "because it's documented" -- with the two exceptions being an edge 
use-case (actually checking midnight) and your call to authority below.

It's a gotcha. Let's remove it. If we don't remove it, let's give ourselves a 
tool to detect it.

> My last comment on this topic. I will invoke the Uncle Timmy rule
> (which I just made up). Never contradict a decision Uncle Timmy
> made. He had his reasons. They were very good reasons. They were
> almost certainly better than any reasons you can come up with as
> counterarguments. Deal with it. :-)
> 

The reason he actually gave (on the ticket)? "because it's documented".

Shai.

From breamoreboy at yahoo.co.uk  Wed Mar  5 19:47:02 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 18:47:02 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403052033.44162.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org> <201403052033.44162.shai@platonix.com>
Message-ID: <lf7rf0$e7m$1@ger.gmane.org>

On 05/03/2014 18:33, Shai Berger wrote:
> On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote:
>> On 05/03/2014 18:12, Amber Yust wrote:
>>> I think this is the wrong stage to evaluate cost. After all, one of the
>>> tenets of open source is that the core devs don't have to be the source
>>> of all change.
>>
>> But the core devs have to review the change and ultimately decide
>> whether or not to commit it.  In this case it looks as if a patch would
>> not be accepted, so why keep going on about it, especially when there's
>> a known work around?
>
> As noted before, the workaround is trivial once you detect the problem.
> Detecting the problem is not easy.
>
> Shai.

Use mocks in unit testing?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From breamoreboy at yahoo.co.uk  Wed Mar  5 19:59:37 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 18:59:37 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
 <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com>
Message-ID: <lf7s6j$rl4$1@ger.gmane.org>

On 05/03/2014 18:26, Amber Yust wrote:
> Where did you see an indication that a patch would not be accepted?
>
> On Mar 5, 2014 10:24 AM, "Mark Lawrence"
> <breamoreboy at yahoo.co.uk
> <mailto:breamoreboy at yahoo.co.uk>> wrote:
>
>     On 05/03/2014 18:12, Amber Yust wrote:
>
>         I think this is the wrong stage to evaluate cost. After all, one
>         of the
>         tenets of open source is that the core devs don't have to be the
>         source
>         of all change.
>
>
>     But the core devs have to review the change and ultimately decide
>     whether or not to commit it.  In this case it looks as if a patch
>     would not be accepted, so why keep going on about it, especially
>     when there's a known work around?
>
>     --
>     My fellow Pythonistas, ask not what our language can do for you, ask
>     what you can do for our language.
>
>     Mark Lawrence
>
>     ---

http://bugs.python.org/issue13936 has been closed as invalid and I don't 
see a rush of core devs backing this proposal.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From amber.yust at gmail.com  Wed Mar  5 20:06:50 2014
From: amber.yust at gmail.com (Amber Yust)
Date: Wed, 5 Mar 2014 11:06:50 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf7s6j$rl4$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
 <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com>
 <lf7s6j$rl4$1@ger.gmane.org>
Message-ID: <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com>

It was closed by one or two individuals, and there has been significant
comment since then. Asking to revisit a decision isn't crazy, and there
have been reasonable suggestions made. I don't see a lack of core dev
"jumping upon" as an active indication that patches would be rejected;
merely that core devs are unlikely to produce such a patch themselves.
On Mar 5, 2014 11:00 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:

> On 05/03/2014 18:26, Amber Yust wrote:
>
>> Where did you see an indication that a patch would not be accepted?
>>
>> On Mar 5, 2014 10:24 AM, "Mark Lawrence"
>> <breamoreboy at yahoo.co.uk
>> <mailto:breamoreboy at yahoo.co.uk>> wrote:
>>
>>     On 05/03/2014 18:12, Amber Yust wrote:
>>
>>         I think this is the wrong stage to evaluate cost. After all, one
>>         of the
>>         tenets of open source is that the core devs don't have to be the
>>         source
>>         of all change.
>>
>>
>>     But the core devs have to review the change and ultimately decide
>>     whether or not to commit it.  In this case it looks as if a patch
>>     would not be accepted, so why keep going on about it, especially
>>     when there's a known work around?
>>
>>     --
>>     My fellow Pythonistas, ask not what our language can do for you, ask
>>     what you can do for our language.
>>
>>     Mark Lawrence
>>
>>     ---
>>
>
> http://bugs.python.org/issue13936 has been closed as invalid and I don't
> see a rush of core devs backing this proposal.
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask what
> you can do for our language.
>
> Mark Lawrence
>
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/007cdeda/attachment-0001.html>

From breamoreboy at yahoo.co.uk  Wed Mar  5 20:19:11 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 19:19:11 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
 <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com>
 <lf7s6j$rl4$1@ger.gmane.org>
 <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com>
Message-ID: <lf7tb9$a8p$1@ger.gmane.org>

On 05/03/2014 19:06, Amber Yust wrote:
> It was closed by one or two individuals, and there has been significant
> comment since then. Asking to revisit a decision isn't crazy, and there
> have been reasonable suggestions made. I don't see a lack of core dev
> "jumping upon" as an active indication that patches would be rejected;
> merely that core devs are unlikely to produce such a patch themselves.
>
> On Mar 5, 2014 11:00 AM, "Mark Lawrence"
> <breamoreboy at yahoo.co.uk
> <mailto:breamoreboy at yahoo.co.uk>> wrote:
>
>     On 05/03/2014 18:26, Amber Yust wrote:
>
>         Where did you see an indication that a patch would not be accepted?
>
>         On Mar 5, 2014 10:24 AM, "Mark Lawrence"
>         <breamoreboy at yahoo.co.uk
>         <mailto:breamoreboy at yahoo.co.uk>
>         <mailto:breamoreboy at yahoo.co.__uk
>         <mailto:breamoreboy at yahoo.co.uk>>>
>         wrote:
>
>              On 05/03/2014 18:12, Amber Yust wrote:
>
>                  I think this is the wrong stage to evaluate cost. After
>         all, one
>                  of the
>                  tenets of open source is that the core devs don't have
>         to be the
>                  source
>                  of all change.
>
>
>              But the core devs have to review the change and ultimately
>         decide
>              whether or not to commit it.  In this case it looks as if a
>         patch
>              would not be accepted, so why keep going on about it,
>         especially
>              when there's a known work around?
>
>              --
>              My fellow Pythonistas, ask not what our language can do for
>         you, ask
>              what you can do for our language.
>
>              Mark Lawrence
>
>              ---
>
>     http://bugs.python.org/__issue13936
>     <http://bugs.python.org/issue13936> has been closed as invalid and I
>     don't see a rush of core devs backing this proposal.
>

There can be lots of comments by Her Majesty Queen Elizabeth II or 
President Obama but they don't really count unless they put forward 
arguments to pursuade the core devs to change their minds about this 
issue. I see no evidence here that this is likely to happen, or do the 
core devs all come to life at the same time as they're all in the same 
time zone?  A patch can be placed on the issue but if a core dev reviews 
and it and decides not to commit it nothing happens?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From shai at platonix.com  Wed Mar  5 20:18:37 2014
From: shai at platonix.com (Shai Berger)
Date: Wed, 5 Mar 2014 21:18:37 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf7rf0$e7m$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org>
Message-ID: <201403052118.38295.shai@platonix.com>

On Wednesday 05 March 2014 20:47:02 Mark Lawrence wrote:
> On 05/03/2014 18:33, Shai Berger wrote:
> > On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote:
> >> On 05/03/2014 18:12, Amber Yust wrote:
> >>> I think this is the wrong stage to evaluate cost. After all, one of the
> >>> tenets of open source is that the core devs don't have to be the source
> >>> of all change.
> >> 
> >> But the core devs have to review the change and ultimately decide
> >> whether or not to commit it.  In this case it looks as if a patch would
> >> not be accepted, so why keep going on about it, especially when there's
> >> a known work around?
> > 
> > As noted before, the workaround is trivial once you detect the problem.
> > Detecting the problem is not easy.
> > 
> > Shai.
> 
> Use mocks in unit testing?

Unless you're aware of the issue, you are not going to unit-test specifically 
for midnight. If you are aware, you don't need to detect it, you write "is not 
None".

From steve at pearwood.info  Wed Mar  5 20:24:19 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 06:24:19 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403052044.55720.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
Message-ID: <20140305192419.GD28804@ando>

On Wed, Mar 05, 2014 at 08:44:55PM +0200, Shai Berger wrote:

> > It is unfortunate for your purposes that bool(datetime.time(0, 0, 0)) is
> > False, but it's a documented property of that object, and is highly unlikely
> > to change.
> 
> I think this is what's called 'begging the question' -- this discussion was 
> not about the validity of boolean testing in general, but about the validity 
> of this property of time objects. Yes, it's documented. I'm trying to make it 
> change. There is a lot of support for that, and the opposition seems to be 
> centered on "because it's documented" -- with the two exceptions being an edge 
> use-case (actually checking midnight) and your call to authority below.

There are some very powerful practical reasons for leaving the status 
quo, namely backwards-compatibility and code-churn. These are real 
costs, not for the person who writes the patch, or the people who write 
the tests, and document the change. But these are potential costs for 
the millions of people using Python who may be *relying* on midnight 
being falsey, or nevertheless will have to change their code if this 
change occurs.

There's also the argument from consistency: midnight is the zero point 
of the day, and zero is falsey. It would be surprising if something 
which is essentially zero became truthy.

Compared to those costs, the benefits are insignificant. For library 
authors, they cannot take advantage of this new always-truthy time 
objects for probably eight or ten years, when they drop all support for 
any prior to 3.6 or 3.7. A tiny benefit, discounted even further because 
it is so far in the distance, versus significant costs.

Application authors, only using a single version of Python, may be able 
to make use of this rather sooner: perhaps as little as three years from 
now.


 
> It's a gotcha. Let's remove it. If we don't remove it, let's give ourselves a 
> tool to detect it.

Um, it's easy to detect it.

    not time_obj

returns True if time_obj is midnight and therefore falsey.



-- 
Steven

From cfkaran2 at gmail.com  Wed Mar  5 20:24:57 2014
From: cfkaran2 at gmail.com (CFK)
Date: Wed, 5 Mar 2014 14:24:57 -0500
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
Message-ID: <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>

On Wed, Mar 5, 2014 at 8:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 5 Mar 2014 22:51, "Paul Moore" <p.f.moore at gmail.com> wrote:
> >
> > On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote:
> > > Thoughts/suggestions?
> >
> > I think the core/stdlib position is that agreeing conventions would be
> > better done once some real world experience of the practical issues
> > and benefits of annotations has been established.
>
> MyPy uses function annotations for optional static typing, which is pretty
> much the use case Guido originally had in mind and the main reason that PEP
> 8 doesn't make combining annotations with an associated decorator
> mandatory: http://www.mypy-lang.org/
>
> You do still have to import a particular module to indicate that all
> annotations in the importing module are to be interpreted as type
> annotations.
>
> Beyond that, the guidance in PEP 8 stands:
>
> """It is recommended that third party experiments with annotations use an
> associated decorator to indicate how the annotation should be
> interpreted."""
>
I like what all of you are suggesting; decorators are the way to go.  If a
project defines its own annotation decorators as
sigtools.modifiers.annotate<http://sigtools.readthedocs.org/en/latest/_modules/sigtools/modifiers.html#annotate>,
mypy, <http://www.mypy-lang.org/> or
pyanno<http://www.fightingquaker.com/pyanno>do, then it shouldn't be
too hard for a project to add its own UUID to the
annotation dictionary.  I'll spend a little while this weekend seeing if I
can come up with some proof-of-concept code to make this work in a portable
way.  If the signatures looked vaguely like the following:

@type_annotation(some_arg, int)
@return_type_annotation(int)
def function(some_arg):
    pass

Would that be appealing to everyone?

One question though, are we allowed to modify a functions __annotation__
dictionary directly?  I know that I can do it, I just don't know if it is
discouraged.

Thanks,
Cem Karan
<http://www.fightingquaker.com/pyanno>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f8031bdd/attachment.html>

From breamoreboy at yahoo.co.uk  Wed Mar  5 20:35:29 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 19:35:29 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <201403052118.38295.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org>
 <201403052118.38295.shai@platonix.com>
Message-ID: <lf7u9q$kdv$1@ger.gmane.org>

On 05/03/2014 19:18, Shai Berger wrote:
> On Wednesday 05 March 2014 20:47:02 Mark Lawrence wrote:
>> On 05/03/2014 18:33, Shai Berger wrote:
>>> On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote:
>>>> On 05/03/2014 18:12, Amber Yust wrote:
>>>>> I think this is the wrong stage to evaluate cost. After all, one of the
>>>>> tenets of open source is that the core devs don't have to be the source
>>>>> of all change.
>>>>
>>>> But the core devs have to review the change and ultimately decide
>>>> whether or not to commit it.  In this case it looks as if a patch would
>>>> not be accepted, so why keep going on about it, especially when there's
>>>> a known work around?
>>>
>>> As noted before, the workaround is trivial once you detect the problem.
>>> Detecting the problem is not easy.
>>>
>>> Shai.
>>
>> Use mocks in unit testing?
>
> Unless you're aware of the issue, you are not going to unit-test specifically
> for midnight. If you are aware, you don't need to detect it, you write "is not
> None".

So you have your solution.  Good, now can we please move on as this is 
getting tedious.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From rymg19 at gmail.com  Wed Mar  5 20:44:01 2014
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 5 Mar 2014 13:44:01 -0600
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
 <20140305014611.GA4396@cskk.homeip.net>
 <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
Message-ID: <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com>

On Tue, Mar 4, 2014 at 8:36 PM, David Mertz <mertz at gnosis.cx> wrote:

> On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote:
>
>> >   foo = 1
>> >   a = $(foo + 1)
>> Definitely nicer. Still irrationally uncomfortable about the "$" though.
>> A thought, though it could break existing code (and nested tuples, alas):
>>
>>     a = (( foo + 1 ))
>>
>
> That looks like unresolvable ambiguity to me.  I confess that I am more
> comfortable with '$(...)' because I'm one of those folks who actually likes
> bash, and uses that spelling often over there (where the meaning isn't the
> *same* as this, but is enough similar for the meaning to carry over)
>
>

But this is Python, which is 10x better. And besides, that syntax gives me
GNU make nightmares.


>  Still, what does this mean?
>>     a = 3 + (( foo + 1 ))
>> I think that would need to be a syntax error, because I can't see it being
>> anything except nonsense otherwise.
>
>
I see it as:

Create an anonymous function object that adds foo to 1. Then, try and add 3
to that resulting object(which obviously would fail). It'd be kind of like:

a = 3 + (lambda: foo+1)

a.k.a:

def myfunc(): return foo+1
a = 3+myfunc

or(somewhat clearer in C++):

SomeType a = 3 + [&]() { return foo+1; };

It's a bit more obvious of the error in the C++ example(or, at least to me).

...
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/b26c4ce0/attachment.html>

From haoyi.sg at gmail.com  Wed Mar  5 20:50:36 2014
From: haoyi.sg at gmail.com (Haoyi Li)
Date: Wed, 5 Mar 2014 11:50:36 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com>
References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com>
 <20140305014611.GA4396@cskk.homeip.net>
 <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com>
Message-ID: <CALruUQJZkn=QKOAv2-9RrGaiCtJQ2Xfk=r+3rKGnUbH7OLPyDg@mail.gmail.com>

> The idea was never to start passing complex expressions around the place.
It was just to allow a function to take an expression containing about one
or two names, and evaluate the expression internally, so this...
>
> func(lambda foo: foo > 0)
>
> ...can become...
>
> func($foo > 0)

>>> from macropy.quick_lambda import macros, f, _
>>> map(f[_ + 1], [1, 2, 3])
[2, 3, 4]

>From an external API, it seems exactly the same. From an internal point of
view, you have to pass *foo* explicitly to the function you capture, but
I'd argue that that's probably a good thing in the majority of cases. There
are cases where you'd want to "evaluate the expression internally" to
automagically inject identifiers into the scope of the expression, but
those are few and far between.


On Wed, Mar 5, 2014 at 11:44 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:

> On Tue, Mar 4, 2014 at 8:36 PM, David Mertz <mertz at gnosis.cx> wrote:
>
>> On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote:
>>
>>> >   foo = 1
>>> >   a = $(foo + 1)
>>> Definitely nicer. Still irrationally uncomfortable about the "$" though.
>>> A thought, though it could break existing code (and nested tuples, alas):
>>>
>>>     a = (( foo + 1 ))
>>>
>>
>> That looks like unresolvable ambiguity to me.  I confess that I am more
>> comfortable with '$(...)' because I'm one of those folks who actually likes
>> bash, and uses that spelling often over there (where the meaning isn't the
>> *same* as this, but is enough similar for the meaning to carry over)
>>
>>
>
> But this is Python, which is 10x better. And besides, that syntax gives me
> GNU make nightmares.
>
>
>>  Still, what does this mean?
>>>     a = 3 + (( foo + 1 ))
>>> I think that would need to be a syntax error, because I can't see it
>>> being
>>> anything except nonsense otherwise.
>>
>>
> I see it as:
>
> Create an anonymous function object that adds foo to 1. Then, try and add
> 3 to that resulting object(which obviously would fail). It'd be kind of
> like:
>
> a = 3 + (lambda: foo+1)
>
> a.k.a:
>
> def myfunc(): return foo+1
> a = 3+myfunc
>
> or(somewhat clearer in C++):
>
> SomeType a = 3 + [&]() { return foo+1; };
>
> It's a bit more obvious of the error in the C++ example(or, at least to
> me).
>
> ...
>> --
>> Keeping medicines from the bloodstreams of the sick; food
>> from the bellies of the hungry; books from the hands of the
>> uneducated; technology from the underdeveloped; and putting
>> advocates of freedom in prisons.  Intellectual property is
>> to the 21st century what the slave trade was to the 16th.
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e9cf69e5/attachment.html>

From abarnert at yahoo.com  Wed Mar  5 21:30:23 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 12:30:23 -0800 (PST)
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <20140305112532.GV28804@ando>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com>
 <20140305112532.GV28804@ando>
Message-ID: <1394051423.82411.YahooMailNeo@web181001.mail.ne1.yahoo.com>

From: Steven D'Aprano <steve at pearwood.info>

Sent: Wednesday, March 5, 2014 3:25 AM


> On Tue, Mar 04, 2014 at 05:05:39PM -0800, Andrew Barnert wrote:
>>  On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> 
> wrote:
>> 
>>  > Steven D'Aprano wrote:
>>  >> What I have in my head is some vague concept that the Python 
> evaluation rules will somehow know when to evaluate the thunk and when to treat 
> it as an object, which is (as I understand it) what happens in Algol.
>>  > 
>>  > But Algol has the benefit of static typing -- the procedure
>>  > being called explicitly declares whether the argument is to
>>  > be passed by name or value. Python has no idea about that at
>>  > compile time.
> 
> I'm not convinced that this really matters, but for the sake of the 
> argument let's say it does.

It's not really static typing that's key here; it's that, unless you have explicit syntax for all uses of thunks, you either need implicit type casts, and static typing is necessary for implicit type casts.

(However, as I said before, this?doesn't have to be a deal-breaker. You can probably get away with making all references either immediate or delayed, so long as the syntax for doing the other explicitly?isn't too obtrusive and the performance cost isn't too high. For example, if referencing a thunk always evaluates, `thunk` just creates a new thunk that will evaluate the old one, and you could optimize that into not much heavier than just passing the old one around?with a JIT, like PyPy, you could even optimize it into just passing the old one around.)

>>  This is the main reason I think it's more productive to think of this 
>>  in terms of Lisp-style quoting than Algol-style thunks.
> 
> Can you give more detail please?


I think I've already explained it, but let me try again in more detail.

A quoted expression and a thunk are similar things: ways to turn a language expression into something that can be evaluated or executed later. But there are two major differences. First, quoted expressions are first-class values, while thunks are not. Second, quoted expressions have an inspectable (or pattern-matchable) structure, while thunks do not.

You could relate that second difference to ASTs vs. code objects in Python?but since idiomatic Python does not use macros or any other generation or parsing of ASTs, it really isn't a visible difference, so the first one is more important.*

And then there's the historical difference: thunks come from static languages, quoting from dynamic languages. Together with being first-class values,?this means quoted expressions need explicit but not-too-intrusive syntax not just for creating them, but also either for evaluating them, or for continuing to delay them.

So, that's why I think quoting is a more apt model for what we're trying to accomplish in this thread.

* As I mentioned before, it does actually matter for _implementation_ whether we use ASTs or code objects, at least if we want dynamic or flat in-place scoping, because the former would allow us to look up names at execution time, while the latter would not, unless we either add a LOAD_DYNAMIC opcode or compile name lookup into explicit dynamic lookup code. But as long as we define what the scoping rules are, users won't care how that's implemented.

From abarnert at yahoo.com  Wed Mar  5 21:32:38 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 12:32:38 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
Message-ID: <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com>

From: Oscar Benjamin <oscar.j.benjamin at gmail.com>

Sent: Wednesday, March 5, 2014 3:29 AM


> On 5 March 2014 08:10, Andrew Barnert <abarnert at yahoo.com> wrote:
>>  From: Mark H. Harris <harrismh777 at gmail.com>
>>> 
>>> ?  I am expecting that (just as in Rexx numbers) defining very clearly 
> what
>>> is a python number? will be key for wide adaptation of the concept. But 
> there
>>> should be no surprises for users, particularly average users. Anyone 
> with
>>> a middle school expectation of a numeric format should be able to use
>>> python numbers without surprises.
>> 
>>  Anyone with a middle school expectation will expect 1/3 to be a 
> fraction--or, at least, something they can multiply by 3 to get exactly 1. Using 
> an inexact decimal float instead of an inexact binary float is no improvement at 
> all.
> 
> I actually think that it is an improvement. Most people are surprised
> by the fact that just writing x = 0.1 causes a rounding error. Python
> only has dedicated syntax for specifying binary floats in terms of
> decimal digits meaning that there is no syntax for exactly specifying
> non integer numbers. I would say that that is clearly a sub-optimal
> situation.
> 
> I don't agree with Mark's proposal in this thread but I would like to
> have decimal literals e.g. 1.23d, and I would also use Fraction
> literals if available e.g. 1/3F.


I agree with you completely on that. That's kind of my point?your suggestion is almost the _opposite_ of the proposal in this thread. You want to make it easier for people to use the appropriate type for each use case; he wants to eliminate that choice entirely so you never have to make it. The latter would be nice if it were doable, but it's not even possible in principle. So the former is the best choice.


From bruce at leapyear.org  Wed Mar  5 22:04:56 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Wed, 5 Mar 2014 13:04:56 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140305192419.GD28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
Message-ID: <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:24 AM, Steven D'Aprano <steve at pearwood.info>wrote:

> There's also the argument from consistency: midnight is the zero point
> of the day, and zero is falsey. It would be surprising if something
> which is essentially zero became truthy.
>

I don't think it's consistent now.

In Boolean contexts ...

   - a timedelta<http://docs.python.org/3.4/library/datetime.html#datetime.timedelta>
object
   is considered to be true if and only if it isn't equal to timedelta(0).
   - all date<http://docs.python.org/3.4/library/datetime.html#datetime.date>
objects
   are considered to be true.
   - all datetime<http://docs.python.org/3.4/library/datetime.html#datetime.datetime>
objects
   are considered to be true.
   - a time <http://docs.python.org/3.4/library/datetime.html#datetime.time>
object
   is considered to be true if and only if, after converting it to minutes and
   subtracting utcoffset() (or 0 if that's None), the result is non-zero.

And the documentation is not exactly clear for understanding when a time
object is falsey. In particular, the "converting it to minutes" part seems
irrelevant as a time of 1 microsecond from midnight is truthy and
timedeltas are not numbers and are not in minutes. Perhaps a reasonable
improvement would be to change this to:

   - a time <http://docs.python.org/3.4/library/datetime.html#datetime.time>
object
   is considered to be true unless it represents exactly midnight local time,
   that is, it's false if subtracting the utcoffset() from the time
   produces an all-zero result (or it's all-zero to start with if the
   utcoffset is None).


--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/7412a561/attachment.html>

From alex.gaynor at gmail.com  Wed Mar  5 22:10:12 2014
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Wed, 5 Mar 2014 21:10:12 +0000 (UTC)
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
References: <201403051216.12392.shai@platonix.com>
Message-ID: <loom.20140305T220007-633@post.gmane.org>

Shai Berger <shai at ...> writes:

> 
> Hi all,
> 
> This is my first post here, following a recommendation from Alexander 
> Belopolsky to use this list, to try to convince the Python developers to 
> reopen a ticket. I am a long-time Python user, and a Django committer.
> 
> http://bugs.python.org/issue13936 is a complaint about the fact that midnight 


I strongly support fixing this issue. This behavior is so fundamentally
nonsensical that resolving it is worth the potential compatibility issues.

First, I'd like to dispense with the notion that ``if foo`` is somehow a bad
practice. This is an EXTREMELY common practice as a shorthand for checking for
None, a great many people consider it more idiomatic, this really isn't a place
to bikeshed that particlar idiom.

Now I'd like to address whether this is correct behavior: midnight is not a
special time. It does not differ from other times in magnitutde (as with a
container), it is not equivilant to the False value (as with 0). It is simply,
sometimes, represented as 00:00:00 (or 12:00:00!). The fact that it's
representation is composed of zeros no more makes it false than the point in
3-space at (0, 0, 0) is false. It's still a completely normal time, which in
no way distinguishes itself from any other.

I don't have a strong perspective on what the exact deprecation or breakage
cycle should like, but it's very clear to me that this behavior is nonsensical.
Out of a sample of half-a-dozen python programmers I've told about this, the
*universal* response was confusion and revulsion. I strongly suspect that the
majority of people writing ``if time_object`` have a bug in their code.

Alex


From ron3200 at gmail.com  Wed Mar  5 22:13:29 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Wed, 05 Mar 2014 15:13:29 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com>
Message-ID: <lf841b$rt6$1@ger.gmane.org>



On 03/05/2014 02:32 PM, Andrew Barnert wrote:
>>> I don't agree with Mark's proposal in this thread but I would like
>>> to have decimal literals e.g. 1.23d, and I would also use Fraction
>>> literals if available e.g. 1/3F.
>
> I agree with you completely on that. That's kind of my point?your
> suggestion is almost the_opposite_  of the proposal in this thread. You
> want to make it easier for people to use the appropriate type for each
> use case; he wants to eliminate that choice entirely so you never have
> to make it. The latter would be nice if it were doable, but it's not
> even possible in principle. So the former is the best choice.

It's also a matter of how far in the future you are looking.  If he waited 
until later to propose something like this, it most likely wouldn't get in. 
  I'm not sure he's expecting a firm answer now, but probably is hoping for 
a "hey lets look into this more and see if it's really possible" kind of 
maybe.  For that, it's good to start early.

On the near term, adding literal syntax's as you describe here would get 
all the pieces into python.  Then an import from future could enable what 
Mark is thinking.  He really needs to flesh out the details first before we 
can make any objective opinions about how it would actually work.

The way I see it, with a unified number type, we will still need context 
like api's to get the more specialised behaviours.  The difference may be a 
decorator on a function that specifies some subset of number types to use 
rather than notating each literal and casting each value.

    @numbers(int, decimal)
    def acounting_foo(...):
       ...
       # using ints for integers
       # using decimal for reals
       ...

That just a guess... maybe Mark can give some examples how what he's 
proposing would work. ?

Cheers,
    Ron


From mertz at gnosis.cx  Wed Mar  5 22:20:59 2014
From: mertz at gnosis.cx (David Mertz)
Date: Wed, 5 Mar 2014 13:20:59 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <20140305114344.GX28804@ando>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <20140305034945.GA67087@cskk.homeip.net>
 <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>
 <20140305114344.GX28804@ando>
Message-ID: <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com>

On Wed, Mar 5, 2014 at 3:43 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> >That is, what's really the point of having
> >
> >   a = $(expr)  # or `expr`, or `(expr), or c"expr"
> >
> > If it's simply a slightly shorter way of spelling:
> >
> >   a = compile(expr, "<string>", "eval")
>
> You can't write it like that. You have to wrap the expression is quotes
> and turn it into a string:
>
>     a = compile("expr", "<string>", "eval")
>

True, I misspelled my example.  Or maybe my 'expr' implicitly meant
string_expr :-).  But the basic point that the literal for a "thunk" is
already essentially available with compile() remains.


> which means you lose syntax highlighting. Perhaps the ability to get
> syntax highlighting is not sufficient to justify this idea.
>

There's no reason you even NEED to lose syntax highlighting.  A code editor
could perfectly well have a highlight rule that strings inside compile()
calls get highlighted.  Sure, that's a pretty special language mode, but
there's nothing that an editor couldn't do in principle.


> Another disadvantage: the ever-present temptation to pass a
> user-generated string to compile:
>
>     a = compile(template % some_string, "<string>", "eval")
>
> If some_string came from an untrusted source, you are now the proud
> owner of a brand new code injection vulnerability. But with syntax, you
> cannot generate a thunk/macro except from code you write yourself. It's
> still code written by you, it's just that evaluation is delayed.
>

The literal hardly saves you from injection attacks.  I could write this
too under the proposed idea:

  foo = get_string_from_attacker()
  a = $(foo)
  b = a.eval()

Now one can say "don't do that!" ... but that advice applies just as well
to 'compile(unsafe_string, ...)'

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/d4fd1e42/attachment.html>

From donald at stufft.io  Wed Mar  5 22:31:36 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 16:31:36 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <loom.20140305T220007-633@post.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
Message-ID: <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>


On Mar 5, 2014, at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:

> Shai Berger <shai at ...> writes:
> 
>> 
>> Hi all,
>> 
>> This is my first post here, following a recommendation from Alexander 
>> Belopolsky to use this list, to try to convince the Python developers to 
>> reopen a ticket. I am a long-time Python user, and a Django committer.
>> 
>> http://bugs.python.org/issue13936 is a complaint about the fact that midnight 
> 
> 
> I strongly support fixing this issue. This behavior is so fundamentally
> nonsensical that resolving it is worth the potential compatibility issues.
> 
> First, I'd like to dispense with the notion that ``if foo`` is somehow a bad
> practice. This is an EXTREMELY common practice as a shorthand for checking for
> None, a great many people consider it more idiomatic, this really isn't a place
> to bikeshed that particlar idiom.
> 
> Now I'd like to address whether this is correct behavior: midnight is not a
> special time. It does not differ from other times in magnitutde (as with a
> container), it is not equivilant to the False value (as with 0). It is simply,
> sometimes, represented as 00:00:00 (or 12:00:00!). The fact that it's
> representation is composed of zeros no more makes it false than the point in
> 3-space at (0, 0, 0) is false. It's still a completely normal time, which in
> no way distinguishes itself from any other.
> 
> I don't have a strong perspective on what the exact deprecation or breakage
> cycle should like, but it's very clear to me that this behavior is nonsensical.
> Out of a sample of half-a-dozen python programmers I've told about this, the
> *universal* response was confusion and revulsion. I strongly suspect that the
> majority of people writing ``if time_object`` have a bug in their code.
> 
> Alex
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

I completely agree with Alex here. This is a ridiculous behavior and it should be fixed.

I find the argument that it would only be fixed in 3.something and thus not be really
fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python
so by that logic we should simply give up on fixing anything in Python ever?

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/1a0e9fc6/attachment.sig>

From rosuav at gmail.com  Wed Mar  5 22:31:47 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Mar 2014 08:31:47 +1100
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <20140305034945.GA67087@cskk.homeip.net>
 <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>
 <20140305114344.GX28804@ando>
 <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com>
Message-ID: <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com>

On Thu, Mar 6, 2014 at 8:20 AM, David Mertz <mertz at gnosis.cx> wrote:
> The literal hardly saves you from injection attacks.  I could write this too
> under the proposed idea:
>
>   foo = get_string_from_attacker()
>   a = $(foo)
>   b = a.eval()
>
> Now one can say "don't do that!" ... but that advice applies just as well to
> 'compile(unsafe_string, ...)'

That'll just be like doing:

b = foo

So it's still safe. That's the point.

ChrisA

From alexander.belopolsky at gmail.com  Wed Mar  5 22:36:14 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 16:36:14 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <loom.20140305T220007-633@post.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
Message-ID: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>

On Wed, Mar 5, 2014 at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:

> I strongly support fixing this issue. This behavior is so fundamentally
> nonsensical that resolving it is worth the potential compatibility issues.
>

Does anyone know a language that (a) has a time-of-day class in stdlib; (b)
allows
instances of that class to be used in boolean context, and (c) has a
midnight
represented by an instance that is truthy?

Here is my example of an entirely independent of Python language [1] that
nevertheless
has the same behavior for time-of-day objects:

q)`boolean$00:00:00.000
0b
q)`boolean$00:00:00.001
1b

[1] http://code.kx.com/wiki/Reference/Datatypes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bdea6e6b/attachment.html>

From mertz at gnosis.cx  Wed Mar  5 22:40:25 2014
From: mertz at gnosis.cx (David Mertz)
Date: Wed, 5 Mar 2014 13:40:25 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com>
References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com>
 <20140305034945.GA67087@cskk.homeip.net>
 <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com>
 <20140305114344.GX28804@ando>
 <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com>
 <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com>
Message-ID: <CAEbHw4ZvObizopJTJ=anBqZ7rfPsQyaAJTZtXm6KHDYtcYAang@mail.gmail.com>

On Wed, Mar 5, 2014 at 1:31 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Thu, Mar 6, 2014 at 8:20 AM, David Mertz <mertz at gnosis.cx> wrote:
> > The literal hardly saves you from injection attacks.  I could write this
> too
> > under the proposed idea:
> >
> >   foo = get_string_from_attacker()
> >   a = $(foo)
> >   b = a.eval()
> >
> > Now one can say "don't do that!" ... but that advice applies just as
> well to
> > 'compile(unsafe_string, ...)'
>
> That'll just be like doing:
>
> b = foo
>
> So it's still safe. That's the point.
>

Doh! You are right.  The literal does make it somewhat harder to shoot
yourself in the foot with code injection, I had a thinko there.  Still,
advice in the docs not to do 'compile(untrusted_string, ...)' feels like it
pretty much does what we actually need.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a1946eea/attachment-0001.html>

From rosuav at gmail.com  Wed Mar  5 22:50:06 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Mar 2014 08:50:06 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
Message-ID: <CAPTjJmoSvEw+cMuxhs5_Kqo6K2jrzQMZM3OcfaZG1dhmj9XzJw@mail.gmail.com>

On Thu, Mar 6, 2014 at 8:36 AM, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> On Wed, Mar 5, 2014 at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>
>> I strongly support fixing this issue. This behavior is so fundamentally
>> nonsensical that resolving it is worth the potential compatibility issues.
>
>
> Does anyone know a language that (a) has a time-of-day class in stdlib; (b)
> allows
> instances of that class to be used in boolean context, and (c) has a
> midnight
> represented by an instance that is truthy?

Pike fits all three of those, but (c) is trivially true: any object in
Pike is truthy. Of course, you can choose to represent time-of-day
with an integer, in which case an integer of 0 will be false; but if
you use floating-point (for sub-second accuracy), then it's still
safe, because any float is truthy (even 0.0 - and if you think that's
illogical, consider that expecting 0.0 to be false is effectively
making an implicit equality comparison, with all the issues of
equality comparisons on floats _plus_ the issues of doing that
implicitly). If Python didn't have to worry about backward
compatibility (that is, if this were being implemented new now), then
I'd say a null time range should be false, but any date, datetime, or
time, regardless of what it represents, is true. Maybe there could be
a "null date" value that isn't a date, but None should be able to do
that.

ChrisA

From greg.ewing at canterbury.ac.nz  Wed Mar  5 22:51:13 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 10:51:13 +1300
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando>
 <5316544D.7050302@canterbury.ac.nz>
 <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
Message-ID: <53179C51.4020001@canterbury.ac.nz>

Masklinn wrote:
> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
>>But Algol has the benefit of static typing 
> 
> That's not really a blocker though, Haskell thunks are implicit and not
> type-encoded. A name may correspond to a (unforced) thunk or to a strict
> value

That comes at the expense of making *everything* a thunk
until its value is needed. This would be a big, far-reaching
change in the way Python works.

Also, Haskell has the advantage of knowing that the value
won't change, so it can replace the thunk with its value
once it's been calculated. Python would have to keep it
as a thunk and reevaluate it every time.

And Haskell doesn't have to provide a way of passimg the
thunk around *without* evaluating it.

-- 
Greg

From mal at egenix.com  Wed Mar  5 23:04:05 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 05 Mar 2014 23:04:05 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
Message-ID: <53179F55.3080501@egenix.com>

I don't know what all the fuzz is about. There are lots of types
in Python that evaluate to False in certain corner cases, e.g.
"", (), [], {}, 0, 0.0, 0j, None, etc.

datetime.time(0,0,0) is just another one :-)

BTW: If you want to test for None, well, then do test for None and
not for some property of None:

if x is None:
    print ("x is None, which implies it's False")
    print ("Oh, and do note that the inverse is not necessarily true :-)")

In logic:

A -> B does not imply B -> A; it only implies (not B) -> (not A).

in Python again:

if bool(x) is not False:
     print ("x is not None")

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From donald at stufft.io  Wed Mar  5 23:10:28 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 17:10:28 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53179F55.3080501@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com>
Message-ID: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>


On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> I don't know what all the fuzz is about. There are lots of types
> in Python that evaluate to False in certain corner cases, e.g.
> "", (), [], {}, 0, 0.0, 0j, None, etc.
> 
> datetime.time(0,0,0) is just another one :-)
> 
> BTW: If you want to test for None, well, then do test for None and
> not for some property of None:
> 
> if x is None:
>    print ("x is None, which implies it's False")
>    print ("Oh, and do note that the inverse is not necessarily true :-)")
> 
> In logic:
> 
> A -> B does not imply B -> A; it only implies (not B) -> (not A).
> 
> in Python again:
> 
> if bool(x) is not False:
>     print ("x is not None")
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Most of those are pretty easy to reason about being ?False?, they all have some
concept of empty, nothing, etc. Midnight however is not any more or less a time
than one minute past midnight. Even worse, if you consider not explicitly
checking for None a bug in that application code, debugging that code is made
much more difficult because of the fact that for some reason "midnight" is
considered False.

That is the real problem here, sure people should be using ``is None`` checks
but until people are perfect and never write buggy code, we have to contend
with the fact that sometimes people will not write ``is None``. Making that
particular error harder to debug is doing nothing but wasting their time and
having Python be punitive for what in many cases may be a mistake.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/b5fd660e/attachment.sig>

From ben+python at benfinney.id.au  Wed Mar  5 23:16:06 2014
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 06 Mar 2014 09:16:06 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com>
Message-ID: <85siqwcmvt.fsf@benfinney.id.au>

"M.-A. Lemburg" <mal at egenix.com> writes:

> I don't know what all the fuzz is about. There are lots of types
> in Python that evaluate to False in certain corner cases, e.g.
> "", (), [], {}, 0, 0.0, 0j, None, etc.
>
> datetime.time(0,0,0) is just another one :-)

The salient difference is:

With the former set, they all embody concepts of either ?empty? or ?zero
magnitude?. They strongly connote ?not there? within their domain of
values, hence map well to Boolean false.

With ?datetime.time? there is no equivalent: midnight is not special in
the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
on a continuum, and is not a ?not there? value. It should not be Boolean
false any more than any other time.

-- 
 \         ?What is it that makes a complete stranger dive into an icy |
  `\   river to save a solid gold baby? Maybe we'll never know.? ?Jack |
_o__)                                                           Handey |
Ben Finney


From mal at egenix.com  Wed Mar  5 23:27:07 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 05 Mar 2014 23:27:07 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>
 <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
Message-ID: <5317A4BB.1090303@egenix.com>

On 05.03.2014 23:10, Donald Stufft wrote:
> 
> On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> I don't know what all the fuzz is about. There are lots of types
>> in Python that evaluate to False in certain corner cases, e.g.
>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>
>> datetime.time(0,0,0) is just another one :-)
>>
> Most of those are pretty easy to reason about being ?False?, they all have some
> concept of empty, nothing, etc. Midnight however is not any more or less a time
> than one minute past midnight. Even worse, if you consider not explicitly
> checking for None a bug in that application code, debugging that code is made
> much more difficult because of the fact that for some reason "midnight" is
> considered False.

The reasoning becomes clear if you regard a time value as
number of seconds since midnight (the time of day is a relative
measure). This is 0 for time(0,0,0).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From donald at stufft.io  Wed Mar  5 23:28:08 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 17:28:08 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317A4BB.1090303@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>
 <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
 <5317A4BB.1090303@egenix.com>
Message-ID: <D25CFB2C-3D48-4D4A-90FC-67E646AE5340@stufft.io>


On Mar 5, 2014, at 5:27 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 05.03.2014 23:10, Donald Stufft wrote:
>> 
>> On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> 
>>> I don't know what all the fuzz is about. There are lots of types
>>> in Python that evaluate to False in certain corner cases, e.g.
>>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>> 
>>> datetime.time(0,0,0) is just another one :-)
>>> 
>> Most of those are pretty easy to reason about being ?False?, they all have some
>> concept of empty, nothing, etc. Midnight however is not any more or less a time
>> than one minute past midnight. Even worse, if you consider not explicitly
>> checking for None a bug in that application code, debugging that code is made
>> much more difficult because of the fact that for some reason "midnight" is
>> considered False.
> 
> The reasoning becomes clear if you regard a time value as
> number of seconds since midnight (the time of day is a relative
> measure). This is 0 for time(0,0,0).

Except time isn?t number of seconds since midnight. 

> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/0f874a63/attachment.sig>

From mal at egenix.com  Wed Mar  5 23:35:02 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 05 Mar 2014 23:35:02 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>
 <85siqwcmvt.fsf@benfinney.id.au>
Message-ID: <5317A696.8040605@egenix.com>

On 05.03.2014 23:16, Ben Finney wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
> 
>> I don't know what all the fuzz is about. There are lots of types
>> in Python that evaluate to False in certain corner cases, e.g.
>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>
>> datetime.time(0,0,0) is just another one :-)
> 
> The salient difference is:
> 
> With the former set, they all embody concepts of either ?empty? or ?zero
> magnitude?. They strongly connote ?not there? within their domain of
> values, hence map well to Boolean false.
> 
> With ?datetime.time? there is no equivalent: midnight is not special in
> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
> on a continuum, and is not a ?not there? value.

This is true in a date/time continuum, but not true if you regard
time as "time of day". In the latter interpretation, the day starts
at midnight, so the day is "not there" yet at midnight.

> It should not be Boolean false any more than any other time.

Just like with most interpretations, you can just as easily find
one which implies the exact opposite.

It's all smoke and mirrors anyway :-)

Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
0.0 seconds some day. That's what mxDateTime does for time values.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From greg.ewing at canterbury.ac.nz  Wed Mar  5 23:36:09 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 11:36:09 +1300
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305120410.GY28804@ando>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
Message-ID: <5317A6D9.5040704@canterbury.ac.nz>

Steven D'Aprano wrote:
> (By the way, I think it is somewhat amusing that Python not only has a 
> built-in complex type, but also *syntax* for creating complex numbers, 
> but no built-in support for exact rationals.)

I gather that Guido's experiences with ABC have led him
to believe such a feature would be an attractive nuisance.
Rationals have some nasty performance traps if you use
them without being fully aware of what you're doing.

-- 
Greg

From donald at stufft.io  Wed Mar  5 23:38:12 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 17:38:12 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317A696.8040605@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>
 <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com>
Message-ID: <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>


On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 05.03.2014 23:16, Ben Finney wrote:
>> "M.-A. Lemburg" <mal at egenix.com> writes:
>> 
>>> I don't know what all the fuzz is about. There are lots of types
>>> in Python that evaluate to False in certain corner cases, e.g.
>>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>> 
>>> datetime.time(0,0,0) is just another one :-)
>> 
>> The salient difference is:
>> 
>> With the former set, they all embody concepts of either ?empty? or ?zero
>> magnitude?. They strongly connote ?not there? within their domain of
>> values, hence map well to Boolean false.
>> 
>> With ?datetime.time? there is no equivalent: midnight is not special in
>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
>> on a continuum, and is not a ?not there? value.
> 
> This is true in a date/time continuum, but not true if you regard
> time as "time of day". In the latter interpretation, the day starts
> at midnight, so the day is "not there" yet at midnight.

The day is absolutely here at midnight. Midnight isn?t some void where
time ceases to exist.

> 
>> It should not be Boolean false any more than any other time.
> 
> Just like with most interpretations, you can just as easily find
> one which implies the exact opposite.
> 
> It's all smoke and mirrors anyway :-)
> 
> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
> 0.0 seconds some day. That's what mxDateTime does for time values.
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/98b1c4db/attachment.sig>

From alexander.belopolsky at gmail.com  Wed Mar  5 23:38:53 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 17:38:53 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
Message-ID: <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 5:16 PM, Ben Finney <ben+python at benfinney.id.au>wrote:

> .. midnight is not special in
> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
> on a continuum, ..
>

Tell that to people who gather in the Times Square on New Year Eve!

It is very common to implement time-of-day in the languages that lack a
dedicated class as seconds since midnight or fraction of a 24 hour day or
any similar scheme.  In all these schemes midnight is false in boolean
context.

Note that when datetime module was introduced, Python did not have a time
class and people were using int or float to denote time-of-day.
Compatibility with older schemes could be part of the original motivation.
 It also made more sense when the dunder method was spelled __nonzero__.

Whatever the reason was - this is a well established feature and it won't
change.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bcce54e8/attachment-0001.html>

From ncoghlan at gmail.com  Wed Mar  5 23:43:59 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 6 Mar 2014 08:43:59 +1000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com>
 <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
Message-ID: <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com>

On 6 Mar 2014 08:11, "Donald Stufft" <donald at stufft.io> wrote:
>
> Most of those are pretty easy to reason about being "False", they all
have some
> concept of empty, nothing, etc. Midnight however is not any more or less
a time
> than one minute past midnight. Even worse, if you consider not explicitly
> checking for None a bug in that application code, debugging that code is
made
> much more difficult because of the fact that for some reason "midnight" is
> considered False.
>
> That is the real problem here, sure people should be using ``is None``
checks
> but until people are perfect and never write buggy code, we have to
contend
> with the fact that sometimes people will not write ``is None``. Making
that
> particular error harder to debug is doing nothing but wasting their time
and
> having Python be punitive for what in many cases may be a mistake.

Right, the core problem here is changing an often harmless style problem
into a subtle behavioural bug. I changed the title on
http://bugs.python.org/issue13936 accordingly, reopened it and described
the appropriate next steps for anyone that is motivated enough about the
issue to actually do the work.

We'll see if the enthusiasm for changing the behaviour translates into
anyone being sufficiently interested to work out the details of the
deprecation warning and the associated documentation and test updates.

Regards,
Nick.

>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
DCFA
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/47b44c79/attachment.html>

From donald at stufft.io  Wed Mar  5 23:45:34 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 17:45:34 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com>
 <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
 <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com>
Message-ID: <6A78ACBF-1628-499D-9A31-F951DDF062D1@stufft.io>


On Mar 5, 2014, at 5:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> 
> On 6 Mar 2014 08:11, "Donald Stufft" <donald at stufft.io> wrote:
> >
> > Most of those are pretty easy to reason about being ?False?, they all have some
> > concept of empty, nothing, etc. Midnight however is not any more or less a time
> > than one minute past midnight. Even worse, if you consider not explicitly
> > checking for None a bug in that application code, debugging that code is made
> > much more difficult because of the fact that for some reason "midnight" is
> > considered False.
> >
> > That is the real problem here, sure people should be using ``is None`` checks
> > but until people are perfect and never write buggy code, we have to contend
> > with the fact that sometimes people will not write ``is None``. Making that
> > particular error harder to debug is doing nothing but wasting their time and
> > having Python be punitive for what in many cases may be a mistake.
> 
> Right, the core problem here is changing an often harmless style problem into a subtle behavioural bug. I changed the title on http://bugs.python.org/issue13936 accordingly, reopened it and described the appropriate next steps for anyone that is motivated enough about the issue to actually do the work.
> 
> We'll see if the enthusiasm for changing the behaviour translates into anyone being sufficiently interested to work out the details of the deprecation warning and the associated documentation and test updates.
> 
> Regards,
> Nick.
> 
> >
> > -----------------
> > Donald Stufft
> > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/


Thanks Nick :)

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/ac1f49fe/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/ac1f49fe/attachment.sig>

From breamoreboy at yahoo.co.uk  Wed Mar  5 23:47:45 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 22:47:45 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com>
Message-ID: <lf89ib$sqs$1@ger.gmane.org>

On 05/03/2014 22:38, Alexander Belopolsky wrote:
>
> On Wed, Mar 5, 2014 at 5:16 PM, Ben Finney <ben+python at benfinney.id.au
> <mailto:ben+python at benfinney.id.au>> wrote:
>
>     .. midnight is not special in
>     the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
>     on a continuum, ..
>
>
> Tell that to people who gather in the Times Square on New Year Eve!
>
> It is very common to implement time-of-day in the languages that lack a
> dedicated class as seconds since midnight or fraction of a 24 hour day
> or any similar scheme.  In all these schemes midnight is false in
> boolean context.
>
> Note that when datetime module was introduced, Python did not have a
> time class and people were using int or float to denote time-of-day.
> Compatibility with older schemes could be part of the original
> motivation.  It also made more sense when the dunder method was spelled
> __nonzero__.
>
> Whatever the reason was - this is a well established feature and it
> won't change.
>

Have you looked at the issue recently? :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From greg.ewing at canterbury.ac.nz  Wed Mar  5 23:49:25 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 11:49:25 +1300
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
Message-ID: <5317A9F5.8050704@canterbury.ac.nz>

Oscar Benjamin wrote:
> In C and some other languages the f suffix indicates a numeric literal
> that has type "float" e.g. "6.0f". You can use upper case there as
> well but the convention is lower case and to include the .0 when it's
> an integer. I just though that 3F looks sufficiently distinct from the
> way it's typically done in C.

I'm not sure it's different enough; I've been looking at
Java code recently that uses uppercase F.

An alternative would be to use 'r' or 'R' for 'rational',
which would eliminate any chance of confusion.

-- 
Greg

From alexander.belopolsky at gmail.com  Wed Mar  5 23:55:46 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 17:55:46 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf89ib$sqs$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com>
 <lf89ib$sqs$1@ger.gmane.org>
Message-ID: <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 5:47 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> Have you looked at the issue recently? :)


Yes, and I added one more myself:

http://bugs.python.org/issue20855

let's see which one fares better. :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/d8a1fee7/attachment.html>

From breamoreboy at yahoo.co.uk  Thu Mar  6 00:02:50 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 05 Mar 2014 23:02:50 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com>
 <lf89ib$sqs$1@ger.gmane.org>
 <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com>
Message-ID: <lf8aek$51k$1@ger.gmane.org>

On 05/03/2014 22:55, Alexander Belopolsky wrote:
>
> On Wed, Mar 5, 2014 at 5:47 PM, Mark Lawrence
> <breamoreboy at yahoo.co.uk
> <mailto:breamoreboy at yahoo.co.uk>> wrote:
>
>     Have you looked at the issue recently? :)
>
>
> Yes, and I added one more myself:
>
> http://bugs.python.org/issue20855
>
> let's see which one fares better. :-)
>

As 20855 has already been closed, with a comment that I tend to agree 
with, I think it's time for me to grab my trenching tool, tin hat, 
camouflage net and flak jacket and start running before the Batley 
Townswomen Guild get involved.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From masklinn at masklinn.net  Thu Mar  6 00:07:06 2014
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 6 Mar 2014 00:07:06 +0100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <53179C51.4020001@canterbury.ac.nz>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
 <53179C51.4020001@canterbury.ac.nz>
Message-ID: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>

On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Masklinn wrote:
>> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>> But Algol has the benefit of static typing 
>> That's not really a blocker though, Haskell thunks are implicit and not
>> type-encoded. A name may correspond to a (unforced) thunk or to a strict
>> value
> 
> That comes at the expense of making *everything* a thunk
> until its value is needed.

Of course not, Haskell allows explicitly forcing a thunk and the
compiler/runtime pair can use strictness analysis and not create thunks
in the first place.

> This would be a big, far-reaching
> change in the way Python works.

Having Haskell-type thunks does not mean using the same default as
Haskell. Python would have the opposite one, instead of creating thunks
by default it would create them only when requested.

> Also, Haskell has the advantage of knowing that the value
> won't change

I do not see why that would be relevant to thunks. The result of the
thunk may be mutable, so what?

> so it can replace the thunk with its value
> once it's been calculated. Python would have to keep it
> as a thunk and reevaluate it every time.

See above, I do not see why that would happen. Consider that a thunk is
a deferral of an expression's evaluation, why would said expression's
evaluation happen more than once? The only thing which changes is *when*
the actual evaluation happens.

> And Haskell doesn't have to provide a way of passimg the
> thunk around *without* evaluating it.

Well no, instead it takes the sensible option to force thunks when they
need to be forced (or when their forcing is explicitly requested e.g.
using ($!) or "bang pattern" extensions, but the need for those is
mostly because haskell defaults to creating thunks I would say). Why
would passing a reference around force a thunk in the first place?

From mal at egenix.com  Thu Mar  6 00:08:20 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 00:08:20 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>	<85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
Message-ID: <5317AE64.90007@egenix.com>

On 05.03.2014 23:38, Donald Stufft wrote:
> 
> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> On 05.03.2014 23:16, Ben Finney wrote:
>>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>>
>>>> I don't know what all the fuzz is about. There are lots of types
>>>> in Python that evaluate to False in certain corner cases, e.g.
>>>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>>>
>>>> datetime.time(0,0,0) is just another one :-)
>>>
>>> The salient difference is:
>>>
>>> With the former set, they all embody concepts of either ?empty? or ?zero
>>> magnitude?. They strongly connote ?not there? within their domain of
>>> values, hence map well to Boolean false.
>>>
>>> With ?datetime.time? there is no equivalent: midnight is not special in
>>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
>>> on a continuum, and is not a ?not there? value.
>>
>> This is true in a date/time continuum, but not true if you regard
>> time as "time of day". In the latter interpretation, the day starts
>> at midnight, so the day is "not there" yet at midnight.
> 
> The day is absolutely here at midnight. Midnight isn?t some void where
> time ceases to exist.

No, but midnight starts the day, just like 0 starts the positive
real numbers (itself not being positive) :-)

>>> It should not be Boolean false any more than any other time.
>>
>> Just like with most interpretations, you can just as easily find
>> one which implies the exact opposite.
>>
>> It's all smoke and mirrors anyway :-)
>>
>> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
>> 0.0 seconds some day. That's what mxDateTime does for time values.
>>
>> -- 
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
>>
>> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>               http://www.egenix.com/company/contact/
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From donald at stufft.io  Thu Mar  6 00:10:12 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 18:10:12 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317AE64.90007@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>	<85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com>
Message-ID: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>


On Mar 5, 2014, at 6:08 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 05.03.2014 23:38, Donald Stufft wrote:
>> 
>> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> 
>>> On 05.03.2014 23:16, Ben Finney wrote:
>>>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>>> 
>>>>> I don't know what all the fuzz is about. There are lots of types
>>>>> in Python that evaluate to False in certain corner cases, e.g.
>>>>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>>>> 
>>>>> datetime.time(0,0,0) is just another one :-)
>>>> 
>>>> The salient difference is:
>>>> 
>>>> With the former set, they all embody concepts of either ?empty? or ?zero
>>>> magnitude?. They strongly connote ?not there? within their domain of
>>>> values, hence map well to Boolean false.
>>>> 
>>>> With ?datetime.time? there is no equivalent: midnight is not special in
>>>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
>>>> on a continuum, and is not a ?not there? value.
>>> 
>>> This is true in a date/time continuum, but not true if you regard
>>> time as "time of day". In the latter interpretation, the day starts
>>> at midnight, so the day is "not there" yet at midnight.
>> 
>> The day is absolutely here at midnight. Midnight isn?t some void where
>> time ceases to exist.
> 
> No, but midnight starts the day, just like 0 starts the positive
> real numbers (itself not being positive) :-)

And yet 0 is False not because it starts the positive real numbers (not sure
what that has to do with a value in an integer type) but because it signifies
an empty or zero magnitude object.

> 
>>>> It should not be Boolean false any more than any other time.
>>> 
>>> Just like with most interpretations, you can just as easily find
>>> one which implies the exact opposite.
>>> 
>>> It's all smoke and mirrors anyway :-)
>>> 
>>> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
>>> 0.0 seconds some day. That's what mxDateTime does for time values.
>>> 
>>> -- 
>>> Marc-Andre Lemburg
>>> eGenix.com
>>> 
>>> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>> ________________________________________________________________________
>>> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
>>> 
>>> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>>> 
>>>  eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>   D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>          Registered at Amtsgericht Duesseldorf: HRB 46611
>>>              http://www.egenix.com/company/contact/
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>> 
>> 
>> -----------------
>> Donald Stufft
>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> 
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/50b4e5bf/attachment-0001.sig>

From tjreedy at udel.edu  Thu Mar  6 00:20:45 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 05 Mar 2014 18:20:45 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <loom.20140305T220007-633@post.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
Message-ID: <lf8bgc$lg2$1@ger.gmane.org>

On 3/5/2014 4:10 PM, Alex Gaynor wrote:

> First, I'd like to dispense with the notion that ``if foo`` is somehow a bad
> practice.

It is when one means 'if foo is not None' as it is as fragile, and in 
the same way, in the same way as using 'a and b or c' as a conditional 
expression.

 > This is an EXTREMELY common practice as a shorthand for checking for
> None, a great many people consider it more idiomatic,

In what universe. I was originally for changing datetime, but seems the 
issue being used as a vehicle to justify and promote buggy programming 
has changed my mind.

-- 
Terry Jan Reedy


From abarnert at yahoo.com  Thu Mar  6 00:20:12 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 15:20:12 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <20140305153529.GC28804@ando>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
 <20140305153529.GC28804@ando>
Message-ID: <AC1CFBD0-FD40-4005-B120-FA6CC425C711@yahoo.com>

On Mar 5, 2014, at 7:35, Steven D'Aprano <steve at pearwood.info> wrote:

> So as you can see, what I've been calling a thunk is not precisely like 
> Algol thunks. One thing which is missing is the dynamic scoping: in the 
> thunk evaluation, the x comes from the caller's scope. But the delayed 
> evaluation part is quite similar, enough that I think the name may be 
> appropriate. Another name for this might be a "promise", as in the 
> promise to execute a computation later.

As I mentioned in one of the previous threads, Alice does lazy evaluation with futures and promises--the same construct it uses for concurrent scheduling, closely related to what Python has in concurrent.futures (Python merges futures and promises into a single object with two different APIs, but the idea is the same).

If `expr` evaluated to an object with the same API as a concurrent.futures.Future, that's not a terrible form of explicit evaluation. But if it's doing something like dynamic scoping, it might look pretty misleading.


From mal at egenix.com  Thu Mar  6 00:22:38 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 00:22:38 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<loom.20140305T220007-633@post.gmane.org>	<CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>	<53179F55.3080501@egenix.com>	<85siqwcmvt.fsf@benfinney.id.au>	<5317A696.8040605@egenix.com>	<CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>	<5317AE64.90007@egenix.com>
 <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
Message-ID: <5317B1BE.4090804@egenix.com>

On 06.03.2014 00:10, Donald Stufft wrote:
> 
> On Mar 5, 2014, at 6:08 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> On 05.03.2014 23:38, Donald Stufft wrote:
>>>
>>> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>
>>>> On 05.03.2014 23:16, Ben Finney wrote:
>>>>> "M.-A. Lemburg" <mal at egenix.com> writes:
>>>>>
>>>>>> I don't know what all the fuzz is about. There are lots of types
>>>>>> in Python that evaluate to False in certain corner cases, e.g.
>>>>>> "", (), [], {}, 0, 0.0, 0j, None, etc.
>>>>>>
>>>>>> datetime.time(0,0,0) is just another one :-)
>>>>>
>>>>> The salient difference is:
>>>>>
>>>>> With the former set, they all embody concepts of either ?empty? or ?zero
>>>>> magnitude?. They strongly connote ?not there? within their domain of
>>>>> values, hence map well to Boolean false.
>>>>>
>>>>> With ?datetime.time? there is no equivalent: midnight is not special in
>>>>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
>>>>> on a continuum, and is not a ?not there? value.
>>>>
>>>> This is true in a date/time continuum, but not true if you regard
>>>> time as "time of day". In the latter interpretation, the day starts
>>>> at midnight, so the day is "not there" yet at midnight.
>>>
>>> The day is absolutely here at midnight. Midnight isn?t some void where
>>> time ceases to exist.
>>
>> No, but midnight starts the day, just like 0 starts the positive
>> real numbers (itself not being positive) :-)
> 
> And yet 0 is False not because it starts the positive real numbers (not sure
> what that has to do with a value in an integer type) but because it signifies
> an empty or zero magnitude object.

Well, the duration between the start of day and midnight is zero
and returning to the interpretation of time of day being a measurement
relative to midnight, this makes time(0,0,0) a zero magnitude
object. q.e.d.

Anyway, it's all up in the air for interpretation :-)

There are reasons to have time(0,0,0) be False just as there
are reasons for it not to be False. There's no clear answer.

>>>>> It should not be Boolean false any more than any other time.
>>>>
>>>> Just like with most interpretations, you can just as easily find
>>>> one which implies the exact opposite.
>>>>
>>>> It's all smoke and mirrors anyway :-)
>>>>
>>>> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
>>>> 0.0 seconds some day. That's what mxDateTime does for time values.
>>>>
>>>> -- 
>>>> Marc-Andre Lemburg
>>>> eGenix.com
>>>>
>>>> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>>> ________________________________________________________________________
>>>> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
>>>>
>>>> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>>>>
>>>>  eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>>   D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>>          Registered at Amtsgericht Duesseldorf: HRB 46611
>>>>              http://www.egenix.com/company/contact/
>>>> _______________________________________________
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>> -----------------
>>> Donald Stufft
>>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>>
>>
>> -- 
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Mar 05 2014)
>>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>> 2014-04-09: PyCon 2014, Montreal, Canada ...               35 days to go
>>
>> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>               http://www.egenix.com/company/contact/
> 
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From alexander.belopolsky at gmail.com  Thu Mar  6 00:22:54 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 18:22:54 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
 <5317AE64.90007@egenix.com>
 <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
Message-ID: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>

On Wed, Mar 5, 2014 at 6:10 PM, Donald Stufft <donald at stufft.io> wrote:

> And yet 0 is False not because it starts the positive real numbers (not
> sure
> what that has to do with a value in an integer type) but because it
> signifies
> an empty or zero magnitude object.
>

No.  It is False because it equals False:

>>> 0 == False
True

Numbers are not containers.  You cannot tell whether a number is empty or
not.  Python interpretation of numbers in boolean context is different from
that of containers.

I think the disagreement stems from different view on what time-of-day is.
 For me, it is the fraction of the day that passed or distance from
midnight and in any case fundamentally some kind of number expressed in a
Babylonian base-60 notation.

I think proponents of bool(time(0)) == True view it as a container of
hours, minutes and seconds.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/97f4cb74/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 00:23:03 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 12:23:03 +1300
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
Message-ID: <5317B1D7.4090305@canterbury.ac.nz>

Carl Smith wrote:
> I was thinking of a short word which describes an 
> expression who's evaluation changes depending on the local 
> circumstances. Like a legal bill, not a dollar bill.

Oh. Well, I must say that this was very far from being
the first interpretation of the word "bill" that sprang
to mind when I saw it.

I wasn't even aware of this meaning of the term "legal
bill" (although admittedly I'm not used to living in a
country where the laws vary wildly from one place
to another).

-- 
Greg



From greg.ewing at canterbury.ac.nz  Thu Mar  6 00:23:15 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 12:23:15 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <201403051259.40289.shai@platonix.com>
 <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com>
Message-ID: <5317B1E3.9020409@canterbury.ac.nz>

Oscar Benjamin wrote:
> The question is surely whether the issue is worth a backwards
> compatibility break not whether the current behaviour is a good idea
> (it clearly isn't).

The currrent behaviour is sufficiently useless that it
seems unlikely there would be much code out there relying
on it.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  6 00:23:28 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 12:23:28 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
Message-ID: <5317B1F0.3070408@canterbury.ac.nz>

Paul Moore wrote:
> Agreed that the odd behaviour of time in a
> boolean context is why this whole class of bugs exists, but it's only
> a subclass if the wider problem that people shouldn't truth-test
> values without thinking

But this bites you even if you *do* think, because there
is no reason to suspect that any time value should be
regarded as false, especially since the docs make no
mention of this.

Note that "fix the docs" is *not* the right response to
this, IMO -- that would just be enshrining a bad design
decision. Better to fix the design while we have the
chance.

-- 
Greg

From oscar.j.benjamin at gmail.com  Thu Mar  6 00:24:46 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Mar 2014 23:24:46 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <5317A6D9.5040704@canterbury.ac.nz>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando> <5317A6D9.5040704@canterbury.ac.nz>
Message-ID: <CAHVvXxTBb1cavd6ipgRDAy7AogSsqyeE1wY-aEQ5saUi17QDBw@mail.gmail.com>

On 5 March 2014 22:36, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Steven D'Aprano wrote:
>>
>> (By the way, I think it is somewhat amusing that Python not only has a
>> built-in complex type, but also *syntax* for creating complex numbers, but
>> no built-in support for exact rationals.)
>
> I gather that Guido's experiences with ABC have led him
> to believe such a feature would be an attractive nuisance.
> Rationals have some nasty performance traps if you use
> them without being fully aware of what you're doing.

I've read statement's to that effect but they were specifically about
having rational as the *default* mode for integer division which I am
not suggesting.

It's already possible to opt-in to using Fractions so that rationals
are used for division. Currently you have to import a module and write
F(1, 7) or F('1/7') everywhere in your code. If it were possible to
write 1/7F or 1/7r or whatever then I've personally written code where
I would have used that so that it would be easier to read, would look
more natural with syntax highlighting etc.


Oscar

From donald at stufft.io  Thu Mar  6 00:28:12 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 18:28:12 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5317B1F0.3070408@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <5317B1F0.3070408@canterbury.ac.nz>
Message-ID: <39FD7C84-C7AB-47EA-9A9C-9216DAE5D286@stufft.io>


On Mar 5, 2014, at 6:23 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Paul Moore wrote:
>> Agreed that the odd behaviour of time in a
>> boolean context is why this whole class of bugs exists, but it's only
>> a subclass if the wider problem that people shouldn't truth-test
>> values without thinking
> 
> But this bites you even if you *do* think, because there
> is no reason to suspect that any time value should be
> regarded as false, especially since the docs make no
> mention of this.
> 
> Note that "fix the docs" is *not* the right response to
> this, IMO -- that would just be enshrining a bad design
> decision. Better to fix the design while we have the
> chance.
> 
> -- 
> Greg
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

The docs do mention it:

> in Boolean contexts, a time object is considered to be true if and only
> if, after converting it to minutes and subtracting utcoffset() (or 0 if that?s
> None), the result is non-zero.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/937e362f/attachment.sig>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 00:33:49 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 12:33:49 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
Message-ID: <5317B45D.6000606@canterbury.ac.nz>

Paul Moore wrote:
> """in Boolean contexts, a time object is considered to be true if and
> only if, after converting it to minutes and subtracting utcoffset()
> (or 0 if that's None), the result is non-zero."""

Aaaaargh... I didn't know this had since been documented.

Wy opinion still stands -- documenting this was the
wrong thing to do. I hope it can still be fixed.

-- 
Greg

From rob.cliffe at btinternet.com  Thu Mar  6 01:21:36 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Mar 2014 00:21:36 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
Message-ID: <5317BF90.9030901@btinternet.com>


On 05/03/2014 17:27, Alexander Belopolsky wrote:
>
> On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com 
> <mailto:shai at platonix.com>> wrote:
>
>     So, I rephrase my request: Would anybody object to a silent
>     warning issued
>     whenever a time-object is evaluated as Boolean?
>
>
> I would.  Users should not be penalized for using a documented behavior.
We hope that the Python documentation describes the semantics of the 
language and packages as completely as is reasonably possible.
So you are effectively saying that no backwards-incompatible change 
should ever be made.
If we accepted that, this discussion would end now.
>  There are legitimate uses for bool(midnight) being False.  Midnight 
> is special in many contexts.  For example, it is uncertain whether 
> midnight belongs to the previous or next day.  If your application 
> wants to group midnight differently from other times - it is perfectly 
> fine to use "if dt.time()" instead of a more verbose "if dt.time() != 
> datetime.time(0, 0)".
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> No virus found in this message.
> Checked by AVG - www.avg.com <http://www.avg.com>
> Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/32cc4221/attachment-0001.html>

From harrismh777 at gmail.com  Thu Mar  6 01:40:17 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Wed, 5 Mar 2014 16:40:17 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lf841b$rt6$1@ger.gmane.org>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com>
 <lf841b$rt6$1@ger.gmane.org>
Message-ID: <a91be33d-6fd9-49c8-8d61-b5d2cdcf2ed4@googlegroups.com>



On Wednesday, March 5, 2014 3:13:29 PM UTC-6, Ron Adam wrote:
>
> It's also a matter of how far in the future you are looking.  If he waited
>  
>
until later to propose something like this, it most likely wouldn't get in. 
>
 

>   I'm not sure he's expecting a firm answer now, but probably is hoping 
> for 
>
> a "hey lets look into this more and see if it's really possible" kind of 
> maybe.  For that, it's good to start early.
>

hi Ron,  yes, correct.   Something like this would require huge effort, 
determined
strategy, and possibly years of planning.  Its not going to happen 
over-night, but
in stages could be brought to fruition given some thought and a lot of 
dialogue.

kind regards, 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/053784ca/attachment.html>

From harrismh777 at gmail.com  Thu Mar  6 01:30:58 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Wed, 5 Mar 2014 16:30:58 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
Message-ID: <28d03184-4934-4840-8f92-082901ae0c30@googlegroups.com>



On Wednesday, March 5, 2014 2:10:18 AM UTC-6, Andrew Barnert wrote:

The types you dislike are not programming concepts that get in the way of 
> human mathematics, they are human mathematical concepts. The integers, the 
> rationals, and the reals all behave differently. And they're not the only 
> types of numbers?Python handles complex numbers natively, and it's very 
> easy to extend it with, say, quaternions or even arbitrary matrices that 
> act like numbers.
>
hi Andrew,  your response reminds me of that scene from 'Star Trek: First 
Contact,'  
when Data was getting a little lippy with her eminence The Borg (the one 
who is many)  
(he was trying to explain to her why she doesn't exist, since her ship was 
destroyed 
just after coming through the chronometric particle vortex--time travel--) 
and she 
responded, "You think so three dimensionally... "

When asked about the Borg hierarchy she stated simply, "You imply disparity 
where none exists..."

Now, that was one hot cybernetic babe.  My point is philosophical, really. 
 I do not dislike 
numerical types.  I simply do not believe they are necessary.  In fact, 
Rexx (in a simple way) 
and other languages too for that matter, have demonstrated that types are 
superficial paradigms in 
symbolic thought that can quite readily be eliminated for most AI 
applications given enough
memory, and enough time.   Fortunately, memory is becoming more plentiful 
(less expensive)
and time is shrinking away through processor speed, pipe-lining, and 
software engineering 
like decimal.Decimal.

Keep in mind that I am NOT proposing the elimination of numbers, I am 
proposing the 
abstraction of *python numbers  (yes, all of them...) *in such a way that 
programmers and
specialists, and accountants and grade school teachers may use Python 
without having
to be computer scientists (all of the work happens under the covers, 
written by someone 
like me, or you) and everything at the top is simple and human readable; 
for them.

Yes, its work.  And, its doable.  A *number *should be no more convoluted 
in a modern age
of symbol set processing than any other abstract idea (just because of 
arbitrary limits 
and abstraction paradigms).     Unifying *python numbers *requires a high 
order paradigm 
shift for sure;  but the concept is absolutely doable.

kind regards,




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a917adbd/attachment.html>

From alexander.belopolsky at gmail.com  Thu Mar  6 01:52:53 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 19:52:53 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317BF90.9030901@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
Message-ID: <CAP7h-xZhNaU-8YqVPe2EhRZXFTejZqNK7xN5G=xoiUv_5FO_Ng@mail.gmail.com>

On Wed, Mar 5, 2014 at 7:21 PM, Rob Cliffe <rob.cliffe at btinternet.com>wrote:

> So you are effectively saying that no backwards-incompatible change should
> ever be made.
> If we accepted that, this discussion would end now.
>

This was the rule for the datetime module for many years and we were able
to put in many nice enhancements in a fully backward compatible way.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/dd2c5ffc/attachment.html>

From harrismh777 at gmail.com  Thu Mar  6 02:06:38 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Wed, 5 Mar 2014 17:06:38 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305135646.GB28804@ando>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
Message-ID: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>



On Wednesday, March 5, 2014 7:56:46 AM UTC-6, Steven D'Aprano wrote:
>
> On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote: 
>
> >    The idea of *python number* means that there are no types, no limits, 
> no 
> > constraints, and that all *python numbers *are dynamic. The upper level 
> > syntax is also very simple; all* python numbers *are simply human. 
>
> What makes this a "python number"? In what way are they "dynamic"? 
>
>
At this point in the discussion please try not to think "implementation," 
rather think
conceptually in an open framework where anything is possible.  Dynamic in 
terms
of *python numbers *is not that far off from the concept of dynamic name 
binding used 
everywhere else in python. 

In terms of unifying numbers as  *python numbers  *which needs to be 
defined at some
point, the idea of dynamic binding means first that numeric symbol sets 
have no type,
and are not 'bound' until run-time, if at all.  Consider:

What is this?   ===>    PI / 2

Is it two numbers with a divisor,?  or a Decimal and an int with a 
divisor,? or a  *python number ?*

Or, is it a string of characters that are readily recognizable to humans 
that means 'the top of
the unit circle," or half_of_PI   whatever I mean by that,  or 
 '1.57079632679489661923132169163975144209858469968755291048747229615390820314'

Dynamic means the symbols sets are bound (name-wise, abstraction-wise) only 
at run-time, and
with some 'smarts' as it were (AI) so that the right things happen under 
the covers (not magic)
because the machine has been taught to think, as it were, about numbers and 
symbols the way
normal humans do (in the kitchen, at high school, at the university, in 
science and business).

Another way to think of this Steven is to think of  'pretty print'  on the 
TI 89 Platinum edition 
graphical programming calculator (my personal favorite).  Anyone who plays 
with it the first time
is frustrated because *numbers *are symbolized instead of printed out in 
strings of numerals. So
if I enter  {1}  [/]  {3}  and press enter,  I get   1/3   <====

Yes, I can press evaluate and get   .33333333333333333   but  1/3 is just 
as good a number as
any other number (and what type is it???)   PI / 2 is another example. 
That's a great number and
I don't really need to see a numeric list of digits to use it, do I... and 
what is its type???  who cares.

Another very simple (and not quite good enough) example is the pdeclib file 
I just created over the
past couple of days.  I have a need within the module to use PI  or half_PI 
 for some method or 
function (but I don't need the user to know that, and I don't want to 
calculate it over and over, and
I want it within accuracy for the context on demand)... so what?  I created 
 __gpi__  global PI. It is
reset to Decimal(0) if the dscale(n) precision context changes.  Then, and 
function that requires PI
of half_PI can check first to see if it is cached  as __gpi__  and if so 
pull it back, or if not calculate
it.   If the context does not change than I half some whopping value of PI 
for the duration of the 
context (but, and this is the big but, if I don't need it --- then it never 
gets bound).

Binding for  *python numbers * must be dynamic, typeless, limitless, and no 
constraints. If I enter a 
formula of symbols into python (whatever I mean by that) python just 
handles it dynamically just
like the TI 89---  if I want explicit control of the processing, I am free 
to ask for that too, just like
the TI 89.

By the by,  complex numbers are no different what-so-ever, nor fractions 
either, nor any other  *number.*
The system needs to be able to parse symbol sets intelligently (Alonzo 
Church, Alan Turning, and John 
McCarthy dreamed of this, but had no way to accomplish it in their day) and 
then dynamically bind 
the naming / representation  to the  abstraction (or implementation) that 
makes sense for that symbol 
set in 'that' context, for 'that' problem.  This is not simply business as 
usual... it requires AI, it requires 
a severe paradigm shift of a very high order.

Kind regards,
marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/8a485596/attachment-0001.html>

From python at mrabarnett.plus.com  Thu Mar  6 02:27:45 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 06 Mar 2014 01:27:45 +0000
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <5317B1D7.4090305@canterbury.ac.nz>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
 <5317B1D7.4090305@canterbury.ac.nz>
Message-ID: <5317CF11.20100@mrabarnett.plus.com>

On 2014-03-05 23:23, Greg Ewing wrote:
> Carl Smith wrote:
>> I was thinking of a short word which describes an
>> expression who's evaluation changes depending on the local
>> circumstances. Like a legal bill, not a dollar bill.
>
> Oh. Well, I must say that this was very far from being
> the first interpretation of the word "bill" that sprang
> to mind when I saw it.
>
> I wasn't even aware of this meaning of the term "legal
> bill" (although admittedly I'm not used to living in a
> country where the laws vary wildly from one place
> to another).
>
In the UK we talk about "Bills" being put before Parliament.

It reminds me of that quotation about the UK and US: ?Two nations
divided by a common language?. In the UK you can pay a bill using a
cheque; in the US you can pay the check using bills (but a "check" is
also a (UK) cheque!).


From tim.peters at gmail.com  Thu Mar  6 02:34:13 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 19:34:13 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317BF90.9030901@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
Message-ID: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>

[Rob Cliffe <rob.cliffe at btinternet.com>]
> We hope that the Python documentation describes the semantics of the
> language and packages as completely as is reasonably possible.
> So you are effectively saying that no backwards-incompatible change should
> ever be made.
> If we accepted that, this discussion would end now.

There should be a very high bias against changing documented behavior
that is in fact working as documented.  The document is a contract
with users.  Contracts shouldn't be violated without truly compelling
cause.

"I am altering the deal. Pray I don't alter it any further."  There's
a reason Darth Vader wasn't proclaimed Python's BDFL ;-)

From tim.peters at gmail.com  Thu Mar  6 02:37:37 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 19:37:37 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317B45D.6000606@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
 <5317B45D.6000606@canterbury.ac.nz>
Message-ID: <CAExdVNmPK8f5vQm7p3RwCXWK0CutaSVc2Bdi+_M2Pv9Jzv749g@mail.gmail.com>

[Paul Moore]
>> """in Boolean contexts, a time object is considered to be true if and
>> only if, after converting it to minutes and subtracting utcoffset()
>> (or 0 if that's None), the result is non-zero."""

[Greg Ewing]
> Aaaaargh... I didn't know this had since been documented.
>
> Wy opinion still stands -- documenting this was the
> wrong thing to do. I hope it can still be fixed.

The quoted text has always been there, from datetime's initial release:

    http://docs.python.org/2.3/lib/datetime-time.html

That was a bit over 9 years ago.

From donald at stufft.io  Thu Mar  6 02:38:41 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 20:38:41 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
Message-ID: <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>


On Mar 5, 2014, at 8:34 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [Rob Cliffe <rob.cliffe at btinternet.com>]
>> We hope that the Python documentation describes the semantics of the
>> language and packages as completely as is reasonably possible.
>> So you are effectively saying that no backwards-incompatible change should
>> ever be made.
>> If we accepted that, this discussion would end now.
> 
> There should be a very high bias against changing documented behavior
> that is in fact working as documented.  The document is a contract
> with users.  Contracts shouldn't be violated without truly compelling
> cause.
> 
> "I am altering the deal. Pray I don't alter it any further."  There's
> a reason Darth Vader wasn't proclaimed Python's BDFL ;-)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

When the documented behavior is both nonsensical and the cause of hard
to debug bugs that is a pretty compelling use case to me, unless you actively
enjoy being user hostile.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/c79bfc98/attachment.sig>

From tim.peters at gmail.com  Thu Mar  6 02:53:02 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 19:53:02 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
Message-ID: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>

[Bruce Leban]
> And the documentation is not exactly clear for understanding when a time
> object is falsey. In particular, the "converting it to minutes" part seems
> irrelevant

That's there because `utcoffset()` is defined to return minutes.  A
rule is needed to specify how the naive time object and `utcoffset()`
interact to define the result.

> as a time of 1 microsecond from midnight is truthy and timedeltas
> are not numbers and are not in minutes

timedeltas have nothing to do with this.  The docs could be expanded
to specify precisely how the naive time object is converted to
minutes, but I think that's so obvious it would just be tedious to do
so.

> Perhaps a reasonable improvement would be to change this to:
>
> a time object is considered to be true unless it represents exactly midnight
> local time, that is, it's false if subtracting the utcoffset() from the time
> produces an all-zero result (or it's all-zero to start with if the utcoffset
> is None).

You still need a rule to define _how_ "subtracting the utcoffset()
from the time" is intended to work, since arithmetic on naive time
objects is not defined.  Add "convert it to minutes" ;-)

From tim.peters at gmail.com  Thu Mar  6 03:08:56 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 20:08:56 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5317A696.8040605@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
Message-ID: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>

[M.-A. Lemburg]
> ...
> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return
> 0.0 seconds some day. That's what mxDateTime does for time values.

Now you're getting close :-)

There was an endless list of features various people said they wanted
in a datetime module.  At the time, we picked a large subset we could
do in time, and released it.  There was every expectation that more
features may be added later.

Among them were various kinds of conversions and various kinds of
arithmetic enhancements.  For example, modular arithmetic on time
objects was a possibility.  In that case, time(0, 0, 0) would become
"a zero value" just as much as 0 or 0.0 or 0j or 0x00 are zero values.
 But if it weren't _treated_ as "a zero value" from the start,
backward compatibility would prevent it from being treated as a zero
value for the first time later.

So, what the heck - like all other zero/empty objects, time(0, 0, 0)
was fiddled to evaluate to False in a Boolean context, AND (something
nobody seems to have noticed yet) plain time() was fiddled to return
it:

>>> bool(int()) is bool(complex()) is bool(float()) is bool(datetime.time())
True

People can say that "midnight" isn't a compelling "sentinel value",
and I agree about the "compelling" part, but in all implementation
respects it does _act_ as "a sentinel value".  If people don't want to
use it as one, fine, that's their choice.

I've used it as one, and it works fine for that purpose.

As things turned out, nobody seemed to care much about modular (are
any other kind of) arithmetic on time objects after all.
Nevertheless, we went about 9 years before anyone complained about the
code added to support the _possibility_ for doing that cleanly.

The "seconds from midnight" view was also in play, with a view toward
clean conversions to float-ish types.

From alexander.belopolsky at gmail.com  Thu Mar  6 03:12:17 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 21:12:17 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
Message-ID: <CAP7h-xbCuUKhsuJ4szik3FW3pONK2ew3CLqwqtLbi_dwW=pTfQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 8:53 PM, Tim Peters <tim.peters at gmail.com> wrote:
>
> That's there because `utcoffset()` is defined to return minutes.

More accurately, it returns "a timedelta object representing a whole number
of minutes":

time.utcoffset()

If tzinfo is None, returns None, else returns self.tzinfo.utcoffset(None),
and raises an exception if the latter doesn?t return None or a timedelta
object representing a whole number of minutes with magnitude less than one
day.

http://docs.python.org/3/library/datetime.html#datetime.time.utcoffset
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e9b28b0c/attachment.html>

From steve at pearwood.info  Thu Mar  6 03:14:57 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 13:14:57 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>
Message-ID: <20140306021457.GF28804@ando>

On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote:

> I find the argument that it would only be fixed in 3.something and thus not be really
> fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python
> so by that logic we should simply give up on fixing anything in Python ever?

Not at all. But it does mean that the benefit of fixing it needs to be 
worth the wait. Suppose we said "Sure, we'll fix it, but you have 
to pay for the fix." How much would you be willing to pay to have it 
fixed? 

("Pay" doesn't have to mean money. It can also mean, how much pain or 
inconvenience are you willing to experience, or how much time and effort 
are you willing to out into this.)

If I said, "Oh by the way, although you have to pay right now, you won't 
see any benefit until 2024", now how much are you willing to pay?

Design flaws are more costly to fix than outright bugs, not because they 
take more time and effort to change, but because you're changing 
behaviour that *somebody out there is relying on*. What you consider a 
terrible misfeature is *somebody's* critical feature, and you're 
proposing that we take that away. That's okay, we can do that after a 
suitable deprecation period. (You wouldn't want us to break your code, 
so you ought to extend the same courtesy to other people.)

But even if there isn't anyone relying on this, and we fix it as soon as 
possible (which will be 3.5), your library *still* can't rely on it 
until it drops all support for versions below 3.5. Since you can't rely 
on that feature being present, you have to write your library code as if 
it wasn't present. So there is *absolutely no difference* between your 
library with this (alleged) bug being fixed, or it not being fixed.

The situation for application developers is a bit different. An 
application developer can simply say, I'm going to immediately migrate 
to 3.5 to take advantage of this new feature/bug fix. That's okay too. 

The job of the core developers is to balance all these competing 
interests: the library developers, the application developers, the users 
who don't care one way or another but do care about some other bug 
that's not being fixed because people are arguing about this one, the 
people (actual or hypothetical) who actually like this (mis)feature the 
way it is, people who don't like it but have written code that relies on 
it, and, yes, even the core developers themselves, who are perfectly 
entitled to say that the effort in fixing this is greater than the 
benefit so they're not going to do it.

If you haven't already read Nick's blog posts on these issues, you 
should:

http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html
http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html


For what it's worth, I think the current behaviour is a misfeature, and 
in a perfect world it would be fixed. But the cost of this is so 
miniscule, and the work-around so trivial, that although the fix is 
cheap, the benefit is even lower. I've only spent a few minutes reading 
and responding to this thread. Those few minutes are already worth far 
more than any benefit to me in fixing this. That's *my* personal opinion 
on this, others may differ, so I'm a +0 on deprecating the behaviour in 
3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately 
in 3.5.



-- 
Steven

From tim.peters at gmail.com  Thu Mar  6 03:15:59 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 20:15:59 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
Message-ID: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>

[Donald Stufft]
> When the documented behavior is both nonsensical and the cause of hard
> to debug bugs that is a pretty compelling use case to me, unless you actively
> enjoy being user hostile.

Breaking code that currently works is as hostile as hostile gets.
Where is the evidence that no code relies on the current behavior?
For that matter, where is the evidence that the current behavior is a
significant cause of bugs?  I know I've used "if some_time_object:",
but it did exactly what I expected it to do.

From steve at pearwood.info  Thu Mar  6 03:19:48 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 13:19:48 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
Message-ID: <20140306021948.GG28804@ando>

On Thu, Mar 06, 2014 at 09:16:06AM +1100, Ben Finney wrote:

> With ?datetime.time? there is no equivalent: midnight is not special in
> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point
> on a continuum, and is not a ?not there? value. It should not be Boolean
> false any more than any other time.

It may only be a convention that an instant before midnight is the end 
of the day and midnight the beginning of the next, but it is a 
convention: midnight is the origin (i.e. zero point) of the day. That 
makes it arguably as falsey as 0, 0.0 and 0j.


-- 
Steven

From donald at stufft.io  Thu Mar  6 03:23:37 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 21:23:37 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
Message-ID: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>


On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [Donald Stufft]
>> When the documented behavior is both nonsensical and the cause of hard
>> to debug bugs that is a pretty compelling use case to me, unless you actively
>> enjoy being user hostile.
> 
> Breaking code that currently works is as hostile as hostile gets.
> Where is the evidence that no code relies on the current behavior?
> For that matter, where is the evidence that the current behavior is a
> significant cause of bugs?  I know I've used "if some_time_object:",
> but it did exactly what I expected it to do.

Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s
the case what you expect it to do isn?t particularly relevant as you?re intimately aware
of it?s implementation.

It?s hard to do any sort of search for this, however in an informal poll where I?ve shown
people this code not a single person thought it made sense, and most of them responded
with ?wtf??. I?ve also seen first hand through Django (which is why the person who started
this thread) get caught by this bug and have to spend hours trying to figure out why it?s
behaving that way.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/6b3e8888/attachment.sig>

From donald at stufft.io  Thu Mar  6 03:33:24 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 5 Mar 2014 21:33:24 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306021457.GF28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>
 <20140306021457.GF28804@ando>
Message-ID: <18676193-CEEA-40E2-B996-ECDCDB01A365@stufft.io>


On Mar 5, 2014, at 9:14 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote:
> 
>> I find the argument that it would only be fixed in 3.something and thus not be really
>> fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python
>> so by that logic we should simply give up on fixing anything in Python ever?
> 
> Not at all. But it does mean that the benefit of fixing it needs to be 
> worth the wait. Suppose we said "Sure, we'll fix it, but you have 
> to pay for the fix." How much would you be willing to pay to have it 
> fixed? 
> 
> ("Pay" doesn't have to mean money. It can also mean, how much pain or 
> inconvenience are you willing to experience, or how much time and effort 
> are you willing to out into this.)
> 
> If I said, "Oh by the way, although you have to pay right now, you won't 
> see any benefit until 2024", now how much are you willing to pay?

Well I?ve offered to not only write the patch (assuming that someone else
doesn?t do it before me since my time is limited), but also shepherd it
through review and commit it myself. So I?m willing to pay with my free time
which is a very limited resource :)

In general I agree that everything has to go through a cost/benefit analysis,
and in this case the cost and benefit are both fairly low in the grand scheme
of things but the benefit does outweigh the cost as far as I?m concerned
because the workaround requires you to know that the workaround is
even required at all.

> 
> Design flaws are more costly to fix than outright bugs, not because they 
> take more time and effort to change, but because you're changing 
> behaviour that *somebody out there is relying on*. What you consider a 
> terrible misfeature is *somebody's* critical feature, and you're 
> proposing that we take that away. That's okay, we can do that after a 
> suitable deprecation period. (You wouldn't want us to break your code, 
> so you ought to extend the same courtesy to other people.)

It?s not obvious to me that design flaws are more costly to fix because
people rely on even buggy behavior :) I don?t think i?ve advocated for
not sending this through a normal deprecation process at all.

> 
> But even if there isn't anyone relying on this, and we fix it as soon as 
> possible (which will be 3.5), your library *still* can't rely on it 
> until it drops all support for versions below 3.5. Since you can't rely 
> on that feature being present, you have to write your library code as if 
> it wasn't present. So there is *absolutely no difference* between your 
> library with this (alleged) bug being fixed, or it not being fixed.
> 

There is a difference in the future when 3.5 (or whatever) is a reasonable
minimum for a library. People say the same sort of thing about *anything*
that is fixed or added to Python 3.x only that can?t be backported easily.

There is a wait time until library authors can depend on anything new or
changed in Python and there always has been an always will be. All that
means is that a change now won?t be widely useful until some years down
the road. But that?s true for *anything*, asyncio, PEP453, yield from, and
this change too.

> The situation for application developers is a bit different. An 
> application developer can simply say, I'm going to immediately migrate 
> to 3.5 to take advantage of this new feature/bug fix. That's okay too. 
> 
> The job of the core developers is to balance all these competing 
> interests: the library developers, the application developers, the users 
> who don't care one way or another but do care about some other bug 
> that's not being fixed because people are arguing about this one, the 
> people (actual or hypothetical) who actually like this (mis)feature the 
> way it is, people who don't like it but have written code that relies on 
> it, and, yes, even the core developers themselves, who are perfectly 
> entitled to say that the effort in fixing this is greater than the 
> benefit so they're not going to do it.
> 
> If you haven't already read Nick's blog posts on these issues, you 
> should:
> 
> http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html
> http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html

I?ve read them. I have ~opinions~ about them but I?m not going to derail this
thread with them.

> 
> 
> For what it's worth, I think the current behaviour is a misfeature, and 
> in a perfect world it would be fixed. But the cost of this is so 
> miniscule, and the work-around so trivial, that although the fix is 
> cheap, the benefit is even lower. I've only spent a few minutes reading 
> and responding to this thread. Those few minutes are already worth far 
> more than any benefit to me in fixing this. That's *my* personal opinion 
> on this, others may differ, so I'm a +0 on deprecating the behaviour in 
> 3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately 
> in 3.5.
> 
> 

As I said, the work around is trivial, *once you know it is needed*. This change
is reducing the cognitive burden and chance for mishap for users who aren?t aware
of this misfeature. It?s making Python a friendlier language to use.

> 
> -- 
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f08922f3/attachment-0001.sig>

From tim.peters at gmail.com  Thu Mar  6 03:33:53 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 20:33:53 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
Message-ID: <CAExdVNnrye3OMJShuXnVHDgUMyrGmiU26zFCXwQ4DjZ82YggVw@mail.gmail.com>

> Forgive me if I'm wrong, but aren't you the author of the date time module?

Yes.

> If that's the case what you expect it to do isn't particularly relevant as you're
> intimately aware of it's implementation.

Drat!  Checkmated again!  I get tired of putting smileys after everything ;-)

> It's hard to do any sort of search for this, however in an informal poll where I've shown
> people this code

Precisely which code did you show them?  What were the backgrounds of
these people?  Etc etc etc.

> not a single person thought it made sense, and most of them responded
> with "wtf?"

Which, via amazing coincidence or foresight, is exactly what I bet you
were expecting ;-)


>. I've also seen first hand through Django (which is why the person who started
> this thread) get caught by this bug and have to spend hours trying to figure out why it's
> behaving that way.

Too sketchy to say much about.  If this is a variation of testing "if
object:" as a shorthand for testing "if object is None:", little
sympathy from me.

In any case, this took more of my time so far than any possible
outcome would be worth to me, so I'll just close with -0 on changing
it (would rather not bother, but won't complain if it is changed).

From harrismh777 at gmail.com  Thu Mar  6 03:21:37 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Wed, 5 Mar 2014 18:21:37 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
Message-ID: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>


>
>
> Then, and function that requires PI
> of half_PI can check first to see if it is cached  as __gpi__  and if so 
> pull it back, or if not calculate
> it.   If the context does not change than I half some whopping value of PI 
> for the duration of the 
> context (but, and this is the big but, if I don't need it --- then it 
> never gets bound).
>
>
um, sorry, I wixed my mords...   please forgive,  

 "Then, and only then, a function that requires PI or half_PI  
will first check to see if  __gpi__  has been cached and if so pull it 
back, or if not calculate it (once).  If the context
does not change then the function can take half of some whopping value of 
PI, for the duration of the context, and
use that,  but if not needed for the context will never bind it !


marcus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/004e934b/attachment.html>

From steve at pearwood.info  Thu Mar  6 04:00:02 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 14:00:02 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
Message-ID: <20140306030002.GH28804@ando>

On Wed, Mar 05, 2014 at 09:23:37PM -0500, Donald Stufft wrote:

> It?s hard to do any sort of search for this, however in an informal poll where I?ve shown
> people this code not a single person thought it made sense, and most of them responded
> with ?wtf??.

Well I don't know who these people are, what their background is, or 
exactly how you phrased the question. But in my experience, most 
programmers have extremely strongly held opinions about the sensibility 
of certain features that have little or nothing with any rational 
analysis of the pros and cons and far more likely to be "that's 
different from the first language I learned, therefore it's rubbish".

Having read Tim's explanation for why time() is falsey, I no longer 
think it's a misfeature, and am now -0.5 on changing it at all.


> I?ve also seen first hand through Django (which is why the person who 
> started this thread) get caught by this bug and have to spend hours 
> trying to figure out why it?s behaving that way.

Again, I don't know the background here, and we all know that sometimes 
tracking down bugs can be difficult, but if somebody is spending "hours" 
trying to debug why "if time_value" sometimes does the wrong thing, 
*without* bothering to read the documentation for time objects to find 
out how they're supposed to behave, then I don't have a lot of sympathy.

Also, if you write "if spam" as a short-cut for "if spam is not None", 
then again any bugs you experience are self-inflicted.


-- 
Steven

From guido at python.org  Thu Mar  6 04:47:24 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Mar 2014 19:47:24 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
Message-ID: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>

I wonder if we need yet another list, python-speculative-ideas? Or
python-waxing-philosophically? I've got a feeling were straying quite far
from the topic of the design of even Python 4, let alone what could
possibly happen in Python 3.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5c850ebf/attachment.html>

From sturla.molden at gmail.com  Thu Mar  6 04:54:24 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 06 Mar 2014 04:54:24 +0100
Subject: [Python-ideas] Add "goto fail" to Python?
Message-ID: <lf8rh3$suv$1@ger.gmane.org>

"goto fail" is a well-known error handling mechanism in open source 
software, widely reputed for its robusteness:

http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c

https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c

I believe Python needs to add support for this superior paradigm.

It would involve a new keyword "fail" and some means of goto'ing to it. 
I suggest "raise to fail":

if (some_error):
    raise to fail

fail:
    <error handling code>

Unless there are many objections, this fantastic idea might be submitted 
in a (short) PEP somewhere around the beginning of next month.

There is some obvious overlap with the rejected "goto PEP" (PEP 3163) 
and the Python 2.3 goto module. However, the superiority of goto fail as 
error generation and error handling paradigm has since then been 
thoroughly proven.

http://legacy.python.org/dev/peps/pep-3136/
http://entrian.com/goto/download.html


Regards,
Sturla Molden








From bruce at leapyear.org  Thu Mar  6 05:00:35 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Wed, 5 Mar 2014 20:00:35 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
Message-ID: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>

On Wed, Mar 5, 2014 at 5:53 PM, Tim Peters <tim.peters at gmail.com> wrote:

> That's there because `utcoffset()` is defined to return minutes.
>

It returns either minutes *or a timedelta*:

datetime.utcoffset()?<http://docs.python.org/2/library/datetime.html#datetime.datetime.utcoffset>

If tzinfo <http://docs.python.org/2/library/datetime.html#datetime.tzinfo>
is None, returns None, else returns self.tzinfo.utcoffset(self), and raises
an exception if the latter doesn't return None, or a
timedelta<http://docs.python.org/2/library/datetime.html#datetime.timedelta>
 object representing a whole number of minutes with magnitude less than one
day.


tzinfo.utcoffset(*self*, *dt*)

Return offset of local time from UTC, in minutes east of UTC. If local time
is west of UTC, this should be negative. Note that this is intended to be
the total offset from UTC; for example, if
atzinfo<http://docs.python.org/2/library/datetime.html#datetime.tzinfo>
object
represents both time zone and DST adjustments,
utcoffset()<http://docs.python.org/2/library/datetime.html#datetime.tzinfo.utcoffset>
should
return their sum. If the UTC offset isn't known, return None. Else the
value returned must be a
timedelta<http://docs.python.org/2/library/datetime.html#datetime.timedelta>object
specifying a whole number of minutes in the range -1439 to 1439 inclusive
(1440 = 24*60; the magnitude of the offset must be less than one day).

A
> rule is needed to specify how the naive time object and `utcoffset()`
> interact to define the result.
>

Not sure why you attach 'naive' to time. Time objects carry units with
them, unlike integers. I would say 'naive time value' in reference to an
integer.

Combining a time and an integer can be done by converting the time to
minutes (which is a bit confusing since in most contexts in this class,
minutes values are integers) or it can be done by interpreting the integer
utcoffset result as minutes (as it is documented).



> > Perhaps a reasonable improvement would be to change this to:
> >
> > a time object is considered to be true unless it represents exactly
> midnight
> > local time,
>

The above part is the key part of my suggested improvement. Reading the
current documentation, the fact that it means midnight local time is falsey
is not obvious.


--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5bbe1659/attachment-0001.html>

From abarnert at yahoo.com  Thu Mar  6 05:07:13 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 20:07:13 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
Message-ID: <DF3B61D8-81D3-448F-BD85-7CA21748DDB6@yahoo.com>

On Mar 5, 2014, at 17:34, Tim Peters <tim.peters at gmail.com> wrote:

> "I am altering the deal. Pray I don't alter it any further."  There's
> a reason Darth Vader wasn't proclaimed Python's BDFL ;-)

I always assumed it was because he never contributed even a single patch.

From tim.peters at gmail.com  Thu Mar  6 05:13:59 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 22:13:59 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
Message-ID: <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>

[Tim]
>> That's there because `utcoffset()` is defined to return minutes.

[Bruce Leban]
> It returns either minutes or a timedelta:

I wrote the code and I wrote the docs.  I know what it does ;-)  Going
into tedious detail (as the docs do) is simply irrelevant to the point
here:  utcoffset() returns minutes.  Whether that's _spelled_ as an
integer or as a timedelta, minutes is minutes is minutes.

> ...

>> A rule is needed to specify how the naive time object and `utcoffset()`
>> interact to define the result.

> Not sure why you attach 'naive' to time. Time objects carry units with them,
> unlike integers. I would say 'naive time value' in reference to an integer.

Start with the second paragraph of the datetime docs:

    There are two kinds of date and time objects: "naive" and "aware".

Etc.  I attached "naive" to "time" to make clear that I mean the time
object stripped of its tzinfo attribute.  An aware time object is a
naive time object with a non-None tzinfo attribute.

> ....
> The above part is the key part of my suggested improvement. Reading the
> current documentation, the fact that it means midnight local time is falsey
> is not obvious.

Before this thread, I had never heard anyone express it as "midnight
local time" before.  Now that I have heard it, eh - I don't find it a
helpful way to view it.

From alexander.belopolsky at gmail.com  Thu Mar  6 05:15:31 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 5 Mar 2014 23:15:31 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
Message-ID: <CAP7h-xZB=UYhO5fF3ujVQKFfx7w1WrpiAYtLZB-o2eMwqLPqjw@mail.gmail.com>

On Wed, Mar 5, 2014 at 11:00 PM, Bruce Leban <bruce at leapyear.org> wrote:

> Not sure why you attach 'naive' to time.


'naive' is a defined term in the context of the datetime module.  It means
tzinfo is None.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e728ae3e/attachment.html>

From abarnert at yahoo.com  Thu Mar  6 05:16:52 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 5 Mar 2014 20:16:52 -0800
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
	while adding new functionality
In-Reply-To: <5317CF11.20100@mrabarnett.plus.com>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
 <5317B1D7.4090305@canterbury.ac.nz> <5317CF11.20100@mrabarnett.plus.com>
Message-ID: <5119B096-30C7-4B36-BB59-BE0FEC9B599D@yahoo.com>

On Mar 5, 2014, at 17:27, MRAB <python at mrabarnett.plus.com> wrote:

> In the UK we talk about "Bills" being put before Parliament.

In the US also, except it's Congress, and most of our political discussion isn't about bills before Congress or anything else of political consequence.

> It reminds me of that quotation about the UK and US: ?Two nations
> divided by a common language?. In the UK you can pay a bill using a
> cheque; in the US you can pay the check using bills (but a "check" is
> also a (UK) cheque!).

How many restaurants still accept cheques (or checks)? The only thing they're really used for anymore is paying money to the government... So this bit of confusion will die away. Pants, on the other hand, aren't going away any time soon.

From greg.ewing at canterbury.ac.nz  Thu Mar  6 05:41:11 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 17:41:11 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140305192419.GD28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
Message-ID: <5317FC67.7060600@canterbury.ac.nz>

Steven D'Aprano wrote:
> There's also the argument from consistency: midnight is the zero point 
> of the day, and zero is falsey.

Regarding midnight as a "zero point" is an arbitrary
convention.

The integer zero is not arbitrary, because it's the
additive identity. There is no corresponding additive
identity for times, because you can't add two times.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  6 05:46:46 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 17:46:46 +1300
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
Message-ID: <5317FDB6.5000806@canterbury.ac.nz>

CFK wrote:
> then it shouldn't be too hard 
> for a project to add its own UUID to the annotation dictionary.

I can't see what benefit there is in bringing UIIDs into
the picture.

If you're suggesting that people write the UUIDs directly
into their code as literals, that's *not* going to fly.
It would be appallingly ugly and error-prone.

The only way to make it usable would be to import the
UUIDs from somewhere under human-readable names. But then
there's no need to use UUIDs, any set of unique sentinel
objects will do.

-- 
Greg

From mertz at gnosis.cx  Thu Mar  6 05:59:48 2014
From: mertz at gnosis.cx (David Mertz)
Date: Wed, 5 Mar 2014 20:59:48 -0800
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <lf8rh3$suv$1@ger.gmane.org>
References: <lf8rh3$suv$1@ger.gmane.org>
Message-ID: <CAEbHw4YQmejBovpcbLqRTw8C2t73RbEJtpYkmut4WCf08fWUsQ@mail.gmail.com>

This is *exactly* what Python does right now!

class ToFailError(Exception): pass

try:
    do_some_stuff()
    if (some_error):
        raise ToFailError
    do_other_stuff()
    try:
       thing_that_might_fail()
    except SomeOtherError:
       handle_it()

except ToFailError:
    here_we_are()


On Wed, Mar 5, 2014 at 7:54 PM, Sturla Molden <sturla.molden at gmail.com>wrote:

> "goto fail" is a well-known error handling mechanism in open source
> software, widely reputed for its robusteness:
>
> http://opensource.apple.com/source/Security/Security-
> 55471/libsecurity_ssl/lib/sslKeyExchange.c
>
> https://www.gitorious.org/gnutls/gnutls/source/
> 6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c
>
> I believe Python needs to add support for this superior paradigm.
>
> It would involve a new keyword "fail" and some means of goto'ing to it. I
> suggest "raise to fail":
>
> if (some_error):
>    raise to fail
>
> fail:
>    <error handling code>
>
> Unless there are many objections, this fantastic idea might be submitted
> in a (short) PEP somewhere around the beginning of next month.
>
> There is some obvious overlap with the rejected "goto PEP" (PEP 3163) and
> the Python 2.3 goto module. However, the superiority of goto fail as error
> generation and error handling paradigm has since then been thoroughly
> proven.
>
> http://legacy.python.org/dev/peps/pep-3136/
> http://entrian.com/goto/download.html
>
>
> Regards,
> Sturla Molden
>
>
>
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/6f030e41/attachment-0001.html>

From bruce at leapyear.org  Thu Mar  6 06:16:49 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Wed, 5 Mar 2014 21:16:49 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
Message-ID: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>

On Wed, Mar 5, 2014 at 8:13 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [Tim]
> >> That's there because `utcoffset()` is defined to return minutes.
>
> [Bruce Leban]
> > It returns either minutes or a timedelta:
>
> I wrote the code and I wrote the docs.  I know what it does ;-)  Going
> into tedious detail (as the docs do) is simply irrelevant to the point
> here:  utcoffset() returns minutes.  Whether that's _spelled_ as an
> integer or as a timedelta, minutes is minutes is minutes.
>

Just because you wrote the docs doesn't mean you know what they mean to
other readers. The point of documentation is to explain it to someone who
doesn't know what it does after all.

>
>
> Start with the second paragraph of the datetime docs:
>
>     There are two kinds of date and time objects: "naive" and "aware".
>

Thanks.

Before this thread, I had never heard anyone express it as "midnight
> local time" before.  Now that I have heard it, eh - I don't find it a
> helpful way to view it.


Well, it's not surprising that you don't find it's helpful. In fact, it's
wrong, which to me indicates how confusing this. Which of the following are
falsey?

00:00:00+01:00
01:00:00+01:00
01:00:00-01:00
07:00:00+07:00
23:00:00-01:00


I find it particularly surprising that the bug that the OP described
finding would occur in the Russia and China, but not in the US. Obviously,
not an accident. :-)

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/098d445e/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:18:47 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:18:47 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5317A4BB.1090303@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io>
 <5317A4BB.1090303@egenix.com>
Message-ID: <53180537.1050708@canterbury.ac.nz>

M.-A. Lemburg wrote:
> The reasoning becomes clear if you regard a time value as
> number of seconds since midnight (the time of day is a relative
> measure).

But there's no clear reason to regard that particular
way of representing a time of day as superior to any
other.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:22:07 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:22:07 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5317A696.8040605@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
Message-ID: <531805FF.8070409@canterbury.ac.nz>

M.-A. Lemburg wrote:
> This is true in a date/time continuum, but not true if you regard
> time as "time of day". In the latter interpretation, the day starts
> at midnight,

That's not always the most convenient way of looking at
it. For example, for booking a stay in a hotel, it may
be better to think of "nights" as stretching from
midday to midday.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:30:21 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:30:21 +1300
Subject: [Python-ideas] One more time... lambda function <--- from ***
 signature def.
In-Reply-To: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando>
 <5316544D.7050302@canterbury.ac.nz>
 <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
 <53179C51.4020001@canterbury.ac.nz>
 <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>
Message-ID: <531807ED.3020003@canterbury.ac.nz>

Masklinn wrote:
> On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

>>Also, Haskell has the advantage of knowing that the value
>>won't change
> 
> I do not see why that would be relevant to thunks. The result of the
> thunk may be mutable, so what?

That's not what I mean. The value of the thunk can
depend on other things that change. In general you
can't get away with evaluating it once and caching
the result.

-- 
Greg



From sturla.molden at gmail.com  Thu Mar  6 06:30:01 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 6 Mar 2014 05:30:01 +0000 (UTC)
Subject: [Python-ideas] Add "goto fail" to Python?
References: <lf8rh3$suv$1@ger.gmane.org>
 <CAEbHw4YQmejBovpcbLqRTw8C2t73RbEJtpYkmut4WCf08fWUsQ@mail.gmail.com>
Message-ID: <444107387415775972.062814sturla.molden-gmail.com@news.gmane.org>

David Mertz <mertz at gnosis.cx> wrote:
> This is *exactly* what Python does right now!

I know, it would by syntaxic sugar for try-except.

I am worried that Python's significant indentation might interfere with
some of the strengths of "goto fail", for example placement of code that
could be vital for national security. We might therefore need an exception
to the indentation syntax so that

if (error):
    raise to fail
    raise to fail
    <suite>

is interpreted as

if (error):
    raise to fail
raise to fail
if 1:
    <suite>


Sturla


From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:34:49 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:34:49 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5317B1BE.4090804@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
 <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
 <5317B1BE.4090804@egenix.com>
Message-ID: <531808F9.1020605@canterbury.ac.nz>

M.-A. Lemburg wrote:
> Well, the duration between the start of day and midnight is zero
> and returning to the interpretation of time of day being a measurement
> relative to midnight,

The point of having time objects as an abstraction is
so that you *don't* have to think of them as an offset
from anything. You simply think of them as times.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:38:40 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:38:40 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
 <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
 <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
Message-ID: <531809E0.6000905@canterbury.ac.nz>

Alexander Belopolsky wrote:
> For me, it is the fraction of the day that passed or distance from 
> midnight and in any case fundamentally some kind of number expressed in 
> a Babylonian base-60 notation.

Yes, and that number is conventionally written as
"12:00", which is *obviously* equal to zero... er,
what?

-- 
Greg

From guido at python.org  Thu Mar  6 06:42:22 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Mar 2014 21:42:22 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <531809E0.6000905@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
 <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
 <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
 <531809E0.6000905@canterbury.ac.nz>
Message-ID: <CAP7+vJKoRJaSZF_bE5sSjiujQWdniffrTNnfRwGSY861LKPbdQ@mail.gmail.com>

The 12:00 notation is US-centric. Or, perhaps, common in Anglophone
countries. The rest of the world does not use the AM/PM conventions and
calls midnight 00:00.


On Wed, Mar 5, 2014 at 9:38 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Alexander Belopolsky wrote:
>
>> For me, it is the fraction of the day that passed or distance from
>> midnight and in any case fundamentally some kind of number expressed in a
>> Babylonian base-60 notation.
>>
>
> Yes, and that number is conventionally written as
> "12:00", which is *obviously* equal to zero... er,
> what?
>
> --
> Greg
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5fb9138b/attachment.html>

From harrismh777 at gmail.com  Thu Mar  6 05:05:37 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Wed, 5 Mar 2014 20:05:37 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
Message-ID: <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>



On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote:
>
> I wonder if we need yet another list, python-speculative-ideas? Or 
> python-waxing-philosophically? 
>

hi Guido,  no probably not.  I just got to thinking again about *number *again 
as 
I was thinking about decimal floating point as default. Its the 
mathematician in me; go
for the general case, and that would be a type-less unified *number *system.

For the near case design issues the following realist ideas might be 
considered in order:

1) decimal literal like  1.23d

2) default decimal floating point with a binary type   1.23b

3) type-less unified *number  *yes, probably way out there into the future


kind regards,
marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bc00238c/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 06:50:10 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 18:50:10 +1300
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <5317CF11.20100@mrabarnett.plus.com>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
 <5317B1D7.4090305@canterbury.ac.nz> <5317CF11.20100@mrabarnett.plus.com>
Message-ID: <53180C92.7070904@canterbury.ac.nz>

MRAB wrote:

> In the UK we talk about "Bills" being put before Parliament.

In NZ too, but once it's passed, it's the law --
it's not up to individual interpretation!

-- 
Greg

From tim.peters at gmail.com  Thu Mar  6 06:53:19 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 5 Mar 2014 23:53:19 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
Message-ID: <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>

[Bruce Leban <bruce at leapyear.org>]
> ...
> Just because you wrote the docs doesn't mean you know what they mean to
> other readers. The point of documentation is to explain it to someone who
> doesn't know what it does after all.

Of course.  What's your point here?

>> ...
>> Before this thread, I had never heard anyone express it as "midnight
>> local time" before.  Now that I have heard it, eh - I don't find it a
>> helpful way to view it.

> Well, it's not surprising that you don't find it's helpful. In fact, it's
> wrong, which to me indicates how confusing this.

Bruce, you're the one who made up the "midnight local time" business,
which has just confused you.  Try again to read what the docs actually
say, without assuming it must mean something else?

> Which of the following are falsey?

It would help a lot if you explained which ones surprised _you_, and
how you read the docs in such a way that what they actually say caused
your confusion.

> 00:00:00+01:00

Convert the naive time to minutes: 0.
Get the UTC offset in minutes  (these offsets are always returned
directly as minutes in the code, but there's an "extra" calculation
step needed here because you're starting from a string representation
that also uses units of hours):  1*60 + 0 = 60.
Subtract:  0-60 = -60
Is that zero?
No, so this one is true.

> 01:00:00+01:00

60 - 60 = 0.  False.

> 01:00:00-01:00

60 - -60 = 120.  True.

> 07:00:00+07:00

420 - 420 = 0.  False.

> 23:00:00-01:00

23*60 - -60 = 24*60.  True.


> I find it particularly surprising that the bug that the OP described finding
> would occur in the Russia and China, but not in the US. Obviously, not an
> accident. :-)

Indeed, it is the key to liberating both Ukraine and Tibet ;-)

From guido at python.org  Thu Mar  6 06:56:02 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Mar 2014 21:56:02 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
Message-ID: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>

Do you actually have a degree in math, or do you just remember your high
school algebra? The numbers in math usually are quite strictly typed: whole
theories only apply to integers or even positive integers, other things to
all reals but not to complex numbers. (And what about quaternions? Or
various infinities? :-)


On Wed, Mar 5, 2014 at 8:05 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

>
>
> On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote:
>>
>> I wonder if we need yet another list, python-speculative-ideas? Or
>> python-waxing-philosophically?
>>
>
> hi Guido,  no probably not.  I just got to thinking again about *number *again
> as
> I was thinking about decimal floating point as default. Its the
> mathematician in me; go
> for the general case, and that would be a type-less unified *number *
> system.
>
> For the near case design issues the following realist ideas might be
> considered in order:
>
> 1) decimal literal like  1.23d
>
> 2) default decimal floating point with a binary type   1.23b
>
> 3) type-less unified *number  *yes, probably way out there into the future
>
>
> kind regards,
> marcus
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/8cc3d198/attachment-0001.html>

From alexander.belopolsky at gmail.com  Thu Mar  6 07:47:12 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 01:47:12 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
Message-ID: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>

On Thu, Mar 6, 2014 at 12:16 AM, Bruce Leban <bruce at leapyear.org> wrote:

> Well, it's not surprising that you don't find it's helpful. In fact, it's
> wrong, which to me indicates how confusing this. Which of the following are
> falsey?
>
> 00:00:00+01:00
> 01:00:00+01:00
> 01:00:00-01:00
> 07:00:00+07:00
> 23:00:00-01:00
>
>
> I find it particularly surprising that the bug that the OP described
> finding would occur in the Russia and China, but not in the US. Obviously,
> not an accident. :-)
>

"Aware" time objects are problematic even before you consider their bool
behavior.  Fortunately, you don't encounter them too often because
dt.time() returns naive time even if dt is an aware datetime instance.

Again, the use case that I find compelling is to test dt.time() for
falsehood to detect datetime objects that fall on midnight.

I don't understand why people are denying that midnights are special
points.  All real numbers are equal, but integers are more equal than
others.  When you plot say hourly timeseries it is not uncommon to mark
midnight ticks on the time axis with dt.date() and other ticks with
dt.hour.  Whenever you need to do something special when you start a new
date in a simulation of a physical process it is convenient to have a
simple test for midnight.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/deeb546e/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 07:51:06 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 19:51:06 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
Message-ID: <53181ADA.2030401@canterbury.ac.nz>

Tim Peters wrote:
> For example, modular arithmetic on time
> objects was a possibility.  In that case, time(0, 0, 0) would become
> "a zero value"

That only follows if you interpret "modular arithmetic on
time objects" as meaning that you can directly add one time
object to another. But that would be confusing times with
timedeltas. The concept of a zero *timedelta* makes sense,
but not a "zero time".

> People can say that "midnight" isn't a compelling "sentinel value",
> and I agree about the "compelling" part, but in all implementation
> respects it does _act_ as "a sentinel value".

If someone is using midnight to represent an unspecified
time, I'd say they have a bug. An appointment scheduled at
midnight is not the same as an appointment that's not
scheduled at all.

-- 
Greg

From bruce at leapyear.org  Thu Mar  6 08:02:34 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Wed, 5 Mar 2014 23:02:34 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
Message-ID: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>

On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky <
alexander.belopolsky at gmail.com> wrote:

> it is convenient to have a simple test for midnight.
>

Except this isn't it. That's only works for naive times. For aware times:

00:00:00+00:00 = midnight England = midnight UTC => False
00:00:00+01:00 = midnight France = 2300 UTC      => True
01:00:00+01:00 = 1 am France = midnight UTC      => False
19:00:00-05:00 = 1 am Boston = midnight UTC      => True

Code that relies that bool(time) is False for midnight is broken IMHO.

--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/9f6da53f/attachment.html>

From alexander.belopolsky at gmail.com  Thu Mar  6 08:20:30 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 02:20:30 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
Message-ID: <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>

On Thu, Mar 6, 2014 at 2:02 AM, Bruce Leban <bruce at leapyear.org> wrote:

>
> On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky <
> alexander.belopolsky at gmail.com> wrote:
>
>> it is convenient to have a simple test for midnight.
>>
>
> Except this isn't it. That's only works for naive times. For aware times:
>
> 00:00:00+00:00 = midnight England = midnight UTC => False
> 00:00:00+01:00 = midnight France = 2300 UTC      => True
> 01:00:00+01:00 = 1 am France = midnight UTC      => False
> 19:00:00-05:00 = 1 am Boston = midnight UTC      => True
>
> Code that relies that bool(time) is False for midnight is broken IMHO.
>
>
You keep fighting your own straw-man.  My use case is not using an
arbitrary time
object that someone decided to imbue with a timezone info.  What I want is
given
a *datetime* instance dt to check whether it falls on midnight in whatever
timezone
dt.tzinfo represent or naively if dt.tzinfo is None.

I claim that "if dt.time()" is a perfectly good shorthand for "if dt.time()
!= datetime.time(0)".

I am not trying to argue that "if not dt.timetz()" test is equally useful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5c6e8f81/attachment.html>

From stephen at xemacs.org  Thu Mar  6 08:29:20 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 16:29:20 +0900
Subject: [Python-ideas] One more time... lambda function <--- from
	***	signature def.
In-Reply-To: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
 <53179C51.4020001@canterbury.ac.nz>
 <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>
Message-ID: <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp>

Masklinn writes:
 > On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
 > > Masklinn wrote:
 > >> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

 > > That comes at the expense of making *everything* a thunk
 > > until its value is needed [in Haskell].
 > 
 > Of course not, Haskell allows explicitly forcing a thunk and the
 > compiler/runtime pair can use strictness analysis and not create thunks
 > in the first place.

That doesn't seem fair to me.  "Explicitly forcing" by definition
means the programmer has decided the value is needed, and "strictness
analysis" is an optimization, not part of the language definition.  Am
I missing something?

 > Consider that a thunk is a deferral of an expression's evaluation,
 > why would said expression's evaluation happen more than once? The
 > only thing which changes is *when* the actual evaluation happens.

Are thunks guaranteed not to leak out of the scope of the
containing expression, so assignment is impossible?

If they can leak, the thunk would be an object, and in Python
"assigning" it to a "variable" actually just creates a reference.
Evaluation could memoize the thunk's value ("evaluation only happens
once") or reevaluate the thunk each time its value is requested.
Either seems potentially surprising to me.

Memoization makes sense if you think of the thunk as the closure of a
computation that conceptually takes place at definition time.  I'm not
sure if in the use cases for thunks it really makes sense to "close"
the thunk at evaluation time, but it seems like a plausible
interpretation to me.

From donald at stufft.io  Thu Mar  6 08:30:39 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 02:30:39 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
 <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
Message-ID: <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>


On Mar 6, 2014, at 2:20 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> 
> 
> 
> On Thu, Mar 6, 2014 at 2:02 AM, Bruce Leban <bruce at leapyear.org> wrote:
> 
> On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> it is convenient to have a simple test for midnight.
> 
> Except this isn't it. That's only works for naive times. For aware times:
> 
> 00:00:00+00:00 = midnight England = midnight UTC => False
> 00:00:00+01:00 = midnight France = 2300 UTC      => True
> 01:00:00+01:00 = 1 am France = midnight UTC      => False
> 19:00:00-05:00 = 1 am Boston = midnight UTC      => True
> 
> Code that relies that bool(time) is False for midnight is broken IMHO.
> 
> 
> You keep fighting your own straw-man.  My use case is not using an arbitrary time
> object that someone decided to imbue with a timezone info.  What I want is given
> a *datetime* instance dt to check whether it falls on midnight in whatever timezone
> dt.tzinfo represent or naively if dt.tzinfo is None.

So check for it falling on midnight. It?s not reasonable to expect that midnight will evaluate
to false, especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!).
That makes the behavior even more surprising and more footgun-ey.

> 
> I claim that "if dt.time()" is a perfectly good shorthand for "if dt.time() != datetime.time(0)?.

And I claim that you?re reaching to justify stupid behavior.

> 
> I am not trying to argue that "if not dt.timetz()" test is equally useful

And yet you can?t control what people do with it. Consistency is a virtue and how objects
act with a tzinfo and how they act without a tzinfo should be as close as possible. 

> .
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cb0038f5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cb0038f5/attachment.sig>

From shai at platonix.com  Thu Mar  6 08:25:26 2014
From: shai at platonix.com (Shai Berger)
Date: Thu, 06 Mar 2014 09:25:26 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306030002.GH28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <20140306030002.GH28804@ando>
Message-ID: <1978788.hqjmn1vSZ0@deblack>

On Thursday 06 March 2014 14:00:02 Steven D'Aprano wrote:
> tracking down bugs can be difficult, but if somebody is spending "hours"
> trying to debug why "if time_value" sometimes does the wrong thing,
> *without* bothering to read the documentation for time objects to find
> out how they're supposed to behave, then I don't have a lot of sympathy.
> 
This argument keeps repeating. It is a gross misunderstanding of the problem.

The hours are not spent trying to debug why an if went the wrong way. The 
hours are spent trying to understand why some feature of a program doesn't 
work as expected -- *sometimes*. It can easily take hours to try and reproduce 
the wrong behavior, in vain, because it only shows up on unique data; and the 
wrong behavior will often not show itself as "an if went the wrong way", but, 
for example, "sometimes a number here is wrong" -- which happens when the 
calculation of the number goes over a collection of objects and does complex 
things including decisions based on time.

Of course, once you get to the "if" it's relatively easy (surprising behavior 
is still surprising, but less mystifying). It's getting to it that's the 
issue.

Thanks,
	Shai.

From greg.ewing at canterbury.ac.nz  Thu Mar  6 08:33:43 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 06 Mar 2014 20:33:43 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
Message-ID: <531824D7.8020302@canterbury.ac.nz>

Bruce Leban wrote:
> Except this isn't it. That's only works for naive times. For aware times:
> 
> 00:00:00+00:00 = midnight England = midnight UTC => False
> 00:00:00+01:00 = midnight France = 2300 UTC      => True
> 01:00:00+01:00 = 1 am France = midnight UTC      => False
> 19:00:00-05:00 = 1 am Boston = midnight UTC      => True

That's... I don't know what it is.

Is there a rationale for this? I'm totally failing
to see it.

-- 
Greg

From alexander.belopolsky at gmail.com  Thu Mar  6 08:40:24 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 02:40:24 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
 <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
 <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>
Message-ID: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>

On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote:

> It?s not reasonable to expect that midnight will evaluate
> to false, ..
>

Only in the world where it is not reasonable to expect programmers to read
library documentation.  In my world it is reasonable to expect that the
behavior that was documented in 10 major versions and for 10 years can be
relied on.



> especially when it doesn?t if you happen to have a tzinfo on the time
> (sometimes!).
>

As long as tzinfo specifies a fixed offset, there is no problem with the
current definition.  If you are unfortunate enough to live in a place with
semi-annual DST adjustment, aware time objects are problematic for reasons
that have nothing to do with the discussion at hand.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b7d44ebd/attachment.html>

From donald at stufft.io  Thu Mar  6 08:43:18 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 02:43:18 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
 <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
 <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>
 <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>
Message-ID: <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io>


On Mar 6, 2014, at 2:40 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> 
> On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote:
> It?s not reasonable to expect that midnight will evaluate
> to false, ..
> 
> Only in the world where it is not reasonable to expect programmers to read library documentation.  In my world it is reasonable to expect that the behavior that was documented in 10 major versions and for 10 years can be relied on. 
> 
>  
> especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!).
> 
> As long as tzinfo specifies a fixed offset, there is no problem with the current definition.  If you are unfortunate enough to live in a place with semi-annual DST adjustment, aware time objects are problematic for reasons that have nothing to do with the discussion at hand.
> 

Nope! It essentially in an aware time would be ?false if whatever time is at the same time as UTC midnight, but only if your UTC offset is positive?. If your UTC offset is negative then you end up with 1440 instead of 0.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b0044cae/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b0044cae/attachment-0001.sig>

From markus at unterwaditzer.net  Thu Mar  6 08:38:41 2014
From: markus at unterwaditzer.net (Markus Unterwaditzer)
Date: Thu, 06 Mar 2014 08:38:41 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140306021457.GF28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>
 <20140306021457.GF28804@ando>
Message-ID: <556cc654-167b-4d06-8c30-77c414e54069@email.android.com>



On 6 March 2014 03:14:57 CET, Steven D'Aprano <steve at pearwood.info> wrote:
>On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote:
>
>> I find the argument that it would only be fixed in 3.something and
>thus not be really
>> fixed for X years disingenuous at best. That fact is true for *ALL*
>bug fixes in Python
>> so by that logic we should simply give up on fixing anything in
>Python ever?
>
>Not at all. But it does mean that the benefit of fixing it needs to be 
>worth the wait. Suppose we said "Sure, we'll fix it, but you have 
>to pay for the fix." How much would you be willing to pay to have it 
>fixed? 
>
>("Pay" doesn't have to mean money. It can also mean, how much pain or 
>inconvenience are you willing to experience, or how much time and
>effort 
>are you willing to out into this.)
>
>If I said, "Oh by the way, although you have to pay right now, you
>won't 
>see any benefit until 2024", now how much are you willing to pay?
>
>Design flaws are more costly to fix than outright bugs, not because
>they 
>take more time and effort to change, but because you're changing 
>behaviour that *somebody out there is relying on*.

Just because "somebody" is relying on a wart in the language it doesn't mean that this person is knowing what they are doing, is representative of the rest of Python users or that the wart shouldn't be removed. In my opinion it even means that the person is a bad programmer in a way, because even though he is relying documented behavior, that behavior is relatively unknown and using it intentionally certainly doesn't help readability. Actually i think that this programmer put extra effort into making their own code less readable. Partially, this point has already been made many times in this thread, but apparently this doesn't suffice to make the opposing side realize or respond to it.

What you consider a 
>terrible misfeature is *somebody's* critical feature, and you're 
>proposing that we take that away. That's okay, we can do that after a 
>suitable deprecation period. (You wouldn't want us to break your code, 
>so you ought to extend the same courtesy to other people.)
>
>But even if there isn't anyone relying on this, and we fix it as soon
>as 
>possible (which will be 3.5), your library *still* can't rely on it 
>until it drops all support for versions below 3.5.

Arguably many libraries already do rely on this currently nonexistent feature.

>Since you can't rely
>
>on that feature being present, you have to write your library code as
>if 
>it wasn't present. So there is *absolutely no difference* between your 
>library with this (alleged) bug being fixed, or it not being fixed.
>
>The situation for application developers is a bit different. An 
>application developer can simply say, I'm going to immediately migrate 
>to 3.5 to take advantage of this new feature/bug fix. That's okay too. 
>
>The job of the core developers is to balance all these competing 
>interests: the library developers, the application developers, the
>users 
>who don't care one way or another but do care about some other bug 
>that's not being fixed because people are arguing about this one, the 
>people (actual or hypothetical) who actually like this (mis)feature the
>
>way it is, people who don't like it but have written code that relies
>on 
>it, and, yes, even the core developers themselves, who are perfectly 
>entitled to say that the effort in fixing this is greater than the 
>benefit so they're not going to do it.
>
>If you haven't already read Nick's blog posts on these issues, you 
>should:
>
>http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html
>http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html
>
>
>For what it's worth, I think the current behaviour is a misfeature, and
>
>in a perfect world it would be fixed. But the cost of this is so 
>miniscule, and the work-around so trivial, that although the fix is 
>cheap, the benefit is even lower. I've only spent a few minutes reading
>
>and responding to this thread. Those few minutes are already worth far 
>more than any benefit to me in fixing this. That's *my* personal
>opinion 
>on this, others may differ, so I'm a +0 on deprecating the behaviour in
>
>3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately 
>in 3.5.


From donald at stufft.io  Thu Mar  6 08:49:11 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 02:49:11 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
 <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
 <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>
 <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>
 <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io>
Message-ID: <CCD4088B-78D5-4C4F-968F-79288C43D4AA@stufft.io>


On Mar 6, 2014, at 2:43 AM, Donald Stufft <donald at stufft.io> wrote:

> 
> On Mar 6, 2014, at 2:40 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> 
>> 
>> On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote:
>> It?s not reasonable to expect that midnight will evaluate
>> to false, ..
>> 
>> Only in the world where it is not reasonable to expect programmers to read library documentation.  In my world it is reasonable to expect that the behavior that was documented in 10 major versions and for 10 years can be relied on. 
>> 
>>  
>> especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!).
>> 
>> As long as tzinfo specifies a fixed offset, there is no problem with the current definition.  If you are unfortunate enough to live in a place with semi-annual DST adjustment, aware time objects are problematic for reasons that have nothing to do with the discussion at hand.
>> 
> 
> Nope! It essentially in an aware time would be ?false if whatever time is at the same time as UTC midnight, but only if your UTC offset is positive?. If your UTC offset is negative then you end up with 1440 instead of 0.
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Actually now that I think about it more, in aware times it?s impossible for it to time objects to ever be false if the tzinfo is negative, because that would require the time to negative. Take -5 for instance:

x - (-5 * 60) = 0
x - (-300) = 0
x + 300 = 0
x = -300

So in order for a time with UTC offset of -5 to ever be False, it would have to be -05:00:00 there.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f17eabc5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f17eabc5/attachment.sig>

From markus at unterwaditzer.net  Thu Mar  6 08:53:46 2014
From: markus at unterwaditzer.net (Markus Unterwaditzer)
Date: Thu, 06 Mar 2014 08:53:46 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
 <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com>
 <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com>
 <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io>
 <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com>
Message-ID: <5392dc1c-26e8-4963-a47e-d5d624fea853@email.android.com>



On 6 March 2014 08:40:24 CET, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote:
>
>> It?s not reasonable to expect that midnight will evaluate
>> to false, ..
>>
>
>Only in the world where it is not reasonable to expect programmers to
>read
>library documentation.

I disagree. Consider this code:

class MyType:
    def __bool__(self):
        return bool(random.random())

Even though the behavior in boolean context is documented, it doesn't have to make any sense or be reasonable.

>In my world it is reasonable to expect that the
>behavior that was documented in 10 major versions and for 10 years can
>be
>relied on.
>

It *is* reasonable to expect to be able to rely on such features, yet IMO it is not reasonable to actually rely on that feature, for reasons of readability already mentioned by me in a reply to Steven D'Aprano. Considering that (from my subjective view) many people new to Python expect the kind of behavior proposed by the OP, i think it might be time for a change. Maybe an actual survey is the only way to find out.

>
>
>> especially when it doesn?t if you happen to have a tzinfo on the time
>> (sometimes!).
>>
>
>As long as tzinfo specifies a fixed offset, there is no problem with
>the
>current definition.  If you are unfortunate enough to live in a place
>with
>semi-annual DST adjustment, aware time objects are problematic for
>reasons
>that have nothing to do with the discussion at hand.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/


From harrismh777 at gmail.com  Thu Mar  6 09:14:11 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 00:14:11 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
Message-ID: <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>



On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote:
>
> Do you actually have a degree in math, or do you just remember your high 
> school algebra? 
>

  hi Guido,  ouch.   Its a story, glad you asked.  My first trip to college 
1974-1977  
(UMKC) was to study EE;  minor in mathematics.  I completed my math course 
work 
(Calc 1-5, Diff Eq, Linear Algebra, Theory of Stats and Prob, &c) ... all 
 (A).  Not 
that it matters, because in 1977 IBM made me an offer in their CE 
department at 
Kansas City... so I joined IBM for the next 25 years and did not complete 
my EE 
degree... but, I did complete my mathematics training.  Of course, prior to 
UMKC
 I attended a high school that offered calculus, math analysis, trig... and 
intro to 
linear algebra. So by the time I joined IBM I got the math twice.  So, 
yeah, I know
what I'm doing. I am an amateur mathematician today, computer scientist, 
and 
computer hobbyist. While at IBM I was a staff software engineer, Tampa, 
Chicago,
Atlanta, and at the lab at Rochester MN (where I left IBM in 2002; yet 
I still dwell there). Education is a life long commitment and I continue to 
study math, comp sci, music
and philosophy. I just completed my course work for the MDiv degree at 
Bethel Seminary.

Back in the day, I added the scientific and transcendental math functions 
to the Rexx
library for the internal VM370 systems, because Rexx (like python) also had 
no decimal 
floating point math package. So, yeah, its one of the things I know how to 
do, and its one
of those things that most people never think about;  but I've noticed that 
they appreciate
having it once the work is done.  My pdeclib package on PyPI is in infancy 
stage, probably 
has bugs (yet nobody has complained yet) and pdeclib will have to mature 
there for some time.
But that has really nothing to do with whether we continue to use IEEE 754 
1985 floats & doubles
nor whether we discuss default decimal floating point arithmetic nor 
whether we discuss a unified
*python number *sytem (sometime in the distant future) that would make | 
allow common average 
ordinary people to leverage mathematics in computer science without having 
to understand the 
underlying mechanics of implementation including but not limited to types. 

We might agree to stick with the discussion, if you're are willing, and 
stay away from *ad hominem* 
attacks, nor credential bashing. A person either knows what they are doing, 
or they don't. And if
they are willing to contribute, why knock it, or them?
 

> The numbers in math usually are quite strictly typed: whole theories only 
> apply to integers or even positive integers, other things to all reals but 
> not to complex numbers.
>

Everyone keeps making the points noted above!  Yes, I know... concur~
    Guido, all data types in most computer high level languages
are all very strictly statically bound also. So what?  You know this better
than anyone, because you have taken so much abuse from trolls about it over 
the
years. We all know that the best way to handle name binding is dynamically. 
And that
was a paradigm shift, was it not?  Go take a look at Rexx. It has NO types. 
None.
Not at the surface, anyway. Everything to the user of the language is a 
string. This is
even true for most of "object" Rexx too. *Numbers *are just strings of 
characters that
are parsed and interpreted by Rexx as valid *Rexx Numbers*. It works. There 
is no
real point in arguing that its a bad idea, because it served the Rexx 
community for
many years... even now, Rexx is not dead. Yes, under the covers (not magic) 
a complex
number is going to be handled differently than a real number.  Yeah, what 
is your point?
They are handled differently in my TI 89 too... so what,  I change the 
context on the 89
and now I'm doing complex number math (not often, I might add). If I set 
the context for
reals, then now I'm doing real math... it really is not that difficult. 
 Think about this for just
a minute. I have used complex math exactly three times in my life.  I used 
it in high school 
to study the concept in math analysis. I used it in my EE classes... 
electrical engineers 
get a lot out of complex numbers.  And I used it when I was interested in 
the Mendlebrot 
set many years ago when it was cool to plot the famous fractal on early cga 
screens.
When was the last time you used complex numbers?  When was the last time a 
bank, or
a hospital, or Montgomery Wards used complex numbers?  If someone needs 
complex
numbers, Python could change the context "dynamically" and move through the 
problem set.  
Why should the user need to "manually" change the context if  (AI)  could 
change it for them 
on-the-fly?  Just try to get a feel for the question, and stop with trying 
to beat me up on 
my credentials.
 

> (And what about quaternions? Or various infinities? :-)
>

What about quaternions?  If you extend the complex number system you change 
the context. That's not hard...
You would not expect the context for "reals" processing to be the same for 
the context of 
processing three dimensional space with pairs of complex numbers, would 
you?  Python
does complex number processing now. But that is a very specialized category 
of use
case that requires special context and (under the covers) typing relevant 
to complex number
pairs. The context can change "dynamically" to suit the problem set, that's 
all I'm saying.

I am not saying in all of my dreaming here that typing is not important 
(down under). Python is
called an object based language, yes?  Not object oriented, right? But we 
all know that down
under the covers (not magic) python uses Classes and instances of 
classes--objects. We 
don't pretend that what makes object based languages work is magic? do we? 
 Unifying the 
number system on a computer does not have to be grounded in paradigms that 
serviced the 
industry (and academy) for the past 40 years; mostly because memory was 
expensive and
processing was slow.  I suggest that if memory had been cheap, back in the 
day, and processing
had been very fast (as it is today) IEEE 754 1985 floats and doubles would 
never have been 
used. We would have used Decimal floating points right from the get-go. 
Initially, that really is
all I'm asking for on the outset--- lets move to default decimal floating 
point arithmetic for real 
numbers processing.  

In the future I am thinking about systems of languages that interact with 
human speech; understanding
*spoken number* just like they will understand *symbol number.* There is 
really no reason (other
than paradigm) that this would not be possible either. 

Otherwise, Guido, we might all of us just continue to use C and code up our 
statically bound types by
hand using nothing but int, long, float, and double.  

Its time to innovate.

Kind regards, BDfL,

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/4c1b9950/attachment-0001.html>

From stephen at xemacs.org  Thu Mar  6 09:34:06 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 17:34:06 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
Message-ID: <87wqg7pvy9.fsf@uwakimon.sk.tsukuba.ac.jp>

Yann Kaiser writes:

 >     if event.date:
 >         schedule_event()
 > 
 > I cannot fathom one example where it could read as "does the party
 > start at midnight?".  Can you?

No, because it obviously reads as "... WTF??!?"[1]

Of course, that's because I've imbibed of the "Python is not C or C++"
Kool-Aid, but I just don't see why we should encourage C idioms in
Python when there are plenty of other cases where use of a bare
expression rather "... is None" can bite.

I find Skip Montanaro's implied issue

    Please reconsider the Boolean evaluation of None

far more plausible.<wink/>

Footnotes: 
[1]  if event.date: print("Happy New Year/Century/Millennium!")


From mal at egenix.com  Thu Mar  6 09:35:59 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 09:35:59 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
Message-ID: <5318336F.7030009@egenix.com>

On 06.03.2014 03:23, Donald Stufft wrote:
> 
> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:
> 
>> [Donald Stufft]
>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that
>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile.
>> 
>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that
>> no code relies on the current behavior? For that matter, where is the evidence that the
>> current behavior is a significant cause of bugs?  I know I've used "if some_time_object:", 
>> but it did exactly what I expected it to do.
> 
> Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case
> what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s
> implementation.

Just in case you're looking for a relevant use case:

Having bool(time(0,0,0)) allows you to quickly check whether you are
dealing with a datetime value with (non-trivial) time part or not.

You run into situations where you have to test for this in
date/time conversions, arithmetics, etc. e.g. if you need to convert
a datetime value to a date and want to prevent truncation of
information, or if you want to format a datetime value in a user
friendly way by omitting the zero time part, or if you're doing
datetime arithmetic that has to follow special requirements w/r to
days (think interest or insurance math) and you want to
quickly check whether you can use the fast path simple
calculation, or you need to do the full blown complicated part.

Now, those use cases may still not appear relevant to you, but then
the argument that "if x:" should not trigger for time(0,0,0)
because x might actually be None, is also not relevant to me,
since it masks a bug in the example code and those should never
go undetected :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/ 
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/ mxODBC, mxDateTime,
>>> mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From donald at stufft.io  Thu Mar  6 09:46:17 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 03:46:17 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5318336F.7030009@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com>
Message-ID: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>


On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 06.03.2014 03:23, Donald Stufft wrote:
>> 
>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:
>> 
>>> [Donald Stufft]
>>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that
>>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile.
>>> 
>>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that
>>> no code relies on the current behavior? For that matter, where is the evidence that the
>>> current behavior is a significant cause of bugs?  I know I've used "if some_time_object:", 
>>> but it did exactly what I expected it to do.
>> 
>> Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case
>> what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s
>> implementation.
> 
> Just in case you're looking for a relevant use case:
> 
> Having bool(time(0,0,0)) allows you to quickly check whether you are
> dealing with a datetime value with (non-trivial) time part or not.

I don?t see how midnight is any more or less trivial than 12:01.

> 
> You run into situations where you have to test for this in
> date/time conversions, arithmetics, etc. e.g. if you need to convert
> a datetime value to a date and want to prevent truncation of
> information, or if you want to format a datetime value in a user
> friendly way by omitting the zero time part, or if you're doing
> datetime arithmetic that has to follow special requirements w/r to
> days (think interest or insurance math) and you want to
> quickly check whether you can use the fast path simple
> calculation, or you need to do the full blown complicated part.

I think these would be better served by actually checking for what you mean.
For the record I also think that checking for a date time to not be None
is also better served by actually checking for what you mean. The
difference is in intent and likelihood of confusion.

For all of those you?re going to be explicitly testing and working with
midnight objects, so you?re going to test them, you?re going to exercise
that code. In the ``is None`` case, Midnight is just another value so that
bug will lay dormant and undetected until someone just happens to
hit an unlucky time. And once you get that it?s likely to be difficult to
reproduce.

Further more this only ever works reliably on naive times and
doesn?t realistically work ?sanely? (if you consider this behavior sane)
on aware times at all.

So to be specific, I think using ``if time_or_something_that_may_be_none:``
is always wrong, but I think the like hood for user confusion and hard to
reproduce bugs are far greater when a more or less arbitrary time (especially
in the sense of aware times) evaluates as false when a user omits the
``is None`` than when someone expects ``if time:`` to be false at midnight.

> 
> Now, those use cases may still not appear relevant to you, but then
> the argument that "if x:" should not trigger for time(0,0,0)
> because x might actually be None, is also not relevant to me,
> since it masks a bug in the example code and those should never
> go undetected :-)
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/ 
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/ mxODBC, mxDateTime,
>>>> mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/75cb9ab0/attachment.sig>

From donald at stufft.io  Thu Mar  6 09:50:46 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 03:50:46 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com>
 <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
Message-ID: <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io>


On Mar 6, 2014, at 3:46 AM, Donald Stufft <donald at stufft.io> wrote:

> So to be specific, I think using ``if time_or_something_that_may_be_none:``
> is always wrong, but I think the like hood for user confusion and hard to
> reproduce bugs are far greater when a more or less arbitrary time (especially
> in the sense of aware times) evaluates as false when a user omits the
> ``is None`` than when someone expects ``if time:`` to be false at midnight.

I realized this was ambiguous, I always consider directly checking the boolean
value of a time object as wrong, as I don?t believe time has any True/False implications
in it. This includes both using ``if thing:`` to detect None and using ``if thing:`` to
detect midnight. I just think the ``if thing:`` to detect None case is more common
and also more difficult to detect when you have a bug because of it and more
surprising behavior than the other case.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/4b6b5934/attachment-0001.sig>

From flying-sheep at web.de  Thu Mar  6 10:22:08 2014
From: flying-sheep at web.de (Philipp A.)
Date: Thu, 6 Mar 2014 10:22:08 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
 <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io>
Message-ID: <CAN8d9gngoVTnEgiE+X2aJKkXz+oX+9cqT7sqghAYF2ngxMmB1Q@mail.gmail.com>

2014-03-06 6:34 GMT+01:00 Greg Ewing <greg.ewing at canterbury.ac.nz>:

> The point of having time objects as an abstraction is
> so that you *don't* have to think of them as an offset
> from anything. You simply think of them as times.


but times *are* offsets from midnight (24h clock) or midnight and noon (12h
clock)

1 o?clock = 1 hour after midnight

12 o?clock pm = 12 hours after noon

they are essentially radial coordinates ? and radial coordinates do have a
0 point.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cdf0ebcc/attachment.html>

From mal at egenix.com  Thu Mar  6 10:32:06 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 10:32:06 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>
 <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
Message-ID: <53184096.3050409@egenix.com>

On 06.03.2014 09:46, Donald Stufft wrote:
> 
> On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> On 06.03.2014 03:23, Donald Stufft wrote:
>>>
>>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:
>>>
>>>> [Donald Stufft]
>>>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that
>>>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile.
>>>>
>>>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that
>>>> no code relies on the current behavior? For that matter, where is the evidence that the
>>>> current behavior is a significant cause of bugs?  I know I've used "if some_time_object:", 
>>>> but it did exactly what I expected it to do.
>>>
>>> Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case
>>> what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s
>>> implementation.
>>
>> Just in case you're looking for a relevant use case:
>>
>> Having bool(time(0,0,0)) allows you to quickly check whether you are
>> dealing with a datetime value with (non-trivial) time part or not.
> 
> I don?t see how midnight is any more or less trivial than 12:01.
> 
>>
>> You run into situations where you have to test for this in
>> date/time conversions, arithmetics, etc. e.g. if you need to convert
>> a datetime value to a date and want to prevent truncation of
>> information, or if you want to format a datetime value in a user
>> friendly way by omitting the zero time part, or if you're doing
>> datetime arithmetic that has to follow special requirements w/r to
>> days (think interest or insurance math) and you want to
>> quickly check whether you can use the fast path simple
>> calculation, or you need to do the full blown complicated part.
> 
> I think these would be better served by actually checking for what you mean.
> For the record I also think that checking for a date time to not be None
> is also better served by actually checking for what you mean. The
> difference is in intent and likelihood of confusion.

FWIW, I expect bool(time(0,0,0)) == False, just like I expect
bool(0.0) == False. I would find it confusing to have
bool(zero_element) == True.

> For all of those you?re going to be explicitly testing and working with
> midnight objects, so you?re going to test them, you?re going to exercise
> that code. In the ``is None`` case, Midnight is just another value so that
> bug will lay dormant and undetected until someone just happens to
> hit an unlucky time. And once you get that it?s likely to be difficult to
> reproduce.

datetime values with zero time are *very* common in practice
(you run into them whenever you convert a date value into a datetime
value), so the "is None" case is easy to reproduce and will hit
you more often than you like.

BTW: Not using "is None" will bite you in many other ways as
well. None it typically used as value for "not initialized" or
"no value provided". This meaning is completely different from
"empty selection" or "empty string", so in order to address all
those cases as well, you'd have to change:

bool(()) == True
bool('') == True
etc.

The "if x is None:" test is also faster than "if x:", so actually
win something by coding correctly ;-)

Please provide a stronger argument for needing to change
bool(time(0,0,0)) than one which is built on buggy code :-)
(I'm running out of smileys...)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at gmail.com  Thu Mar  6 10:34:32 2014
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 6 Mar 2014 09:34:32 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53184096.3050409@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com>
 <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io>
 <53184096.3050409@egenix.com>
Message-ID: <CAKCKLWzWyhsxhxT2tvoF4KrUAF5okYp1aTqUZ96C3X5T90kGuQ@mail.gmail.com>

On 6 March 2014 09:32, M.-A. Lemburg <mal at egenix.com> wrote:

> On 06.03.2014 09:46, Donald Stufft wrote:
> >
> > On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >
> >> On 06.03.2014 03:23, Donald Stufft wrote:
> >>>
> >>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:
> >>>
> >>>> [Donald Stufft]
> >>>>> When the documented behavior is both nonsensical and the cause of
> hard to debug bugs that
> >>>>> is a pretty compelling use case to me, unless you actively enjoy
> being user hostile.
> >>>>
> >>>> Breaking code that currently works is as hostile as hostile gets.
> Where is the evidence that
> >>>> no code relies on the current behavior? For that matter, where is the
> evidence that the
> >>>> current behavior is a significant cause of bugs?  I know I've used
> "if some_time_object:",
> >>>> but it did exactly what I expected it to do.
> >>>
> >>> Forgive me if I'm wrong, but aren't you the author of the date time
> module? If that's the case
> >>> what you expect it to do isn't particularly relevant as you're
> intimately aware of it's
> >>> implementation.
> >>
> >> Just in case you're looking for a relevant use case:
> >>
> >> Having bool(time(0,0,0)) allows you to quickly check whether you are
> >> dealing with a datetime value with (non-trivial) time part or not.
> >
> > I don't see how midnight is any more or less trivial than 12:01.
> >
> >>
> >> You run into situations where you have to test for this in
> >> date/time conversions, arithmetics, etc. e.g. if you need to convert
> >> a datetime value to a date and want to prevent truncation of
> >> information, or if you want to format a datetime value in a user
> >> friendly way by omitting the zero time part, or if you're doing
> >> datetime arithmetic that has to follow special requirements w/r to
> >> days (think interest or insurance math) and you want to
> >> quickly check whether you can use the fast path simple
> >> calculation, or you need to do the full blown complicated part.
> >
> > I think these would be better served by actually checking for what you
> mean.
> > For the record I also think that checking for a date time to not be None
> > is also better served by actually checking for what you mean. The
> > difference is in intent and likelihood of confusion.
>
> FWIW, I expect bool(time(0,0,0)) == False, just like I expect
> bool(0.0) == False. I would find it confusing to have
> bool(zero_element) == True.
>
> > For all of those you're going to be explicitly testing and working with
> > midnight objects, so you're going to test them, you're going to exercise
> > that code. In the ``is None`` case, Midnight is just another value so
> that
> > bug will lay dormant and undetected until someone just happens to
> > hit an unlucky time. And once you get that it's likely to be difficult to
> > reproduce.
>
> datetime values with zero time are *very* common in practice
> (you run into them whenever you convert a date value into a datetime
> value), so the "is None" case is easy to reproduce and will hit
> you more often than you like.
>
> BTW: Not using "is None" will bite you in many other ways as
> well. None it typically used as value for "not initialized" or
> "no value provided". This meaning is completely different from
> "empty selection" or "empty string", so in order to address all
> those cases as well, you'd have to change:
>
> bool(()) == True
> bool('') == True
> etc.
>
> The "if x is None:" test is also faster than "if x:", so actually
> win something by coding correctly ;-)
>
> Please provide a stronger argument for needing to change
> bool(time(0,0,0)) than one which is built on buggy code :-)
> (I'm running out of smileys...)
>
>

I'm very surprised that "midnight evaluates to False is insane" is not in
and of itself a strong enough argument to change it. :-/

Michael




> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Mar 06 2014)
> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f31ac7ce/attachment-0001.html>

From solipsis at pitrou.net  Thu Mar  6 10:34:55 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 10:34:55 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
Message-ID: <lf9ffh$8ml$1@ger.gmane.org>

Le 06/03/2014 03:15, Tim Peters a ?crit :
> [Donald Stufft]
>> When the documented behavior is both nonsensical and the cause of hard
>> to debug bugs that is a pretty compelling use case to me, unless you actively
>> enjoy being user hostile.
>
> Breaking code that currently works is as hostile as hostile gets.
> Where is the evidence that no code relies on the current behavior?
> For that matter, where is the evidence that the current behavior is a
> significant cause of bugs?  I know I've used "if some_time_object:",
> but it did exactly what I expected it to do.

The current behaviour being highly unintuitive is a likely source of 
bugs, IMO.

Regards

Antoine.



From solipsis at pitrou.net  Thu Mar  6 10:52:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 10:52:12 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5318336F.7030009@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com>
Message-ID: <lf9gft$jnh$1@ger.gmane.org>

Le 06/03/2014 09:35, M.-A. Lemburg a ?crit :
> Just in case you're looking for a relevant use case:
>
> Having bool(time(0,0,0)) allows you to quickly check whether you are
> dealing with a datetime value with (non-trivial) time part or not.

This sounds like the kind of shortcut that only an expert would know to 
take, not something that's actually a good design decision.
It's as though pathlib.Path('') evaluated to false for some obscure use 
case that I cared about.

It is not a coincidence, IMO, that the three persons arguing that the 
current behaviour is reasonable are implementors of datetime modules: 
you, Tim and Alexander. All other people find the behaviour baffling.

Regards

Antoine.



From ncoghlan at gmail.com  Thu Mar  6 11:11:21 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 6 Mar 2014 20:11:21 +1000
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <lf8rh3$suv$1@ger.gmane.org>
References: <lf8rh3$suv$1@ger.gmane.org>
Message-ID: <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>

On 6 Mar 2014 13:55, "Sturla Molden" <sturla.molden at gmail.com> wrote:
>
> "goto fail" is a well-known error handling mechanism in open source
software, widely reputed for its robusteness:
>
>
http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c
>
>
https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c
>
> I believe Python needs to add support for this superior paradigm.

Unfortunately, we can't throw stones about those, since we currently have
hostname matching off by default and expect developers to turn it on.

That needs to change, but there are a few thorny practical issues to sort
out in order to get there (Christian Heimes has been working through
several of them).

Cheers,
Nick.

>
> It would involve a new keyword "fail" and some means of goto'ing to it. I
suggest "raise to fail":
>
> if (some_error):
>    raise to fail
>
> fail:
>    <error handling code>
>
> Unless there are many objections, this fantastic idea might be submitted
in a (short) PEP somewhere around the beginning of next month.
>
> There is some obvious overlap with the rejected "goto PEP" (PEP 3163) and
the Python 2.3 goto module. However, the superiority of goto fail as error
generation and error handling paradigm has since then been thoroughly
proven.
>
> http://legacy.python.org/dev/peps/pep-3136/
> http://entrian.com/goto/download.html
>
>
> Regards,
> Sturla Molden
>
>
>
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/a6fbc1f8/attachment.html>

From solipsis at pitrou.net  Thu Mar  6 11:13:08 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:13:08 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com>
Message-ID: <lf9hn6$56a$1@ger.gmane.org>

Le 06/03/2014 07:47, Alexander Belopolsky a ?crit :
> I don't understand why people are denying that midnights are special
> points.

Because they are not?
Really, this is getting ridiculous. In any intuitive and common notion 
of time, midnight is not a special point at all. There is a reason most 
people on this thread disagree with you.
(and, no, it's not because they are being dense or ignorant)

Regards

Antoine.



From solipsis at pitrou.net  Thu Mar  6 11:19:46 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:19:46 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <loom.20140305T220007-633@post.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
Message-ID: <lf9i3j$56a$2@ger.gmane.org>


Le 05/03/2014 22:10, Alex Gaynor a ?crit :
> First, I'd like to dispense with the notion that ``if foo`` is somehow a bad
> practice. This is an EXTREMELY common practice as a shorthand for checking for
> None, a great many people consider it more idiomatic, this really isn't a place
> to bikeshed that particlar idiom.

Agreed. If we didn't want people to rely on that idiom then __bool__ 
would have to raise TypeError on most types except a select ones.

Regards

Antoine.



From solipsis at pitrou.net  Thu Mar  6 11:27:22 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:27:22 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306021457.GF28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io>
 <20140306021457.GF28804@ando>
Message-ID: <lf9ihs$fos$1@ger.gmane.org>

Le 06/03/2014 03:14, Steven D'Aprano a ?crit :
>
> For what it's worth, I think the current behaviour is a misfeature, and
> in a perfect world it would be fixed. But the cost of this is so
> miniscule, and the work-around so trivial, that although the fix is
> cheap, the benefit is even lower.

Again, the problem is not the cost of the workaround, but the cost of 
sporadic unexpected failures when you don't *know* your code is 
vulnerable to this issue.

Regards

Antoine.



From mal at egenix.com  Thu Mar  6 11:33:08 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 11:33:08 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9gft$jnh$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org>
Message-ID: <53184EE4.80105@egenix.com>

On 06.03.2014 10:52, Antoine Pitrou wrote:
> Le 06/03/2014 09:35, M.-A. Lemburg a ?crit :
>> Just in case you're looking for a relevant use case:
>>
>> Having bool(time(0,0,0)) allows you to quickly check whether you are
>> dealing with a datetime value with (non-trivial) time part or not.
> 
> This sounds like the kind of shortcut that only an expert would know to take, not something that's
> actually a good design decision.
> It's as though pathlib.Path('') evaluated to false for some obscure use case that I cared about.

It doesn't ? :-)

> It is not a coincidence, IMO, that the three persons arguing that the current behaviour is
> reasonable are implementors of datetime modules: you, Tim and Alexander. All other people find the
> behaviour baffling.

I'm not so sure. Of course, people who have been doing date/time
stuff will jump into discussions that deal with them, but the same
argument triggering this thread could have applied to other Python
types and you would then have a different group of people argue
for plausibility of the choice.

I find it strange that the only argument against having time(0,0,0)
evaluate to False in a boolean context is a wrongly used
None test. This argument can be applied to lots of other types as
well, where the buggy None test masks errors.

Perhaps someone could present at least one compelling argument
that is not based on wrong usage of None in if statements.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Thu Mar  6 11:39:20 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:39:20 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com>
 <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
 <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
Message-ID: <lf9j8a$ole$1@ger.gmane.org>

Le 06/03/2014 00:22, Alexander Belopolsky a ?crit :
>
> No.  It is False because it equals False:
>
>  >>> 0 == False
> True

I think this is a misunderstanding on your part. Python considered 0 as 
false-y before the bool type was introduced in the language. Actually, 0 
was *the* spelling of "false" before "False" was introduced.

 > I think proponents of bool(time(0)) == True view it as a container of
 > hours, minutes and seconds.

No, they view it as an abstraction around the common concept of time.

Regards

Antoine.



From solipsis at pitrou.net  Thu Mar  6 11:43:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:43:12 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306021948.GG28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <20140306021948.GG28804@ando>
Message-ID: <lf9jfh$ole$2@ger.gmane.org>

Le 06/03/2014 03:19, Steven D'Aprano a ?crit :
>
> It may only be a convention that an instant before midnight is the end
> of the day and midnight the beginning of the next, but it is a
> convention: midnight is the origin (i.e. zero point) of the day. That
> makes it arguably as falsey as 0, 0.0 and 0j.

A zero in algebra has a special meaning with respect to an operator 
(e.g. addition). Midnight doesn't have such a special meaning.

Regards

Antoine.



From solipsis at pitrou.net  Thu Mar  6 11:46:13 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 11:46:13 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53184EE4.80105@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com>
Message-ID: <lf9jl7$ole$3@ger.gmane.org>

Le 06/03/2014 11:33, M.-A. Lemburg a ?crit :
>
> I find it strange that the only argument against having time(0,0,0)
> evaluate to False in a boolean context is a wrongly used
> None test.

It's only "wrongly used" because of that particular wart.
In other words, the "wrongly used" None test *exposes* the wart, it is 
not its *cause*.

There are other contexts with implicit boolean conversion, such as any() 
and all().

Regards

Antoine.



From steve at pearwood.info  Thu Mar  6 11:52:53 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 21:52:53 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5317FC67.7060600@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <5317FC67.7060600@canterbury.ac.nz>
Message-ID: <20140306105253.GI28804@ando>

On Thu, Mar 06, 2014 at 05:41:11PM +1300, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >There's also the argument from consistency: midnight is the zero point 
> >of the day, and zero is falsey.
> 
> Regarding midnight as a "zero point" is an arbitrary
> convention.

True, as I mentioned in another post.

Nevertheless, we have many arbitrary conventions enshrined in Python.
Using base 10 for the representation of numbers, instead of, say, 
balanced ternary notation, is an arbitrary convention. Numbering 
sequences from 0 is another.

If you would prefer another convention for the zero point of the day 
(say, midday, dusk, dawn, or 2:15pm) then feel free to subclass 
datetime.time :-)

Or even no zero point at all. Tossing out a wild idea, why not leave 
datetime.time alone and add a new class which is exactly the same but is 
always truthy? That doesn't break backward compatibility, and so it 
could happen immediately (too late for 3.4, but certainly for 3.5).

All we need is an appropriate name, and a decision as to which ought to 
inherit from which. The bike-shedding should be over by 3.5 alpha 1 
*wink*

Despite my quip about the bike-shedding, I am actually serious. 

-- 
Steven

From masklinn at masklinn.net  Thu Mar  6 11:53:59 2014
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 6 Mar 2014 11:53:59 +0100
Subject: [Python-ideas] One more time... lambda function <--- from ***
	signature def.
In-Reply-To: <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando>
 <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com>
 <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com>
 <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz>
 <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net>
 <53179C51.4020001@canterbury.ac.nz>
 <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net>
 <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4E54B6BB-2AA6-4D57-96BC-3C7D4C807030@masklinn.net>

On 2014-03-06, at 08:29 , Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Masklinn writes:
>> On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>> Masklinn wrote:
>>>> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
>>> That comes at the expense of making *everything* a thunk
>>> until its value is needed [in Haskell].
>> 
>> Of course not, Haskell allows explicitly forcing a thunk and the
>> compiler/runtime pair can use strictness analysis and not create thunks
>> in the first place.
> 
> That doesn't seem fair to me.  "Explicitly forcing" by definition
> means the programmer has decided the value is needed

No, it means the programmer wants the value to be strictly evaluated
(this is generally a space optimisation). It has no impact on program
behavior (assuming infinite resources, one issue of accumulating
thunks is the risk of OOM if they remain live but unevaluated).

> and "strictness
> analysis" is an optimization, not part of the language definition.  Am
> I missing something?

No? It just demonstrates that not everything is a thunk in haskell, there
are both thunks and strict values living around in the runtime, and
that's not visible in the type system.

>> Consider that a thunk is a deferral of an expression's evaluation,
>> why would said expression's evaluation happen more than once? The
>> only thing which changes is *when* the actual evaluation happens.
> 
> Are thunks guaranteed not to leak out of the scope of the
> containing expression, so assignment is impossible?

I'm not sure I understand the question.

> If they can leak, the thunk would be an object

I think that's just one possible implementation, a thunk could also be
some sort of reference indirection, a tagged pointer, or a deeply
integrated proxy-ish object.

> and in Python
> "assigning" it to a "variable" actually just creates a reference.
> Evaluation could memoize the thunk's value ("evaluation only happens
> once") or reevaluate the thunk each time its value is requested.
> Either seems potentially surprising to me.

Absolutely.

> Memoization makes sense if you think of the thunk as the closure of a
> computation that conceptually takes place at definition time.  I'm not
> sure if in the use cases for thunks it really makes sense to "close"
> the thunk at evaluation time, but it seems like a plausible
> interpretation to me.

I find it the most *useful* definition: for the other one we already
have functions and I'm not sure thunks would have much use as shorter
arguments-less functions.

It also has the property that "client" code (code receiving thunks
without necessarily being aware of it) remains valid and unsurprising
in that

    a = foo
    b = foo
    assert a is b

remains true regardless of `foo` being a thunk (the only difference
being that if `foo` is a thunk it may remain unforced throughout).


From p.f.moore at gmail.com  Thu Mar  6 11:57:22 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 6 Mar 2014 10:57:22 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53184EE4.80105@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com>
Message-ID: <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com>

On 6 March 2014 10:33, M.-A. Lemburg <mal at egenix.com> wrote:
>> It is not a coincidence, IMO, that the three persons arguing that the current behaviour is
>> reasonable are implementors of datetime modules: you, Tim and Alexander. All other people find the
>> behaviour baffling.
>
> I'm not so sure. Of course, people who have been doing date/time
> stuff will jump into discussions that deal with them, but the same
> argument triggering this thread could have applied to other Python
> types and you would then have a different group of people argue
> for plausibility of the choice.

I had promised to stay out of this discussion as I no longer thing it
adds much value (note that the bug has been reopened long since) and I
was getting tired of having people decide that I'd said something I
didn't mean.

But just to add a perspective from someone who *doesn't* develop a
datetime module, I encounter "special" midnight values very often.
Databases (at least Oracle, and I believe others) typically store date
values ("Thursday 6th March 2014") as a timestamp value with midnight
as a time part. So checking the time part of a datetime value for
midnight as a "special, means omitted" value is not uncommon in that
domain.

Paul

From stephen at xemacs.org  Thu Mar  6 11:57:27 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 19:57:27 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <201403052118.38295.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org>
 <201403052118.38295.shai@platonix.com>
Message-ID: <87vbvrppbc.fsf@uwakimon.sk.tsukuba.ac.jp>

Shai Berger writes:

 > Unless you're aware of the issue, you are not going to unit-test
 > specifically for midnight. If you are aware, you don't need to
 > detect it, you write "is not None".

The issue, as Skip has argued at length, is that by using the implicit
cast to Boolean you've changed the nature of the test from "membership
in valid values" to "a true-ish value".[1]

This can be detected generically with grep, something like:

    grep -E '^ *if [^<>=]*:' | grep -v '^ *if.* is .*:'

with a (possibly large) number of false positives.  (I suspect it's
not worth worrying about no false negatives; if the 'is' operator is
present, the test is boolean except if it's part of an arithmetic
expression such as "3 + (2 is not None)".)  You might want to include
modulo (%) in the list of operators.  Or it might be more accurate to
check for bare variables only and assume any expression should OK.

The false positives can be drastically reduced by checking for a
Boolean or container initializer in the case of "if var:".  None of
pep8, pylint, or pychecker does this now AFAICT, but they could, I
think.

Footnotes: 
[1]  Implementing a membership test as a test for a sentinel is a
bit fishy in this context of arguing about the "nature of the test",
but given the much less obvious nature of the conversion to bool in
complex types like dates and times, I'll go with Skip any time.


From mal at egenix.com  Thu Mar  6 12:04:56 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 12:04:56 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9jl7$ole$3@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>	<lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
Message-ID: <53185658.5080507@egenix.com>

On 06.03.2014 11:46, Antoine Pitrou wrote:
> Le 06/03/2014 11:33, M.-A. Lemburg a ?crit :
>>
>> I find it strange that the only argument against having time(0,0,0)
>> evaluate to False in a boolean context is a wrongly used
>> None test.
> 
> It's only "wrongly used" because of that particular wart.
> In other words, the "wrongly used" None test *exposes* the wart, it is not its *cause*.

Wait. Let's be clear on this:

Writing

    if x: print ('x is None')

or

    if x == None: print ('x is None')

is wrong code.

None is a singleton, so you have to use the "is" operator, i.e.

    if x is None: print ('x is None')

is the only correct way of testing for None.

Adding a test to pylint for this would probably be more helpful
and address the real problem in a more productive way, than the
current discussion to making the above problematic code work in
the single use case of x being a datetime.time object.

> There are other contexts with implicit boolean conversion, such as any() and all().

If you expect those to work correctly for None testing, then
you're writing wrong code as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Thu Mar  6 12:09:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 12:09:12 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53185658.5080507@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>	<lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
Message-ID: <lf9l09$9nv$1@ger.gmane.org>

Le 06/03/2014 12:04, M.-A. Lemburg a ?crit :
>
> Writing
>
>      if x: print ('x is None')
>
> or
>
>      if x == None: print ('x is None')
>
> is wrong code.

No it isn't.

> None is a singleton, so you have to use the "is" operator, i.e.
>
>      if x is None: print ('x is None')
>
> is the only correct way of testing for None.

No it isn't.  x == None is a perfectly well-defined way of doing it, 
even if it isn't the stylistically preferred one.

Regards

Antoine.



From stefan at bytereef.org  Thu Mar  6 12:13:36 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 6 Mar 2014 11:13:36 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Please_reconsider_the_Boolean_evaluation?=
	=?utf-8?q?_of=09midnight?=
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
Message-ID: <loom.20140306T120303-396@post.gmane.org>

Antoine Pitrou <solipsis at ...> writes:
> Le 06/03/2014 09:35, M.-A. Lemburg a ?crit :
> > Just in case you're looking for a relevant use case:
> >
> > Having bool(time(0,0,0)) allows you to quickly check whether you are
> > dealing with a datetime value with (non-trivial) time part or not.
> 
> This sounds like the kind of shortcut that only an expert would know to 
> take, not something that's actually a good design decision.
> It's as though pathlib.Path('') evaluated to false for some obscure use 
> case that I cared about.

On the other hand I would not dare to assume it's True without
checking.


> It is not a coincidence, IMO, that the three persons arguing that the 
> current behaviour is reasonable are implementors of datetime modules: 
> you, Tim and Alexander. All other people find the behaviour baffling.

Not all. :)  I think viewing midnight as (unix_time % 86400) is
quite reasonable.


Stefan Krah 



From mal at egenix.com  Thu Mar  6 12:15:47 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 12:15:47 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53185658.5080507@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>	<lf9gft$jnh$1@ger.gmane.org>	<53184EE4.80105@egenix.com>
 <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com>
Message-ID: <531858E3.3030805@egenix.com>

On 06.03.2014 12:04, M.-A. Lemburg wrote:
> On 06.03.2014 11:46, Antoine Pitrou wrote:
>> Le 06/03/2014 11:33, M.-A. Lemburg a ?crit :
>>>
>>> I find it strange that the only argument against having time(0,0,0)
>>> evaluate to False in a boolean context is a wrongly used
>>> None test.
>>
>> It's only "wrongly used" because of that particular wart.
>> In other words, the "wrongly used" None test *exposes* the wart, it is not its *cause*.
> 
> Wait. Let's be clear on this:
> 
> Writing
> 
>     if x: print ('x is None')

The above is obviously wrong :-). Let's try again:

      if not x: print ('x is None')

> or
> 
>     if x == None: print ('x is None')
> 
> is wrong code.
> 
> None is a singleton, so you have to use the "is" operator, i.e.
> 
>     if x is None: print ('x is None')
> 
> is the only correct way of testing for None.
> 
> Adding a test to pylint for this would probably be more helpful
> and address the real problem in a more productive way, than the
> current discussion to making the above problematic code work in
> the single use case of x being a datetime.time object.
> 
>> There are other contexts with implicit boolean conversion, such as any() and all().
> 
> If you expect those to work correctly for None testing, then
> you're writing wrong code as well.
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Thu Mar  6 12:22:47 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 12:22:47 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9l09$9nv$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>	<lf9gft$jnh$1@ger.gmane.org>	<53184EE4.80105@egenix.com>
 <lf9jl7$ole$3@ger.gmane.org>	<53185658.5080507@egenix.com>
 <lf9l09$9nv$1@ger.gmane.org>
Message-ID: <53185A87.5010109@egenix.com>

On 06.03.2014 12:09, Antoine Pitrou wrote:
> Le 06/03/2014 12:04, M.-A. Lemburg a ?crit :
>>
>> Writing
>>
>>      if x: print ('x is None')
>>
>> or
>>
>>      if x == None: print ('x is None')
>>
>> is wrong code.
> 
> No it isn't.
> 
>> None is a singleton, so you have to use the "is" operator, i.e.
>>
>>      if x is None: print ('x is None')
>>
>> is the only correct way of testing for None.
> 
> No it isn't.  x == None is a perfectly well-defined way of doing it, even if it isn't the
> stylistically preferred one.

Depends on what kind of type x is and how that type implements
the comparison slots. It is perfectly well possible to define a
type that returns True for (x == None), even though x is not None :-)

>>> class C:
...     def __eq__(self, other):
...         return (other is None)
...
>>> o = C()
>>> o == None
True

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From rob.cliffe at btinternet.com  Thu Mar  6 12:36:40 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Mar 2014 11:36:40 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com>
 <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com>
Message-ID: <53185DC8.3000707@btinternet.com>


On 06/03/2014 10:57, Paul Moore wrote:
>
> I had promised to stay out of this discussion as I no longer thing it
> adds much value (note that the bug has been reopened long since) and I
> was getting tired of having people decide that I'd said something I
> didn't mean.
>
> But just to add a perspective from someone who *doesn't* develop a
> datetime module, I encounter "special" midnight values very often.
> Databases (at least Oracle, and I believe others) typically store date
> values ("Thursday 6th March 2014") as a timestamp value with midnight
> as a time part. So checking the time part of a datetime value for
> midnight as a "special, means omitted" value is not uncommon in that
> domain.
>
> Paul
You're joking, right?
How do you distinguish "omitted" from "provided, equals midnight" ?
Rob Cliffe
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14
>
>


From rob.cliffe at btinternet.com  Thu Mar  6 12:40:21 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Mar 2014 11:40:21 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140306105253.GI28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando>
Message-ID: <53185EA5.3060802@btinternet.com>


On 06/03/2014 10:52, Steven D'Aprano wrote:
>
> True, as I mentioned in another post.
>
> Nevertheless, we have many arbitrary conventions enshrined in Python.
> Using base 10 for the representation of numbers, instead of, say,
> balanced ternary notation, is an arbitrary convention. Numbering
> sequences from 0 is another.
>
> If you would prefer another convention for the zero point of the day
> (say, midday, dusk, dawn, or 2:15pm) then feel free to subclass
> datetime.time :-)
>
> Or even no zero point at all. Tossing out a wild idea, why not leave
> datetime.time alone and add a new class which is exactly the same but is
> always truthy? That doesn't break backward compatibility, and so it
> could happen immediately (too late for 3.4, but certainly for 3.5).
>
> All we need is an appropriate name, and a decision as to which ought to
> inherit from which. The bike-shedding should be over by 3.5 alpha 1
> *wink*
>
> Despite my quip about the bike-shedding, I am actually serious.
You're joking, right?
Suppose you were considering taking up a new language and you found 
something in the docs which said:
"Here's a class that does so-and-so.  And here's another slightly 
different one that you can use instead, because we didn't get the first 
one quite right."
Would that be a turn-on?
Rob Cliffe
>   
>


From stephen at xemacs.org  Thu Mar  6 12:51:29 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 20:51:29 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <201403051924.05252.shai@platonix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <201403051924.05252.shai@platonix.com>
Message-ID: <87txbbpmta.fsf@uwakimon.sk.tsukuba.ac.jp>

Shai Berger writes:

 > At issue here is the fact that the idiom doesn't fit where one
 > would expect that it should.

This is the real problem.  People are programming in un-Pythonic style
by analogy to either other languages or to other cases in Python where
the __len__ method provides a natural cast to Boolean, and deciding
since they like the idiom, the behavior of time.time is "unexpected."

But your claim is *not* a "fact".  Quite the reverse, I was mildly
surprised even the last time around that anybody would argue that it's
a good idea to use a sentinel to indicate "uninitialized" and then
fail to test explicitly for the sentinel.

For example, I wouldn't be surprised to see something like this in an
MTA or other program that should not let data out of its sight without
a timestamp:

    my_time = None

    if not my_time and x:
        my_time = x.supposed_to_be_time_but_in_fact_sometimes_0

    if not my_time:
        my_time = time.now()

What are the odds that you'll detect the bug in initializing 'x' in a
timely fashion?  OTOH, if you test for "is None" you'll probably get a
quick TypeError the first time the bug manifests.  (No guarantees, of
course, but why trade a potentially easy bug for a probably hard one?)

If you asked me to implement a Time class, would I have any false
instances?  Quite possibly, yes.  Of course I might implement it as a
namedtuple (h, m, s), in which case it would always be true.  OTOH, I
might implement it as seconds-since-midnight, and probably not bother
to give it a __bool__ method to ensure bool(midnight) == True.

So I don't really care *why* Uncle Timmy chose to make midnight be
false, I only offer kudos for documenting it. :-)

From p.f.moore at gmail.com  Thu Mar  6 12:54:34 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 6 Mar 2014 11:54:34 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53185DC8.3000707@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com>
 <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com>
 <53185DC8.3000707@btinternet.com>
Message-ID: <CACac1F-Ed0A-5PNp6b+VVs9CzeMDYYGq_H8=mHW_N2P4LkEQcw@mail.gmail.com>

On 6 March 2014 11:36, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> You're joking, right?

No. You just sent 2 "you're joking" messages. You're joking, right?
Any time anyone presents a use case or proposal that contradicts what
you want it must be a joke?

I really am sick of being misinterprested and misrepresented on this thread.

> How do you distinguish "omitted" from "provided, equals midnight" ?

I will not explain all the details of how database applications
process dates here. All I offered was a perspective that some
application domains, in certain cases, see "midnight" as having a
special meaning distinct from the specific point in time. I did so
because other people's views were being characterised as irrelevant
because they developed datetime modules (as if that made them somehow
*less* qualified to have an opinion) and I sympathised with them over
the hostile reception they were getting. I didn't comment on whether
that meant the Python behaviour was good or bad, or whether the
practice of truth-testing time values was good or not. Feel free to
investigate the details if you care, or ignore my post if you don't.
But I'm not trolling, nor do I appreciate the implication that I am.

Paul.

From rob.cliffe at btinternet.com  Thu Mar  6 12:56:31 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Mar 2014 11:56:31 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
Message-ID: <5318626F.5080203@btinternet.com>


On 06/03/2014 02:15, Tim Peters wrote:
> [Donald Stufft]
>> When the documented behavior is both nonsensical and the cause of hard
>> to debug bugs that is a pretty compelling use case to me, unless you actively
>> enjoy being user hostile.
> Breaking code that currently works is as hostile as hostile gets.
> Where is the evidence that no code relies on the current behavior?
> For that matter, where is the evidence that the current behavior is a
> significant cause of bugs?  I know I've used "if some_time_object:",
> but it did exactly what I expected it to do.
Well, let's start a survey.  So far we have:

Instances of existing code that would be broken by the change: 0
Instances of code where the unexpected behaviour caused a bug: 1 (the 
OP's code).

I invite everyone to add instances that they know about.:-)

Rob Cliffe
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14
>
>


From ncoghlan at gmail.com  Thu Mar  6 13:08:54 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 6 Mar 2014 22:08:54 +1000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53185658.5080507@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
Message-ID: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>

On 6 March 2014 21:04, M.-A. Lemburg <mal at egenix.com> wrote:
> Wait. Let's be clear on this:
>
> Writing
>
>     if x: print ('x is None')
>
> or
>
>     if x == None: print ('x is None')

The case in question is essentially this one:

    if x:
        assert x is not None # Always valid!
        ....
    else:
        assert x is None # Valid for most user defined types

There is a learned intuition that people naturally acquire when learning Python:

    - numbers may be false (it means zero)
    - containers may be false (it means empty)
    - everything else is always true (as that's the default for user
defined classes)

Where datetime().time() is confusing is the fact the expected
behaviour changes based on *which* category you put the number in.

The vast majority of Python's users will place structured date and
time objects in the "arbitrary object" category, and expect them to
always be true. When using None as a sentinel for such types, writing
"if x:" when you really mean "if x is not None:" is a harmless style
error, with no practical ill-effects.

When dealing with time objects, such users are unlikely to be crafting
test cases to ensure that "midnight UTC" is handled correctly, so
their "harmless style error" is in fact a subtle data driven bug
waiting to bite them. It is *really* hard for a static analyser to
pick this up, because at point of use "if x:" gives no information
about the type of "x", and hence any such alert would have an
unacceptably high false positive rate.

Now, let's consider the time with the *best* possible claim to being
false: timestamp zero. How does that behave?

Python 3.3:

>>> bool(dt.datetime.fromtimestamp(0))
True
>>> bool(dt.datetime.fromtimestamp(0).date())
True
>>> bool(dt.datetime.fromtimestamp(0).time())
True

Huh, if times are supposed to be valid as truth values, that looks
rather weird. What's going on?

>>> dt.datetime.fromtimestamp(0).time()
datetime.time(10, 0)

Oh, I'm in Brisbane - *of course* the truthiness of timestamps should
depend on my timezone! Clearly, what I really meant was timestamp
-36000, or perhaps 50400:

>>> bool(dt.datetime.fromtimestamp(-36000).time())
False
>>> bool(dt.datetime.fromtimestamp(50400).time())
False

So, unless I happen to live in UTC, it's highly unlikely that I'm
going to infer from Python's *behaviour* that datetime.time() (unlike
datetime.date() and datetime.datetime()) belong in the "number"
category, rather than the "arbitrary object" category.

Perhaps it behaves like a number in other ways:

>>> utcmidnight = dt.datetime.fromtimestamp(50400).time()
>>> utcmidnight + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'datetime.time' and 'int'
>>> utcmidnight * 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'datetime.time' and 'int'
>>> int(utcmidnight)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: int() argument must be a string or a number, not 'datetime.time'

Hmm, nope. And that last one *explicitly* tells me it's not a number!

There's a great saying in the usability world: "You can't document
your way out of a usability problem". What it means is that if all the
affordances of your application (or programming language!) push users
towards a particular logical conclusion (in this case, "datetime.time
values are not numbers"), having a caveat in your documentation isn't
going to help, because people aren't even going to think to ask the
question. It doesn't matter if you originally had a good reason for
the behaviour, you've ended up in a place where your behaviour is
confusing and inconsistent, because there is one piece of behaviour
that is out of line with an otherwise consistent mental model.

But perhaps I've been told "midnight is false in boolean context". But
which midnight? There are three that apply to me:

>>> naivemidnight
datetime.time(0, 0)
>>> utcmidnight
datetime.time(0, 0, tzinfo=datetime.timezone.utc)
>>> localmidnight
datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000)))

Are they all False? No, no they're not (unless your local timezone is UTC):

>>> bool(utcmidnight)
False
>>> bool(naivemidnight)
False
>>> bool(localmidnight)
True

There's a phrase for APIs like this one: "expert friendly". Experts
like a particular behaviour because it lets them do advanced things
(like leave the door open for modular arithmetic on timestamp.time()
values). However, that's not normally something we see as a virtue
when designing APIs for Python - instead, we generally aim for layered
complexity, where simple things are simple, and we provide power tools
for advanced users that want them.

Now, suppose this boolean behaviour went away. How would I detect if a
value was midnight or not? Well, first, I would need to clarify my
question. Do I mean midnight UTC? Or do I mean midnight local time? It
makes a difference for aware objects, after all. Or perhaps I'm not
interested in aware objects at all, and only care about naive
midnight. Using appropriately named values like those I defined above,
those questions are all very easy to express explicitly in ways that
don't rely on readers understanding that "bool(x)" on a datetime.time
object means the same thing as "x is naive midnight or UTC midnight":

    if x not in (naivemidnight, utcmidnight):
        # This is the current bool(x)
        ....
   if x != naivemidnight:
        # This has no current shorthand
       ....
    if x != utcmidnight:
        # This has no current shorthand
       ....
    if x != localmidnight:
        # This has no current shorthand
        ....

And just to demonstrate that equivalence is accurate:

>>> local_10am = localmidnight.replace(hour=10)
>>> local_10am
datetime.time(10, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000)))
>>> bool(local_10am)
False
>>> local_10am != utcmidnight
False
>>> local_10am not in (naivemidnight, utcmidnight)
False

While it was originally the desire to reduce the impact of a very
common bug that prompted me to reopen the issue, it is the improved
consistency in the behaviour presented to users of Python 3.6+ that
makes me belief this is actually worth fixing. We embarked on the
whole Python 3 exercise in the name of making the language easier to
learn by removing legacy features and fixing some problematic defaults
and design warts - this is a tiny tweak by comparison, and will still
go through the full normal deprecation cycle (warning in 3.5,
behavioural change in 3.6)

Regards,
Nick.

From stephen at xemacs.org  Thu Mar  6 13:18:26 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 21:18:26 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051553.19311.shai@platonix.com>
 <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com>
 <201403051719.09276.shai@platonix.com>
 <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com>
 <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com>
Message-ID: <87siqvplkd.fsf@uwakimon.sk.tsukuba.ac.jp>

Ryan Hiebert writes:

 > he'd just like to see Python be better.

We all agree that we'd like Python to be better.  But many of us feel
that the proposed change does not make Python better.

It merely makes it more comfortable for what we (many of us, anyway)
consider to be badly written code, and encourages people to write more
code in poor style as we see it.

 > it seems that we are disagreeing on whether this strange behavior
 > should ever change.

Well, yes, for Python 2 (obviously) and Python 3.  I don't think
anybody would object to changing it in Python 4.  (Honestly, I would,
on the principle above -- it really is only useful if you want to
write code in bad style -- but it doesn't take a French soldier on the
parapet to let you know in which general direction the wind blows.)

 > I think that, at some point, it should. If we ever agree on that,
 > then we can start thinking about when.

I really don't think you'll achieve consensus before we start
discussing Python 4 seriously.  That's when the hunting season on
backward incompatible changes (that don't correct showstoppers) opens.


From steve at pearwood.info  Thu Mar  6 13:35:20 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 6 Mar 2014 23:35:20 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <53185EA5.3060802@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando>
 <53185EA5.3060802@btinternet.com>
Message-ID: <20140306123520.GL28804@ando>

On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote:
> 
> On 06/03/2014 10:52, Steven D'Aprano wrote:

> >Or even no zero point at all. Tossing out a wild idea, why not leave
> >datetime.time alone and add a new class which is exactly the same but is
> >always truthy? That doesn't break backward compatibility, and so it
> >could happen immediately (too late for 3.4, but certainly for 3.5).
> >
> >All we need is an appropriate name, and a decision as to which ought to
> >inherit from which. The bike-shedding should be over by 3.5 alpha 1
> >*wink*
> >
> >Despite my quip about the bike-shedding, I am actually serious.

> You're joking, right?

What part of "I am actually serious" suggests to you that I am joking? 
No, I am not joking.

When you have a misdesigned API that cannot be fixed without breaking 
backwards compatibility, or at least cannot be fixed *quickly*, it may 
be sensible to leave the old API in place and place a new one 
side-by-side. If the old one is actively harmful, it can even be 
deprecated for eventual removal, otherwise it can remain indefinitely. 
Does the term "legacy API" sound familiar?

This is *extremely common* in the software industry, at least in that 
part that isn't run by cowboy coders who have no concern about breaking 
their users' code. It's quite common in Python too: consider the legacy 
API of os.system, the deprecated API of the popen2 module, and the 
modern subprocess API.


> Suppose you were considering taking up a new language and you found 
> something in the docs which said:
> "Here's a class that does so-and-so.  And here's another slightly 
> different one that you can use instead, because we didn't get the first 
> one quite right."
> Would that be a turn-on?

Yes. This would tell me that the developers of this language took 
backward compatibility seriously, and that they weren't afraid to admit 
to mistakes.



-- 
Steven

From stefan at bytereef.org  Thu Mar  6 13:36:46 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 6 Mar 2014 12:36:46 +0000 (UTC)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
Message-ID: <loom.20140306T131407-724@post.gmane.org>

Mark H. Harris <harrismh777 at ...> writes:
> At this point in the discussion please try not to think "implementation,"
> rather think conceptually in an open framework where anything is possible. 

As I understand, you are suggesting some kind of number system with
symbolic representations that can be converted to decimal if needed.

The topic is so vast that the only chance of it happening is for
someone to provide an implementation on PyPI and see if people
like it.  There are packages for symbolic mathematics like sympy,
perhaps that is a start.


Even if people like it, it would be nearly impossible to integrate
it into the core language without breaking third-party packages on
a very large scale. So it is unlikely that such a thing will make
it into the (hypothetical) 4.0.


Regarding decimal literals:

Possible in 3.x, but it would require some investigation if
people really want arbitrary precision arithmetic by default
rather than e.g. IEEE Decimal64.



Regarding decimal floats by default:

Perhaps in 4.0, but IMO only with *full* backing by the large SciPy
and NumPy communities. My guess is that it's not going to happen, not
in the least because the literature for floating point algorithms
(with proofs!) has an overwhelming focus on binary floating point.







From donald at stufft.io  Thu Mar  6 13:46:26 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 07:46:26 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140306123520.GL28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando>
 <53185EA5.3060802@btinternet.com> <20140306123520.GL28804@ando>
Message-ID: <3C649111-A44E-46D8-BCCB-76A5BFACFA41@stufft.io>


On Mar 6, 2014, at 7:35 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote:
>> 
>> On 06/03/2014 10:52, Steven D'Aprano wrote:
> 
>>> Or even no zero point at all. Tossing out a wild idea, why not leave
>>> datetime.time alone and add a new class which is exactly the same but is
>>> always truthy? That doesn't break backward compatibility, and so it
>>> could happen immediately (too late for 3.4, but certainly for 3.5).
>>> 
>>> All we need is an appropriate name, and a decision as to which ought to
>>> inherit from which. The bike-shedding should be over by 3.5 alpha 1
>>> *wink*
>>> 
>>> Despite my quip about the bike-shedding, I am actually serious.
> 
>> You're joking, right?
> 
> What part of "I am actually serious" suggests to you that I am joking? 
> No, I am not joking.
> 
> When you have a misdesigned API that cannot be fixed without breaking 
> backwards compatibility, or at least cannot be fixed *quickly*, it may 
> be sensible to leave the old API in place and place a new one 
> side-by-side. If the old one is actively harmful, it can even be 
> deprecated for eventual removal, otherwise it can remain indefinitely. 
> Does the term "legacy API" sound familiar?
> 
> This is *extremely common* in the software industry, at least in that 
> part that isn't run by cowboy coders who have no concern about breaking 
> their users' code. It's quite common in Python too: consider the legacy 
> API of os.system, the deprecated API of the popen2 module, and the 
> modern subprocess API.
> 
> 
>> Suppose you were considering taking up a new language and you found 
>> something in the docs which said:
>> "Here's a class that does so-and-so.  And here's another slightly 
>> different one that you can use instead, because we didn't get the first 
>> one quite right."
>> Would that be a turn-on?
> 
> Yes. This would tell me that the developers of this language took 
> backward compatibility seriously, and that they weren't afraid to admit 
> to mistakes.
> 
> 
> 
> -- 
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

I think the decision to deprecate and change in place or create a new API
depends greatly on how much code is going to break as a result of the
deprecation, how hard it will be to restore the previous behavior, and how
different the new API is. Language designers have to both consider backwards
compatibility and the cognitive burden of having two (or more!) ways of doing
the same thing. If you don't strike this balance and you never break backwards
compatibility you very quickly end up with a massive number of new APIs.
Remember that every bug fix is a potential backwards compatibility break for
someone (https://xkcd.com/1172/ is relevant).

In the case of datetime.time()?s boolean evaluation, I don?t believe a new
API is warranted. I don?t believe (but I do not have proof) that there is
going to be a lot of code out that that is depending on datetime.time
evaluating to False in the cases it does (midnight in naive times, midnight utc
when the utc offset is 0 or positive in aware times). I believe it is quite
simple to replace the checks with a more explicit check when the user has been
warned through the deprecation process. Finally I believe that the difference
between the two APIs would be so slight as to increase the cognitive overhead
for very little benefit.

If we were talking about a major shift in how datetime.time() functions then
maybe that would be warranted, however this is a small usability fix in an
obscure corner of the API. It's not hardly, in my opinion, worth an entire
new API revision.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/340e4de0/attachment.sig>

From stephen at xemacs.org  Thu Mar  6 13:46:55 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 21:46:55 +0900
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305135646.GB28804@ando>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
Message-ID: <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > The problem is that fractions can be unbounded in memory,

Seminumerical Algorithms has a few problems on "floating slash", ie a
representation of numbers where the total memory allocated to a
fraction is fixed, but the amounts allocated to numerator and
denominator are variable.  I don't know if it has ever been tried in
practice, though.  And of course any reasonable fixed size would have
very limited ability to express very large or very small magnitudes.


From oscar.j.benjamin at gmail.com  Thu Mar  6 13:51:01 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 6 Mar 2014 12:51:01 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <loom.20140306T131407-724@post.gmane.org>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <loom.20140306T131407-724@post.gmane.org>
Message-ID: <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com>

On 6 March 2014 12:36, Stefan Krah <stefan at bytereef.org> wrote:
>
> Regarding decimal literals:
>
> Possible in 3.x, but it would require some investigation if
> people really want arbitrary precision arithmetic by default
> rather than e.g. IEEE Decimal64.

Interesting. I wasn't aware of Decimal64.

I don't understand your question/point though. The Decimal module
doesn't provide "arbitrary" precision arithmetic by default. It
provides 28 digit precision arithmetic by default. It does however
allow the creation of Decimal instances with effectively arbitrary
precision from strings, integers etc.

So I would assume that the idea was that nothing would change about
decimal arithmetic (by default or when importing the module). The
difference would just be that you could write 1.12345e-23d which would
be equivalent to Decimal('1.12345e-23').

In this way it would be possible with a literal to express any decimal
value exactly (even if it exceeds the arithmetic precision). It would
be possible to do e.g. "if x <= 0.00001d:" and know that the test was
exact.

> Regarding decimal floats by default:
>
> Perhaps in 4.0, but IMO only with *full* backing by the large SciPy
> and NumPy communities. My guess is that it's not going to happen, not
> in the least because the literature for floating point algorithms
> (with proofs!) has an overwhelming focus on binary floating point.

Yes, it's very hard to gauge the effect of something like this.
Understanding where in a numeric code base 64-bit binary floating
point is assumed would be harder than disentangling unicode, from
bytes, from 8-bit encodings I expect.


Oscar

From stephen at xemacs.org  Thu Mar  6 14:02:21 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 22:02:21 +0900
Subject: [Python-ideas] Another way to avoid clumsy lambdas,
 while adding new functionality
In-Reply-To: <53180C92.7070904@canterbury.ac.nz>
References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com>
 <5317B1D7.4090305@canterbury.ac.nz>
 <5317CF11.20100@mrabarnett.plus.com>
 <53180C92.7070904@canterbury.ac.nz>
Message-ID: <87pplzpjj6.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > MRAB wrote:
 > 
 > > In the UK we talk about "Bills" being put before Parliament.
 > 
 > In NZ too, but once it's passed, it's the law --
 > it's not up to individual interpretation!

NZ is a common law country, is it not?


From mal at egenix.com  Thu Mar  6 14:21:52 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 06 Mar 2014 14:21:52 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org>	<53184EE4.80105@egenix.com>
 <lf9jl7$ole$3@ger.gmane.org>	<53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
Message-ID: <53187670.6070302@egenix.com>

On 06.03.2014 13:08, Nick Coghlan wrote:
> But perhaps I've been told "midnight is false in boolean context". But
> which midnight? There are three that apply to me:
> 
>>>> naivemidnight
> datetime.time(0, 0)
>>>> utcmidnight
> datetime.time(0, 0, tzinfo=datetime.timezone.utc)
>>>> localmidnight
> datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000)))
> 
> Are they all False? No, no they're not (unless your local timezone is UTC):
> 
>>>> bool(utcmidnight)
> False
>>>> bool(naivemidnight)
> False
>>>> bool(localmidnight)
> True

Now this is a what I consider a valid argument for making a change.

Thanks, Nick.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From rob.cliffe at btinternet.com  Thu Mar  6 14:47:05 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 06 Mar 2014 13:47:05 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140306123520.GL28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando>
 <53185EA5.3060802@btinternet.com> <20140306123520.GL28804@ando>
Message-ID: <53187C59.4080903@btinternet.com>


On 06/03/2014 12:35, Steven D'Aprano wrote:
> On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote:
>> On 06/03/2014 10:52, Steven D'Aprano wrote:
>>> Or even no zero point at all. Tossing out a wild idea, why not leave
>>> datetime.time alone and add a new class which is exactly the same but is
>>> always truthy? That doesn't break backward compatibility, and so it
>>> could happen immediately (too late for 3.4, but certainly for 3.5).
>>>
>>> All we need is an appropriate name, and a decision as to which ought to
>>> inherit from which. The bike-shedding should be over by 3.5 alpha 1
>>> *wink*
>>>
>>> Despite my quip about the bike-shedding, I am actually serious.
>> You're joking, right?
> What part of "I am actually serious" suggests to you that I am joking?
> No, I am not joking.
>
> When you have a misdesigned API that cannot be fixed without breaking
> backwards compatibility, or at least cannot be fixed *quickly*, it may
> be sensible to leave the old API in place and place a new one
> side-by-side. If the old one is actively harmful, it can even be
> deprecated for eventual removal, otherwise it can remain indefinitely.
> Does the term "legacy API" sound familiar?
>
> This is *extremely common* in the software industry, at least in that
> part that isn't run by cowboy coders who have no concern about breaking
> their users' code. It's quite common in Python too: consider the legacy
> API of os.system, the deprecated API of the popen2 module, and the
> modern subprocess API.
>
>
>> Suppose you were considering taking up a new language and you found
>> something in the docs which said:
>> "Here's a class that does so-and-so.  And here's another slightly
>> different one that you can use instead, because we didn't get the first
>> one quite right."
>> Would that be a turn-on?
> Yes. This would tell me that the developers of this language took
> backward compatibility seriously, and that they weren't afraid to admit
> to mistakes.
>
>
Fair point.  (It just feels untidy to me!)  At least having two classes 
would highlight the difference in behaviour.
Which makes me think, maybe the right thing to do now is to draw more 
attention to the behaviour in the docs.
Section 8.1.5 time Objects currently states

"in Boolean contexts, a time 
<http://docs.python.org/2/library/datetime.html#datetime.time> object is 
considered to be true if and only if, after converting it to minutes and 
subtracting utcoffset() (or 0 if that's None), the result is non-zero."

This could be expanded, using the sort of examples Nick Coghlan 
produced, and a warning added that using bool on time objects is 
discouraged.

And section 3.1 Truth Value Testing which claims to explicitly list all 
the falsey objects includes

"instances of user-defined classes, if the class defines a __nonzero__() 
or __len__() method, when that method returns the integer zero or bool 
value |False|"

I think this could be reworded.  "user-defined classes" does not suggest 
to me that it includes classes in the stdlib.  Would it be going too far 
to include in this section a short warning that using bool (explicitly 
or implicitly) on time objects is discouraged, or at least may not 
behave as expected?

Rob Cliffe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/569a6541/attachment.html>

From donald at stufft.io  Thu Mar  6 14:49:29 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 08:49:29 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53187670.6070302@egenix.com>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>	<1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>	<5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org>	<53184EE4.80105@egenix.com>
 <lf9jl7$ole$3@ger.gmane.org>	<53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
Message-ID: <1B2F38D1-F5D0-4CD7-AC2B-CD51C2E3DA6A@stufft.io>


On Mar 6, 2014, at 8:21 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 06.03.2014 13:08, Nick Coghlan wrote:
>> But perhaps I've been told "midnight is false in boolean context". But
>> which midnight? There are three that apply to me:
>> 
>>>>> naivemidnight
>> datetime.time(0, 0)
>>>>> utcmidnight
>> datetime.time(0, 0, tzinfo=datetime.timezone.utc)
>>>>> localmidnight
>> datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000)))
>> 
>> Are they all False? No, no they're not (unless your local timezone is UTC):
>> 
>>>>> bool(utcmidnight)
>> False
>>>>> bool(naivemidnight)
>> False
>>>>> bool(localmidnight)
>> True
> 
> Now this is a what I consider a valid argument for making a change.
> 
> Thanks, Nick.
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Yea Nick put into words what I?ve been attempting to say better than I ever could have :)

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ad2ed353/attachment-0001.sig>

From rosuav at gmail.com  Thu Mar  6 15:04:01 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 01:04:01 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53185EA5.3060802@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz>
 <20140306105253.GI28804@ando> <53185EA5.3060802@btinternet.com>
Message-ID: <CAPTjJmrk=UyRdSOrBiQN-aqPEL4WxfFs9Cdon+imOmNxMz9gvg@mail.gmail.com>

On Thu, Mar 6, 2014 at 10:40 PM, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> Suppose you were considering taking up a new language and you found
> something in the docs which said:
> "Here's a class that does so-and-so.  And here's another slightly different
> one that you can use instead, because we didn't get the first one quite
> right."
> Would that be a turn-on?

If the first one is marked deprecated, then I'd just ignore that one.
The only (minor) downside is remembering which one's the deprecated
one, so I avoid wasting time going to its docs and then going to the
other. Compare:

http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Sql/postgres.html
http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Sql/pgsql.html

In new code, which of those would you use? Is it sufficiently clear? I
think it is.

With Python's time types, though, there may be links with datetime.
Which type does datetime.time() return? Will there be two parallel
methods to resolve that? It's slightly more complicated. Still, it's
not a killer. I've worked with far messier APIs - look at PHP and
MySQL and the "real" functions. Which ones do you need _real_ in and
which not?

ChrisA

From stephen at xemacs.org  Thu Mar  6 15:33:10 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Mar 2014 23:33:10 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <53187670.6070302@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
Message-ID: <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>

M.-A. Lemburg writes:

 > > Are they all False? No, no they're not (unless your local timezone is UTC):
 > > 
 > >>>> bool(utcmidnight)
 > > False
 > >>>> bool(naivemidnight)
 > > False
 > >>>> bool(localmidnight)
 > > True
 > 
 > Now this is a what I consider a valid argument for making a change.
 > 
 > Thanks, Nick.

I don't understand this turnabout.  After all, time with time zones is
the temporal equivalent of ISO 2022.  It's just madness encapsulated
in a formal data type definition.[1]  Nobody should expect to get sane
results from manipulating that stuff, only hope to get lucky.

The only argument I've seen that corresponds to sane programming
practice is Nick's statement that in Python, None, zeros, and empty
containers are naturally false-ys, and all other objects are expected
to be truth-ys.  I find *that* compelling, even though it is backward
incompatible with the documented behavior of time.time, and would not
affect code using time.time objects that was written in good style.

But I still don't like the way actually making the change would
validate using "if var:" where "if var is not None:" is appropriate.
I don't care if the change is made, but I hope it will be made
absolutely clear that it's purely to work around *old* bugs that may
never get fixed, and that "if var:" which is actually testing for
identity with a false-y sentinel is still strongly discouraged.

Footnotes: 
[1]  Time counted in seconds from an epoch is hard enough, just Google
for "monotonic clock". :-(


From solipsis at pitrou.net  Thu Mar  6 15:39:38 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 06 Mar 2014 15:39:38 +0100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <lfa1as$8bm$1@ger.gmane.org>

Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit :
> I don't care if the change is made, but I hope it will be made
> absolutely clear that it's purely to work around *old* bugs that may
> never get fixed, and that "if var:" which is actually testing for
> identity with a false-y sentinel is still strongly discouraged.

Why "strongly discouraged"? This is more of a style issue than anything 
else?

Regards

Antoine.



From alexander.belopolsky at gmail.com  Thu Mar  6 16:16:44 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 10:16:44 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9j8a$ole$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io>
 <5317AE64.90007@egenix.com>
 <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io>
 <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com>
 <lf9j8a$ole$1@ger.gmane.org>
Message-ID: <CAP7h-xYRr9h5FeioXc1_+3cRV_Yt5cvQZC_454ZdOvfF+TAetQ@mail.gmail.com>

On Thu, Mar 6, 2014 at 5:39 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> > I think proponents of bool(time(0)) == True view it as a container of
> > hours, minutes and seconds.
>
> No, they view it as an abstraction around the common concept of time.
>

OK, it looks like we are getting closer to the root of the disagreement.

We are talking about the datetime.time type here.  It is not "an
abstraction around
the common concept of time."  That role belongs to the datetime.datetime
type.

The datetime.time type is an implementation of the concept of the *time of
day*.

In other words, for a datetime instance dt, dt.time() is the fractional
part of dt
and dt.date() is the integral part.

A test for dt.time() is just as natural for datetime as a test for t %
(24*60*60)
for posix time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/fcdd679d/attachment.html>

From stefan at bytereef.org  Thu Mar  6 16:49:04 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 6 Mar 2014 15:49:04 +0000 (UTC)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <loom.20140306T131407-724@post.gmane.org>
 <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com>
Message-ID: <loom.20140306T163813-515@post.gmane.org>

Oscar Benjamin <oscar.j.benjamin at ...> writes:
> On 6 March 2014 12:36, Stefan Krah <stefan at ...> wrote:
> > Regarding decimal literals:
> >
> > Possible in 3.x, but it would require some investigation if
> > people really want arbitrary precision arithmetic by default
> > rather than e.g. IEEE Decimal64.
> 
> Interesting. I wasn't aware of Decimal64.
> 
> I don't understand your question/point though. The Decimal module
> doesn't provide "arbitrary" precision arithmetic by default. It
> provides 28 digit precision arithmetic by default. It does however
> allow the creation of Decimal instances with effectively arbitrary
> precision from strings, integers etc.

Well, yes. You can also emulate Decimal64 with the appropriate
context parameters.

I meant "arbitrary precision facilities by setting context parameters",
but probably many people want to keep those.

Due to the compact representation (bid64 or bid128), Decimal64 or
Decimal128 would be interesting for doing arithmetic on huge matrices. 


Stefan Krah





From alexander.belopolsky at gmail.com  Thu Mar  6 17:13:57 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 11:13:57 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53187670.6070302@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
Message-ID: <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com>

On Thu, Mar 6, 2014 at 8:21 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> > Are they all False? No, no they're not (unless your local timezone is
> UTC):
> >
> >>>> bool(utcmidnight)
> > False
> >>>> bool(naivemidnight)
> > False
> >>>> bool(localmidnight)
> > True
>
> Now this is a what I consider a valid argument for making a change.


FWIW, I would be +0 for making a change so that t.tzinfo is not None
implies bool(t.timetz()) is True.

My intuition goes like this.  Given

>>> from datetime import *
>>> t1 = datetime(2014, 1, 1)
>>> t2 = datetime(2014, 1, 1, 12)
>>> t3 = datetime(2014, 1, 1, tzinfo=timezone.utc)

Instance t1 has no time information, so t1.time() is false-y.  Instance t2
has time information, so t2.time() is true-y.  Instance t3 is such that
t3.time() == t1.time(), but t3.timetz() has an additional information

>>> t3.timetz()
datetime.time(0, 0, tzinfo=datetime.timezone.utc)
>>> t3.time()
datetime.time(0, 0)

So t3.timetz() is not the same object as returned by the constructor with
no arguments - time().
This is reason enough to make it true-y.

This also does not affect my use-case because t3.time() is still false-y in
this logic:

>>> [bool(t) for t in [t1.time(), t2.time(), t3.time(), time()]]
[False, True, False, False]

Which is consistent with

>>> t3.time() == t1.time() == time()
True
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/9f1b8972/attachment-0001.html>

From alexander.belopolsky at gmail.com  Thu Mar  6 17:36:11 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 11:36:11 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
Message-ID: <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>

On Thu, Mar 6, 2014 at 12:53 AM, Tim Peters <tim.peters at gmail.com> wrote:

> [Bruce Leban <bruce at leapyear.org>]
> > ...
> > Just because you wrote the docs doesn't mean you know what they mean to
> > other readers. The point of documentation is to explain it to someone who
> > doesn't know what it does after all.
>
> Of course.  What's your point here?
>

I am with Bruce here.  I tried to digest your notes on timezone algebra [1]
for years
and I still don't understand it.

[1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/2e8975aa/attachment.html>

From oscar.j.benjamin at gmail.com  Thu Mar  6 17:41:34 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 6 Mar 2014 16:41:34 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <loom.20140306T163813-515@post.gmane.org>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <loom.20140306T131407-724@post.gmane.org>
 <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com>
 <loom.20140306T163813-515@post.gmane.org>
Message-ID: <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com>

On 6 March 2014 15:49, Stefan Krah <stefan at bytereef.org> wrote:
> Oscar Benjamin <oscar.j.benjamin at ...> writes:
>> On 6 March 2014 12:36, Stefan Krah <stefan at ...> wrote:
>> > Regarding decimal literals:
>> >
>> > Possible in 3.x, but it would require some investigation if
>> > people really want arbitrary precision arithmetic by default
>> > rather than e.g. IEEE Decimal64.
>>
>> Interesting. I wasn't aware of Decimal64.
>>
>> I don't understand your question/point though. The Decimal module
>> doesn't provide "arbitrary" precision arithmetic by default. It
>> provides 28 digit precision arithmetic by default. It does however
>> allow the creation of Decimal instances with effectively arbitrary
>> precision from strings, integers etc.
>
> Well, yes. You can also emulate Decimal64 with the appropriate
> context parameters.
>
> I meant "arbitrary precision facilities by setting context parameters",
> but probably many people want to keep those.
>
> Due to the compact representation (bid64 or bid128), Decimal64 or
> Decimal128 would be interesting for doing arithmetic on huge matrices.

I'm still not totally clear here.

Do you mean that decimals created with a decimal literal e.g. 1.123d
would be some other kind of decimal object that always used a special
arithmetic context so that they behaved like a fixed width Decimal64
type? And then someone who wants to do decimal arithmetic with other
contexts would still need to import decimal and do the context stuff
themselves?


Oscar

From tim.peters at gmail.com  Thu Mar  6 17:50:53 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 10:50:53 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
Message-ID: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>

[Bruce Leban <bruce at leapyear.org>]
>>> ...
>>> Just because you wrote the docs doesn't mean you know what they mean to
>>> other readers. The point of documentation is to explain it to someone
>>> who doesn't know what it does after all.

[Tim]
>> Of course.  What's your point here?

[Alexander Belopolsky <alexander.belopolsky at gmail.com>]
> I am with Bruce here.  I tried to digest your notes on timezone algebra [1]
> for years and I still don't understand it.
>
> [1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922

But those aren't user docs - they're notes for developers who are
deeply steeped in internal details.  And they're hard!  Mucking with
"all possible" time zones raises mountains of excruciatingly subtle
details and corner cases.  The masses of tedious "algebra" were needed
to justify how the internals get away with doing "so little" - and to
explain why the code still wouldn't work right in all cases if the
"standard offset" of a single time zone changed (or changes) over
time.

Do you claim you still don't understand when bool(time) returns True
and False?  Those are user docs.  I don't claim that reading them will
make anyone feel good ;-), but I do believe the computation is
described clearly enough to enable a reasonable user to predict the
result in all cases.

From donald at stufft.io  Thu Mar  6 17:53:31 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 11:53:31 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
Message-ID: <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io>


On Mar 6, 2014, at 11:50 AM, Tim Peters <tim.peters at gmail.com> wrote:

> [Bruce Leban <bruce at leapyear.org>]
>>>> ...
>>>> Just because you wrote the docs doesn't mean you know what they mean to
>>>> other readers. The point of documentation is to explain it to someone
>>>> who doesn't know what it does after all.
> 
> [Tim]
>>> Of course.  What's your point here?
> 
> [Alexander Belopolsky <alexander.belopolsky at gmail.com>]
>> I am with Bruce here.  I tried to digest your notes on timezone algebra [1]
>> for years and I still don't understand it.
>> 
>> [1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922
> 
> But those aren't user docs - they're notes for developers who are
> deeply steeped in internal details.  And they're hard!  Mucking with
> "all possible" time zones raises mountains of excruciatingly subtle
> details and corner cases.  The masses of tedious "algebra" were needed
> to justify how the internals get away with doing "so little" - and to
> explain why the code still wouldn't work right in all cases if the
> "standard offset" of a single time zone changed (or changes) over
> time.
> 
> Do you claim you still don't understand when bool(time) returns True
> and False?  Those are user docs.  I don't claim that reading them will
> make anyone feel good ;-), but I do believe the computation is
> described clearly enough to enable a reasonable user to predict the
> result in all cases.

I don?t think a reasonable user would predict that midnight UTC in a timezone
with a UTC offset of +5 would be false, while midnight UTC in a timezone with
a UTC offset of -5 would not be false.

> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ce9216f7/attachment.sig>

From tim.peters at gmail.com  Thu Mar  6 18:01:56 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 11:01:56 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
 <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io>
Message-ID: <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com>

[TIm]
>> ...
>> Do you claim you still don't understand when bool(time) returns True
>> and False?  Those are user docs.  I don't claim that reading them will
>> make anyone feel good ;-), but I do believe the computation is
>> described clearly enough to enable a reasonable user to predict the
>> result in all cases.

[Donald Stufft <donald at stufft.io>]
> I don't think a reasonable user would predict that midnight UTC in a timezone
> with a UTC offset of +5 would be false, while midnight UTC in a timezone with
> a UTC offset of -5 would not be false.

As I said, I didn't claim a reasonable user would feel good about it,
and all you're saying here is that they wouldn't feel good about it.
I'm not arguing that they should feel good about it.

I said that the docs explained the computation clearly enough that a
reasonable user - who read the docs and earnestly tried to apply them
- _could_ predict the results.  And you just showed that you, for
example, can predict them.  Since you're the very definition of
"reasonable", I close my case for that ;-)

From alexander.belopolsky at gmail.com  Thu Mar  6 18:02:36 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 12:02:36 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
Message-ID: <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com>

On Thu, Mar 6, 2014 at 11:50 AM, Tim Peters <tim.peters at gmail.com> wrote:

> Do you claim you still don't understand when bool(time) returns True
> and False?  Those are user docs.  I don't claim that reading them will
> make anyone feel good ;-), but I do believe the computation is
> described clearly enough to enable a reasonable user to predict the
> result in all cases.
>

I have to admit that before this discussion, I did not know about the
following
consequence of the documented behavior:

>>> from datetime import *
>>> t = datetime(2014, 1, 1, tzinfo=timezone.utc)
>>> tz1 = timezone(timedelta(hours=5))
>>> tz2 = timezone(timedelta(hours=-5))
>>> bool(t.timetz())
False
>>> bool(t.astimezone(tz1).timetz())
False
>>> bool(t.astimezone(tz2).timetz())
True
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/78af5167/attachment.html>

From tim.peters at gmail.com  Thu Mar  6 18:05:39 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 11:05:39 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
 <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com>
Message-ID: <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com>

[Alexander Belopolsky <alexander.belopolsky at gmail.com>]
> I have to admit that before this discussion, I did not know about the
> following consequence of the documented behavior:
>
>>>> from datetime import *
>>>> t = datetime(2014, 1, 1, tzinfo=timezone.utc)
>>>> tz1 = timezone(timedelta(hours=5))
>>>> tz2 = timezone(timedelta(hours=-5))
>>>> bool(t.timetz())
> False
>>>> bool(t.astimezone(tz1).timetz())
> False
>>>> bool(t.astimezone(tz2).timetz())
> True

I'm not sure I did either!  An "aware" time object is a bizarre beast,
and I've never used one outside of writing datetime test cases.  I'm
not even sure why aware time objects exist - we'd have to dig into
Guido's subconscious for that one ;-)

From donald at stufft.io  Thu Mar  6 18:10:23 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 12:10:23 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
 <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io>
 <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com>
Message-ID: <B4BDA6D6-6CA3-40A2-A608-BA7EF2F3ADD3@stufft.io>


On Mar 6, 2014, at 12:01 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [TIm]
>>> ...
>>> Do you claim you still don't understand when bool(time) returns True
>>> and False?  Those are user docs.  I don't claim that reading them will
>>> make anyone feel good ;-), but I do believe the computation is
>>> described clearly enough to enable a reasonable user to predict the
>>> result in all cases.
> 
> [Donald Stufft <donald at stufft.io>]
>> I don't think a reasonable user would predict that midnight UTC in a timezone
>> with a UTC offset of +5 would be false, while midnight UTC in a timezone with
>> a UTC offset of -5 would not be false.
> 
> As I said, I didn't claim a reasonable user would feel good about it,
> and all you're saying here is that they wouldn't feel good about it.
> I'm not arguing that they should feel good about it.
> 
> I said that the docs explained the computation clearly enough that a
> reasonable user - who read the docs and earnestly tried to apply them
> - _could_ predict the results.  And you just showed that you, for
> example, can predict them.  Since you're the very definition of
> "reasonable", I close my case for that ;-)

Actually I wasn?t able to predict it :) Someone else posted a number
of times offsets and their boolean values. I was really confused
when it turned out that 01:00:00+01:00 (1AM France/Midnight UTC)
was False but 19:00:00-05:00 (7PM Boston/Midnight UTC) wasn?t. I
only realized that the cause was the specific formula that was
used when I reversed it and tried to figure out what time in Boston
would be False and realized it was -05:00.

I don?t think it?s reasonable by any definition of the word that in order
to predict the results a user would have to turn it into an algebraic
formula and solve for X to determine that the only way for Boston
to have a False time is if time was negative.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ab100030/attachment.sig>

From alexander.belopolsky at gmail.com  Thu Mar  6 18:11:33 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 6 Mar 2014 12:11:33 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051924.05252.shai@platonix.com>
 <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com>
 <201403052044.55720.shai@platonix.com>
 <20140305192419.GD28804@ando>
 <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com>
 <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com>
 <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com>
 <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com>
 <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com>
 <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com>
 <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com>
 <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com>
 <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com>
 <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com>
Message-ID: <CAP7h-xbd7zH_jUGhTccuaptav6BLS9_WGf5p1TfD2TYgzupGJA@mail.gmail.com>

On Thu, Mar 6, 2014 at 12:05 PM, Tim Peters <tim.peters at gmail.com> wrote:

> I'm not sure I did either!  An "aware" time object is a bizarre beast,
> and I've never used one outside of writing datetime test cases.  I'm
> not even sure why aware time objects exist - we'd have to dig into
> Guido's subconscious for that one ;-)
>

They are occasionally useful if you want to split datetimes into date and
time
parts, process those separately and recombine in the end without loosing
tzinfo.

I don't think anything involving time.utcoffset() makes much sense and it
is unfortunate
that time.utcoffset() can be invoked implicitly when aware time is used in
boolean context.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/7cd673a0/attachment.html>

From tim.peters at gmail.com  Thu Mar  6 18:13:36 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 11:13:36 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9gft$jnh$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org>
Message-ID: <CAExdVN=Nhx8V179nwFzw9dFkieSWkf0A9sJgj50or+tBkg_kuw@mail.gmail.com>

[Antoine Pitrou <solipsis at pitrou.net>]
> ...
> It is not a coincidence, IMO, that the three persons arguing that the
> current behaviour is reasonable are implementors of datetime modules: you,
> Tim and Alexander. All other people find the behaviour baffling.

The behavior of "aware" time objects is indeed bizarre, but then aware
time objects are bizarre beasts from the start (a time zone attached
to a mere time-of-day, divorced from date info, doesn't make any real
sense to me).

The reasons I already explained (possible extensions to naive time's
repertoire, in the areas of type conversions and arithmetic) account
for why bool(time()) returns False.  _As things turned out_ no such
extensions were made, so we're left with a foundation sticking out of
the ground with nothing built on top of it.

Yawn ;-)

From songofacandy at gmail.com  Thu Mar  6 18:23:31 2014
From: songofacandy at gmail.com (INADA Naoki)
Date: Fri, 7 Mar 2014 02:23:31 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53184096.3050409@egenix.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> <53184096.3050409@egenix.com>
Message-ID: <CAEfz+TwU6JStj=s2ScN2fq-=ejXLPQnxvBgx3M7akFg2p9rsZA@mail.gmail.com>

I feel zero value of non abelian group should not mean False in bool context.

() + () == ()
"" + "" == ""
0 + 0 == 0
timedelta() + timedelta() == timedelta()
time() + time() => TypeError



On Thu, Mar 6, 2014 at 6:32 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 06.03.2014 09:46, Donald Stufft wrote:
>>
>> On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>>> On 06.03.2014 03:23, Donald Stufft wrote:
>>>>
>>>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote:
>>>>
>>>>> [Donald Stufft]
>>>>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that
>>>>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile.
>>>>>
>>>>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that
>>>>> no code relies on the current behavior? For that matter, where is the evidence that the
>>>>> current behavior is a significant cause of bugs?  I know I've used "if some_time_object:",
>>>>> but it did exactly what I expected it to do.
>>>>
>>>> Forgive me if I'm wrong, but aren't you the author of the date time module? If that's the case
>>>> what you expect it to do isn't particularly relevant as you're intimately aware of it's
>>>> implementation.
>>>
>>> Just in case you're looking for a relevant use case:
>>>
>>> Having bool(time(0,0,0)) allows you to quickly check whether you are
>>> dealing with a datetime value with (non-trivial) time part or not.
>>
>> I don't see how midnight is any more or less trivial than 12:01.
>>
>>>
>>> You run into situations where you have to test for this in
>>> date/time conversions, arithmetics, etc. e.g. if you need to convert
>>> a datetime value to a date and want to prevent truncation of
>>> information, or if you want to format a datetime value in a user
>>> friendly way by omitting the zero time part, or if you're doing
>>> datetime arithmetic that has to follow special requirements w/r to
>>> days (think interest or insurance math) and you want to
>>> quickly check whether you can use the fast path simple
>>> calculation, or you need to do the full blown complicated part.
>>
>> I think these would be better served by actually checking for what you mean.
>> For the record I also think that checking for a date time to not be None
>> is also better served by actually checking for what you mean. The
>> difference is in intent and likelihood of confusion.
>
> FWIW, I expect bool(time(0,0,0)) == False, just like I expect
> bool(0.0) == False. I would find it confusing to have
> bool(zero_element) == True.
>
>> For all of those you're going to be explicitly testing and working with
>> midnight objects, so you're going to test them, you're going to exercise
>> that code. In the ``is None`` case, Midnight is just another value so that
>> bug will lay dormant and undetected until someone just happens to
>> hit an unlucky time. And once you get that it's likely to be difficult to
>> reproduce.
>
> datetime values with zero time are *very* common in practice
> (you run into them whenever you convert a date value into a datetime
> value), so the "is None" case is easy to reproduce and will hit
> you more often than you like.
>
> BTW: Not using "is None" will bite you in many other ways as
> well. None it typically used as value for "not initialized" or
> "no value provided". This meaning is completely different from
> "empty selection" or "empty string", so in order to address all
> those cases as well, you'd have to change:
>
> bool(()) == True
> bool('') == True
> etc.
>
> The "if x is None:" test is also faster than "if x:", so actually
> win something by coding correctly ;-)
>
> Please provide a stronger argument for needing to change
> bool(time(0,0,0)) than one which is built on buggy code :-)
> (I'm running out of smileys...)
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Mar 06 2014)
>>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-04-09: PyCon 2014, Montreal, Canada ...               34 days to go
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
INADA Naoki  <songofacandy at gmail.com>

From tim.peters at gmail.com  Thu Mar  6 18:29:37 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 11:29:37 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53181ADA.2030401@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
Message-ID: <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>

[Tim Peters]
>> For example, modular arithmetic on time
>> objects was a possibility.  In that case, time(0, 0, 0) would become
>> "a zero value"

[Greg Ewing <greg.ewing at canterbury.ac.nz>]
> That only follows if you interpret "modular arithmetic on
> time objects" as meaning that you can directly add one time
> object to another.

Of course.

> But that would be confusing times with timedeltas. The concept of
> a zero *timedelta* makes sense, but not a "zero time".

The only difference between time and timedelta is in implementation
details.  In effect, time values are (or _can_ be viewed as) a subset
of timedelta values, restricted to non-negative durations strictly
less than 24 hours.  From that point of view, it's obvious how to do
modular (mod 24 hours) arithmetic on time values.  And some people
said they wanted that.

It wasn't obviously useful enough to implement at first, but neither
did we want to preclude it in the future.  I don't personally want it.


>> People can say that "midnight" isn't a compelling "sentinel value",
>> and I agree about the "compelling" part, but in all implementation
>> respects it does _act_ as "a sentinel value".

> If someone is using midnight to represent an unspecified
> time, I'd say they have a bug. An appointment scheduled at
> midnight is not the same as an appointment that's not
> scheduled at all.

Again obviously so.  In much the same way, there's nothing forbidden
about -1 as an integer, but it's ubiquitously used as an error return
in CPython's implementation.  That doesn't mean CPython is buggy, it
implies that CPython has other ways to distinguish whether a -1 return
does or does not mean "error!".  For example, in some applications
(not all), "exactly midnight"  simply can't happen as a legitimate
starting or ending time.

From stephen at xemacs.org  Thu Mar  6 18:40:42 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Mar 2014 02:40:42 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <lfa1as$8bm$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
 <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <lfa1as$8bm$1@ger.gmane.org>
Message-ID: <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit :

 > > never get fixed, and that "if var:" which is actually testing for
 > > identity with a false-y sentinel is still strongly discouraged.
 > 
 > Why "strongly discouraged"? This is more of a style issue than anything 
 > else?

No, it's not just a style issue.  As Skip points out, it's a
*semantic* issue as well.  You're really testing for not for the
boolean value of values of one type, but for a difference of type
("initialized whatever" vs "uninitialized variable").  Pragmatically,
this would be half a dozen of one, six of the other, except that there
are two more possibilities: a false-y of a different type, and a
truth-y of a different type.  I don't think the spurious truth-y
matters very much ... you'd expect a TypeError to be raised pretty
quickly, and if not, it's a bad bug (by the time you notice the bad
value, it's really hard to work back to where and why it's been
injected).

The spurious false-y, on the other hand, seems quite likely to lead to
spurious values *of the expected type*.  To restate my earlier example:

    my_time = None

    if not my_time and x:
        my_time = x.expected_to_be_parsed_time_but_parse_bug_returns_0

    if not my_time:
        my_time = time.now()

You may never catch *that* parser bug, because the value of 'my_time'
always looks right (ie, like a datetime.time value).  But in

    my_time = None

    if my_time is None and x:
        my_time = x.expected_to_be_parsed_time_but_parse_bug_returns_0

    if my_time is None:
        my_time = time.now()

you're on the same footing as with the truth-y -- just a bit lucky and
you get a quick TypeError.  This is an objective benefit (although you
can say "the probability that there will be a bug to catch is too low
to care about" if you like, it's obviously bigger than zero).

I also value the psychological benefit of being precise here.  I'm not
even sure it's possible to fool 'is' into thinking some other object
is None, but people do a great job of convincing themselves of things
like "datetime.time objects can't be false" all the time.  Use the
"is None" idiom and you know exactly what you're doing.  This latter
is close to "just style", of course.

Steve



From guido at python.org  Thu Mar  6 18:48:53 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 6 Mar 2014 09:48:53 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
Message-ID: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>

On Thu, Mar 6, 2014 at 12:14 AM, Mark H. Harris <harrismh777 at gmail.com>wrote:

> On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote:
>>
>> Do you actually have a degree in math, or do you just remember your high
>> school algebra?
>>
>
>   hi Guido,  ouch.   Its a story, glad you asked.  My first trip to
> college 1974-1977 [...]
>

I must have hit a nerve -- I wasn't requesting a transcript! :-) A simple
"I dropped out of math in college to pursue a career in programming" would
have been sufficient (my own story is pretty similar :-).


> We might agree to stick with the discussion, if you're are willing, and
> stay away from *ad hominem*
> attacks, nor credential bashing. A person either knows what they are
> doing, or they don't. And if
> they are willing to contribute, why knock it, or them?
>

I was asking because I am having a hard time getting your point, and it
feels like you keep repeating it without clarifying it.


> The numbers in math usually are quite strictly typed: whole theories only
>> apply to integers or even positive integers, other things to all reals but
>> not to complex numbers.
>>
>
> Everyone keeps making the points noted above!  Yes, I know... concur~
>     Guido, all data types in most computer high level languages
> are all very strictly statically bound also.
>

I'm not sure that that description applies to Python; it sure doesn't match
how I *think* about numbers in Python.


> So what?  You know this better
> than anyone, because you have taken so much abuse from trolls about it
> over the
> years.
>

Actually, I haven't -- I can smell a troll a mile away and just mute the
conversation.


> We all know that the best way to handle name binding is dynamically.
>

That's your opinion. There are pros and cons and both static and dynamic
binding have their place.


> And that was a paradigm shift, was it not?
>

I've never claimed such a hyperbole.


> Go take a look at Rexx. It has NO types. None.
> Not at the surface, anyway. Everything to the user of the language is a
> string. This is
> even true for most of "object" Rexx too. *Numbers *are just strings of
> characters that
> are parsed and interpreted by Rexx as valid *Rexx Numbers*.
>

Why do you italicize this?


> It works. There is no
> real point in arguing that its a bad idea, because it served the Rexx
> community for
> many years... even now, Rexx is not dead. Yes, under the covers (not
> magic) a complex
> number is going to be handled differently than a real number.  Yeah, what
> is your point?
> They are handled differently in my TI 89 too... so what,  I change the
> context on the 89
> and now I'm doing complex number math (not often, I might add).
>

By "context", here, do you mean some specific state in the computer, or are
you referring to a different way of thinking about it (i.e. in your head)?


> If I set the context for
> reals, then now I'm doing real math... it really is not that difficult.
>

So I suppose depending on how you set the context, the square root of -1
either returns a complex number, or raises an error? And comparisons are
right out once your context is set for complex numbers? Or are they still
allowed when the imaginary part is exactly zero? (What if it is nearly
zero?)


> Think about this for just
> a minute. I have used complex math exactly three times in my life.  I used
> it in high school
> to study the concept in math analysis. I used it in my EE classes...
> electrical engineers
> get a lot out of complex numbers.  And I used it when I was interested in
> the Mendlebrot
> set many years ago when it was cool to plot the famous fractal on early
> cga screens.
> When was the last time you used complex numbers?  When was the last time a
> bank, or
> a hospital, or Montgomery Wards used complex numbers?  If someone needs
> complex
> numbers, Python could change the context "dynamically" and move through
> the problem set.
> Why should the user need to "manually" change the context if  (AI)  could
> change it for them
> on-the-fly?  Just try to get a feel for the question, and stop with trying
> to beat me up on
> my credentials.
>

I still don't understand the question. Can you be more precise instead of
using more rhetoric? Is your point just that you don't want to have to deal
with complex numbers? That's fine, and it's already (mostly) how Python
works -- you must use a complex literal or the cmath module before any
complex numbers appear (the exception is that x**y returns a complex number
when x is negative and y is not a whole number -- but arguably that's the
best answer).


> (And what about quaternions? Or various infinities? :-)
>>
>
> What about quaternions?  If you extend the complex number system you
> change the context. That's not hard...
>

So, by context you really seem to be referring to some specific state of
the program. Perhaps you are even talking about extending the existing
notion of numeric context used by Python's Decimal type. But it's not very
clear because "you change the context" is also a common term for changing
the topic of a discussion.

You would not expect the context for "reals" processing to be the same for
> the context of
> processing three dimensional space with pairs of complex numbers, would
> you?
>

Actually I'm not sure I *need* a context for this. In Python things like
this are usually just solved through a combination of operator overloading
and dynamic binding, so that e.g. x*y can be a number when x and y are
numbers, but it can mean a vector product of some sort if x and/or y are
vectors.


> Python
> does complex number processing now. But that is a very specialized
> category of use
> case that requires special context and (under the covers) typing relevant
> to complex number
> pairs. The context can change "dynamically" to suit the problem set,
> that's all I'm saying.
>

I'm sorry, but I am *still* unsure about what you mean by "context". Python
does not need a context object to switch between complex and real number
processing -- it's all done through operator overloading.


> I am not saying in all of my dreaming here that typing is not important
> (down under). Python is
> called an object based language, yes?  Not object oriented, right?
>

Actually, modern Python is considered object-oriented. I called it
object-based in a *very* early stage, when there was no class statement in
the language. But IIRC it was added before the first public release (or
soon after), although another round of improvements in this area happened
in the early 2000s (eradicating most of the differences between classes
defined by C and Python code).


> But we all know that down
> under the covers (not magic) python uses Classes and instances of
> classes--objects. We
> don't pretend that what makes object based languages work is magic? do we?
>

"We"?


> Unifying the
> number system on a computer does not have to be grounded in paradigms that
> serviced the
> industry (and academy) for the past 40 years; mostly because memory was
> expensive and
> processing was slow.  I suggest that if memory had been cheap, back in the
> day, and processing
> had been very fast (as it is today) IEEE 754 1985 floats and doubles would
> never have been
> used. We would have used Decimal floating points right from the get-go.
> Initially, that really is
> all I'm asking for on the outset--- lets move to default decimal floating
> point arithmetic for real
> numbers processing.
>

Now you've got a specific proposal and we can discuss pros and cons. This
is a topic that comes up occasionally and it is indeed a pretty interesting
trade-off. I suspect the large (and fast-growing) SciPy community would
prefer to keep binary floating point, because that's what their libraries
written in Fortran or C++ want, and extra binary/decimal conversions at the
boundaries would just slow things down. But I'm really speculating here --
maybe they don't need high-performance conversion between binary and
decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe
it varies per application.


> In the future I am thinking about systems of languages that interact with
> human speech; understanding
> *spoken number* just like they will understand *symbol number.* There is
> really no reason (other
> than paradigm) that this would not be possible either.
>

This is the kind of talk that I wish we could keep out of the python-ideas
list. (Except for the symbolic algebra, for which various third-party
Python libraries already exist.)


> Otherwise, Guido, we might all of us just continue to use C and code up
> our statically bound types by
> hand using nothing but int, long, float, and double.
>
> Its time to innovate.
>

Rhetoric again.


> Kind regards, BDfL,
>
> marcus
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5f15a08a/attachment-0001.html>

From stefan at bytereef.org  Thu Mar  6 19:13:28 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 6 Mar 2014 19:13:28 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <loom.20140306T131407-724@post.gmane.org>
 <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com>
 <loom.20140306T163813-515@post.gmane.org>
 <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com>
Message-ID: <20140306181328.GC31090@sleipnir.bytereef.org>

Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> > Well, yes. You can also emulate Decimal64 with the appropriate
> > context parameters.
> >
> > I meant "arbitrary precision facilities by setting context parameters",
> > but probably many people want to keep those.
> >
> > Due to the compact representation (bid64 or bid128), Decimal64 or
> > Decimal128 would be interesting for doing arithmetic on huge matrices.
> 
> I'm still not totally clear here.
> 
> Do you mean that decimals created with a decimal literal e.g. 1.123d
> would be some other kind of decimal object that always used a special
> arithmetic context so that they behaved like a fixed width Decimal64
> type? And then someone who wants to do decimal arithmetic with other
> contexts would still need to import decimal and do the context stuff
> themselves?

That would be one option. I'm not sure if it is a *good* option.

Another option is to use the regular Decimal objects with the
IEEE context and perhaps add functions to convert to bid64 etc.


Yet another issue is whether a Decimal core type should use the
function names from IEEE 754-2008. The names are different, though
the behavior is largely the same.


My main point here is that even for the Decimal literal (which seems
innocent enough) there is a lot to discuss, since we can specify the
semantics only once.


Stefan Krah




From abarnert at yahoo.com  Thu Mar  6 19:47:41 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 10:47:41 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <loom.20140306T120303-396@post.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <loom.20140306T120303-396@post.gmane.org>
Message-ID: <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com>

On Mar 6, 2014, at 3:13, Stefan Krah <stefan at bytereef.org> wrote:

> Antoine Pitrou <solipsis at ...> writes:
>> It is not a coincidence, IMO, that the three persons arguing that the 
>> current behaviour is reasonable are implementors of datetime modules: 
>> you, Tim and Alexander. All other people find the behaviour baffling.
> 
> Not all. :)  I think viewing midnight as (unix_time % 86400) is
> quite reasonable.

I agree with Stefan here. Treating midnight as zero is not at all baffling. There's a reason modular arithmetic is taught to kids as "clock arithmetic", after all. And it's not unheard of for other languages/libraries/apps to treat midnight as a 0, or even to use it as a null time. Someone else already mentioned that many RDBMS's don't have a separate date type and store dates as datetimes with midnight time. IEEE datetimes also do this--leaving off any trailing component is equivalent to having 0's there.

And this isn't some experts-only thing. Novices regularly ask why we need the date class when a datetime with 0 time does the same thing--often because that's what their first language did, but sometimes just as an a priori expectation. 


On the other hand, the alternative isn't baffling either. Unlike, say, Oracle, Python has separate types for date, time, and datetime, so we don't need to use 0 for "n/a", and it has tz-aware values for all three types, making midnight much less special, and it doesn't have modular arithmetic on times (and timedeltas), etc.

So honestly, if handed the Python time class, I wouldn't know whether to expect midnight to be treated as 0 (and therefore falsey) or not, and would have to read the docs rather than guessing.

When a choice really is arbitrary like this, "it does what it's documented to do" is the only notion of correct that makes sense. 

(As a disclaimer, I have implemented datetime libraries for C and C++, but not for Python.)

From abarnert at yahoo.com  Thu Mar  6 20:01:16 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 11:01:16 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
Message-ID: <5175E062-E30B-4442-828E-4BDA8A01AD9E@yahoo.com>

On Mar 6, 2014, at 4:08, Nick Coghlan <ncoghlan at gmail.com> wrote:

> if x not in (naivemidnight, utcmidnight):
>        # This is the current bool(x)
>        ....
>   if x != naivemidnight:
>        # This has no current shorthand
>       ....
>    if x != utcmidnight:
>        # This has no current shorthand
>       ....
>    if x != localmidnight:
>        # This has no current shorthand
>        ....

This is a great argument. The problem here isn't that we use midnight as 0, but that naive and aware times are the same type and we use UTC midnight as 0 for aware types. The other languages/libraries/apps that treat midnight as 0 are generally either naive-only, or much more explicit about awareness.

Since that isn't going to change in Python, maybe it would be better to provide these values (or functions that return them) as names in the module, and explicitly encourage testing against them in place of bool testing, and define bool testing exactly like this, whether or not it's deprecated?

From tjreedy at udel.edu  Thu Mar  6 20:32:43 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 06 Mar 2014 14:32:43 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf9gft$jnh$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>	<201403051805.15363.shai@platonix.com>	<CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>	<201403051838.25342.shai@platonix.com>	<CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>	<5317BF90.9030901@btinternet.com>	<CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>	<E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>	<CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
Message-ID: <lfaigr$8t4$1@ger.gmane.org>

On 3/6/2014 4:52 AM, Antoine Pitrou wrote:

> It is not a coincidence, IMO, that the three persons arguing that the
> current behaviour is reasonable are implementors of datetime modules:
> you, Tim and Alexander. All other people find the behaviour baffling.

Wrong. I don't. What I find baffling and annoying is the endlessly 
repeated false claim that there is nothing special about midnight, as if 
a thousands repetitions will change the fact. Ditto for the claim that 
there is nothing problematical about using 'a' as a substitute for 'a is 
not None'.

Surely you know that the subset of people who pay attention to an 
endless drumbeat thread like this is severely biased.

-- 
Terry Jan Reedy


From barry at python.org  Thu Mar  6 21:21:36 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Mar 2014 15:21:36 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
 <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <lfa1as$8bm$1@ger.gmane.org>
 <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140306152136.537d79e6@anarchist.wooz.org>

On Mar 07, 2014, at 02:40 AM, Stephen J. Turnbull wrote:

>I also value the psychological benefit of being precise here.  I'm not
>even sure it's possible to fool 'is' into thinking some other object
>is None, but people do a great job of convincing themselves of things
>like "datetime.time objects can't be false" all the time.  Use the
>"is None" idiom and you know exactly what you're doing.  This latter
>is close to "just style", of course.

Not just "you" (i.e. the author of the code), but everyone who comes after you
and reads it.  Being more explicit with identity tests against None assists
future minions in understanding the intent of the code.  When using the
Socratic Method Of Code Reviews, I always ask about whether the author really
expects the 'var' in "if var:" to be any truth-y value.  The answer is usually
"no", in which case I encourage tightening up the conditional.

There are valid cases where a test is really trying to catch any truth-y or
false-y value.  I usually encourage a comment to clarify the author's intent
in those cases.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/e5450795/attachment-0001.sig>

From barry at python.org  Thu Mar  6 21:29:36 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Mar 2014 15:29:36 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
Message-ID: <20140306152936.3c990c46@anarchist.wooz.org>

On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote:

>Instances of existing code that would be broken by the change: 0

You can't know this because you can't audit all the Python code in the world.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/36a8aba3/attachment.sig>

From g.brandl at gmx.net  Thu Mar  6 22:36:23 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 06 Mar 2014 22:36:23 +0100
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
Message-ID: <lfapo9$t5i$1@ger.gmane.org>

Am 06.03.2014 11:11, schrieb Nick Coghlan:
> 
> On 6 Mar 2014 13:55, "Sturla Molden"
> <sturla.molden at gmail.com
> <mailto:sturla.molden at gmail.com>> wrote:
>>
>> "goto fail" is a well-known error handling mechanism in open source software,
> widely reputed for its robusteness:
>>
>>
> http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c
>>
>>
> https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c
>>
>> I believe Python needs to add support for this superior paradigm.
> 
> Unfortunately, we can't throw stones about those, since we currently have
> hostname matching off by default and expect developers to turn it on.

And on another level, we use the same error handling method in C code
(which is not bad per se of course).  The literal code "goto fail;" actually
occurs about 200 times in the CPython sources :)

Aaand... we also don't have a policy to always use braces...

Georg


From kaiser.yann at gmail.com  Thu Mar  6 22:41:56 2014
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Thu, 6 Mar 2014 22:41:56 +0100
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <lfapo9$t5i$1@ger.gmane.org>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
Message-ID: <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>

What does this add that we can't do with control flow tools such as
try..finally or context managers?

On 6 March 2014 22:36, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 06.03.2014 11:11, schrieb Nick Coghlan:
>>
>> On 6 Mar 2014 13:55, "Sturla Molden"
>> <sturla.molden at gmail.com
>> <mailto:sturla.molden at gmail.com>> wrote:
>>>
>>> "goto fail" is a well-known error handling mechanism in open source software,
>> widely reputed for its robusteness:
>>>
>>>
>> http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c
>>>
>>>
>> https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c
>>>
>>> I believe Python needs to add support for this superior paradigm.
>>
>> Unfortunately, we can't throw stones about those, since we currently have
>> hostname matching off by default and expect developers to turn it on.
>
> And on another level, we use the same error handling method in C code
> (which is not bad per se of course).  The literal code "goto fail;" actually
> occurs about 200 times in the CPython sources :)
>
> Aaand... we also don't have a policy to always use braces...
>
> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From ethan at stoneleaf.us  Thu Mar  6 22:35:36 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 06 Mar 2014 13:35:36 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
Message-ID: <5318EA28.3040305@stoneleaf.us>

On 03/05/2014 07:42 AM, Paul Moore wrote:
> On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote:
>>   Let me show you an actual zero:
>>
>>      if event.num_attendants:
>>          prepare_cake()
>>      else:
>>          cancel_event()
>>
>> When no one is coming to your party, is is clearly a different
>> condition than if any number of people are coming to your event.  When
>> you read "if num_attendants", you can clearly tell this is going to do
>> something depending on if people are coming or not.
>
> I'm sorry, are you really trying to say that the above code is better than
>
>      if event.num_attendants != 0:
>          prepare_cake()
>      else:
>          cancel_event()
>
> ?

Yes.  This is Python.  blah blah blah != 0 is unnecessary boiler-plate [1].

--
~Ethan~


[1] In most cases.  I have used `== 0` or `!= 0` when it makes the code clearer, but that's pretty rare.

--
~Ethan~

From harrismh777 at gmail.com  Thu Mar  6 23:23:44 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 14:23:44 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
Message-ID: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>



On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote:
{snip}

 hi Guido,  thanks for your responses, your candor, and your kindness;  I 
think 
dialogue will work, and I promise to keep the rhetoric to a bare minimum, 
if at all.    :)

There really are only two questions I am going to try to answer here, and 
some of 
the answer goes back to Stefan's (and others) questions as well, so I hope 
its helpful.

 

> So, by context you really seem to be referring to some specific state of 
> the program. Perhaps you are even talking about extending the existing 
> notion of numeric context used by Python's Decimal type. But it's not very 
> clear because "you change the context" is also a common term for changing 
> the topic of a discussion.
>

       This is what I mean by "context," for this entire discussion, mostly 
having to do with 
Decimal and the "context manager" and these two lines:

       getcontext()

       with localcontext(ctx=None) as context_manager

       Allow me to back up a bit...  the pdeclib module was not difficult 
for me because of the
mathematics (no brag) I can code transcendental functions in my sleep. The 
hard part of
that project last week was coming to grips with the context manger, and the 
parameterization
of context attributes in the Decimal class. Back in the day when I did this 
work for Rexx at IBM
I used NUMERIC DIGITS and NUMERIC FUZZ  to emulate (in a crude way) what 
Decimal is
doing correctly with its context, with the context manager. I credit Oscar 
Benjamin for helping
me get up to speed with this concept (and I am so glad he did too, because 
its key for this
entire discussion.

       Before we go on, allow me to explain my italics of *Number  *and  
*PythonNumber.*  These are
each an abstract base class.  They are a class that is never instantiated, 
but from which all 
other "numbers" are derived as objects.   (more below) 


> I'm sorry, but I am *still* unsure about what you mean by "context". 
> Python does not need a context object to switch between complex and real 
> number processing -- it's all done through operator overloading.
>

       (more on this below, but I am thinking beyond overloading/ 
 actually, virtual functions and templates)


> Actually, modern Python is considered object-oriented. 
>

       Thank you.  This has everything to do with this discussion, but 
allow me to take a humorous
break and relate an anecdote... I have had numerous discussions with T.R. 
and C.A and S.D. and
others where in the end python was object based.  Ha!   Nope, you have seen 
it right here, the
BDfL says its "object oriented," and that's the end of the argument!   (ok, 
tiny rhetoric)

>
> Unifying the 
>> number system on a computer does not have to be grounded in paradigms 
>> that serviced the 
>> industry (and academy) for the past 40 years;  
>>
>       {snip} 

>
> Now you've got a specific proposal and we can discuss pros and cons. This 
> is a topic that comes up occasionally and it is indeed a pretty interesting 
> trade-off. I suspect the large (and fast-growing) SciPy community would 
> prefer to keep binary floating point, because that's what their libraries 
> written in Fortran or C++ want, and extra binary/decimal conversions at the 
> boundaries would just slow things down. But I'm really speculating here -- 
> maybe they don't need high-performance conversion between binary and 
> decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe 
> it varies per application.
>

       Here is the main response for this discussion, and I will endeavor 
to keep the 
rhetoric down, I promise. 
       This may even be easier to accomplish in 3.x without breaking 
anyone, or any third party software either; depends

       In my view numbers are objects (like circles, squares and triangles; 
in polymorphism discussions). 
Numbers are nothing more than objects which have "contexts" just like 
circles and squares and triangles 
are nothing more than objects that have "contexts". In the following 
discussion think in terms of object
oriented (as in Grady Booch) rather than any specific language--- and 
certainly not python syntax.  um, 
think C++ for now, if you must think of any.  A  *Number  *is an abstract 
base class with "context" attributes
and pure virtual functions--- it will never be instantiated---and  *PythonNumber 
 *will derive from that; also
an abstract base class.   Here is the Class hierarchy I am proposing:

*Number*

       *PythonNumber*

              Decimal

                     Real         <======= this is the  default

              BinaryIEEE85

                     Real_b

               &c (...)   omitted for clarity

               Complex

                       Quaternion

                Rational

                        Fraction

                &c--------------------------------------------------

       Well, there it is.   The idea is that any function or method that 
takes a  *Number* reference
or a  *PythonNumber  *reference does not have to interrogate what type of 
*Number* it is... it just
works (why?) glad you asked, because of polymorphism and a three pointer 
hop down a virtual
function table! 
       If a function or method gets a *Number *parm in as in    method(num) 
   then to get the pretty
print representation of the number  is simply    num.pretty_print(),   or 
to get an evaluation 
of the number is   num.eval(), or to get the string rep of the number is 
 num.str()...  and on and on.
The point is that other parts of the system no longer need to know what 
"type" of number object 
it is,  the right code gets called because of virtual functions, operator 
overloading, and the beautiful
virtual function table (all under the covers, not MAGIC, but polymorphism).
       Python at present is disjointed when it comes to number as a 
concept. In my view numbers 
need to be objects, and the Class hierarchy (see my simplified version 
above) needs to unify numbers
across the python boards (so to speak) so that "types" are only important 
to the implementors, and
so that adding new "types" is easy, concise, and coherent. 
       
       I realize that this suggestion is over the top in terms of size and 
weight (Stefan's concern above). 
But not really, when you think of conceptualizing the unification of 
numbers  under *Number  *and 
*PythonNumber *.  There may be other kinds of numbers too, besides these... 
 like maybe *GmpyNumber*
of *BC_Number*, or others. 

       The bottom line is that we have an object oriented language in front 
of us that has the 
software engineering advantages which would allow this kind of hierarchy 
and conceptualization 
of a unified number system.  Let's leverage our technology for mutual 
advantage across python
communities?  

Thanks for your consideration,

Sincerely,

Kind regards,

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3bbe3c10/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  6 23:38:42 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Mar 2014 11:38:42 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
Message-ID: <5318F8F2.5010209@canterbury.ac.nz>

Tim Peters wrote:
> The only difference between time and timedelta is in implementation
> details.

Only if you define it that way. To my way of thinking,
a time-of-day is like a mod-24-hour datetime, not a
mod-24-hour timedelta.

It seems you're thinking of it as a mod-24-hour timedelta.
Given that, I can see why you think that it's bizarre
to attach a timezone to one. In fact, I'd say if that's
really how a time object is meant to be interpreted,
they shouldn't even be *allowed* to have a timezone,
any more than a timedelta can.

In any case, I don't see how this interpretation
justifies the truthiness behaviour with timezones.
 From my reading of the docs, the *intention* seems
to be for midnight UTC to be false and all other
times to be true. I would never have suspected the
actual behaviour with negative offsets, which looks
like an outright bug to me. Or is there some reason
for it that I'm not seeing?

-- 
Greg

From rosuav at gmail.com  Thu Mar  6 23:41:08 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 09:41:08 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
Message-ID: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>

On Fri, Mar 7, 2014 at 9:23 AM, Mark H. Harris <harrismh777 at gmail.com> wrote:
>        Well, there it is.   The idea is that any function or method that
> takes a  Number reference
> or a  PythonNumber  reference does not have to interrogate what type of
> Number it is... it just
> works (why?) glad you asked, because of polymorphism and a three pointer hop
> down a virtual
> function table!

That's way too concrete for Pythonic style :)

All you need to do is have each numeric type implement the same
operations, and there you are, as the Lord Chancellor said, out of
your difficulty at once!

And that's what we already have. You can add two numbers and they'll
simply add. In fact, a C++ style "hop down a virtual function table"
couldn't handle that. I could make a class hierarchy like you
describe, and have each one have an add() method or operator+(), but
somewhere along the line, something still has to cope with the fact
that it could be given any sort of number - there has to be code
someplace that handles the adding of a (Decimal) Real and a Real_b,
because some day, someone's going to do it. (The correct response
might be to raise TypeError, on the basis that that action merges two
separate inaccuracies. But more likely, the correct response is to
convert one of them to the other type.)

What does your proposed hierarchy offer that numbers.Number doesn't?

ChrisA

From rosuav at gmail.com  Thu Mar  6 23:52:28 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 09:52:28 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5318F8F2.5010209@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
Message-ID: <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 9:38 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> It seems you're thinking of it as a mod-24-hour timedelta.
> Given that, I can see why you think that it's bizarre
> to attach a timezone to one. In fact, I'd say if that's
> really how a time object is meant to be interpreted,
> they shouldn't even be *allowed* to have a timezone,
> any more than a timedelta can.

Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00
America/New_York? If you add a date to it, you could get an aware
datetime that represents an exact instant, and compare them. Can you
compare them, as they are? Is it reliable?

This is nothing to do with timedelta versus datetime. It's to do with
attaching timezone information to a dateless time.

ChrisA

From barry at python.org  Thu Mar  6 23:45:47 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Mar 2014 17:45:47 -0500
Subject: [Python-ideas] Add "goto fail" to Python?
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
Message-ID: <20140306174547.00eca988@anarchist.wooz.org>

On Mar 06, 2014, at 10:36 PM, Georg Brandl wrote:

>Aaand... we also don't have a policy to always use braces...

Yeah, I've been having an interesting conversation on FB about this very thing
(spurred by a code sample of the now-infamous Apple goto fail fail :).

I personally wouldn't be opposed to changing PEP 7 to require it.  But I
remember us at least *discussing* it at the time and explicitly deciding
against requiring those braces.

-Barry

(OTOH, I had to physically torture Emacs to write the Apple goto fail example,
so I guess the answer is to just use a decent text editor.  M-x wink RET).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/dfd3caf1/attachment.sig>

From shai at platonix.com  Thu Mar  6 23:55:29 2014
From: shai at platonix.com (Shai Berger)
Date: Fri, 07 Mar 2014 00:55:29 +0200
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306152136.537d79e6@anarchist.wooz.org>
References: <201403051216.12392.shai@platonix.com>
 <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140306152136.537d79e6@anarchist.wooz.org>
Message-ID: <2750802.jzuIGEshV5@deblack>

On Thursday 06 March 2014 15:21:36 Barry Warsaw wrote:
> On Mar 07, 2014, at 02:40 AM, Stephen J. Turnbull wrote:
> >I also value the psychological benefit of being precise here.  I'm not
> >even sure it's possible to fool 'is' into thinking some other object
> >is None, but people do a great job of convincing themselves of things
> >like "datetime.time objects can't be false" all the time.  Use the
> >"is None" idiom and you know exactly what you're doing.  This latter
> >is close to "just style", of course.
> 
> Not just "you" (i.e. the author of the code), but everyone who comes after
> you and reads it.  Being more explicit with identity tests against None
> assists future minions in understanding the intent of the code.  When using
> the Socratic Method Of Code Reviews, I always ask about whether the author
> really expects the 'var' in "if var:" to be any truth-y value.  The answer
> is usually "no", in which case I encourage tightening up the conditional.
> 

Wait -- you just argued for replacing

	if time_obj:

not by

	if time_obj is not None:

-- which is a more relaxed conditional -- but rather by

	if isinstance(time_obj,  datetime.time):

That seems quite un-pythonic, and *that* - from you - is quite unlikely, so I 
probably misunderstood. Do you mind clarifying?

[A guess: The argument you intended to make was about an "if not var:" test 
changed to "if var is None:". It does not survive inversion of sense well.]

Thanks,
	Shai.


From dholth at gmail.com  Thu Mar  6 23:56:14 2014
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 6 Mar 2014 17:56:14 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5318EA28.3040305@stoneleaf.us>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <5318EA28.3040305@stoneleaf.us>
Message-ID: <CAG8k2+6o7kezaet2m8jLCrPU3jnex2-K4wz=q4AOEyOK27F4Rg@mail.gmail.com>

The obvious solution is just to fix the Python if statement to remove
implicit bool coercion. So they would all evaluate to if isinstance (x,
bool) and x. Problem solved.
On Mar 6, 2014 5:27 PM, "Ethan Furman" <ethan at stoneleaf.us> wrote:

> On 03/05/2014 07:42 AM, Paul Moore wrote:
>
>> On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote:
>>
>>>   Let me show you an actual zero:
>>>
>>>      if event.num_attendants:
>>>          prepare_cake()
>>>      else:
>>>          cancel_event()
>>>
>>> When no one is coming to your party, is is clearly a different
>>> condition than if any number of people are coming to your event.  When
>>> you read "if num_attendants", you can clearly tell this is going to do
>>> something depending on if people are coming or not.
>>>
>>
>> I'm sorry, are you really trying to say that the above code is better than
>>
>>      if event.num_attendants != 0:
>>          prepare_cake()
>>      else:
>>          cancel_event()
>>
>> ?
>>
>
> Yes.  This is Python.  blah blah blah != 0 is unnecessary boiler-plate [1].
>
> --
> ~Ethan~
>
>
> [1] In most cases.  I have used `== 0` or `!= 0` when it makes the code
> clearer, but that's pretty rare.
>
> --
> ~Ethan~
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f135fa49/attachment-0001.html>

From harrismh777 at gmail.com  Thu Mar  6 22:37:31 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 13:37:31 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <loom.20140306T131407-724@post.gmane.org>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <loom.20140306T131407-724@post.gmane.org>
Message-ID: <f5d71e93-c96b-471b-abde-04bec53feef3@googlegroups.com>



On Thursday, March 6, 2014 6:36:46 AM UTC-6, Stefan Krah wrote:
>
>
> As I understand, you are suggesting some kind of number system with 
> symbolic representations that can be converted to decimal if needed. 
>
> The topic is so vast that the only chance of it happening is for 
> someone to provide an implementation on PyPI and see if people 
> like it.  There are packages for symbolic mathematics like sympy, 
> perhaps that is a start. 


       Yes, that is correct (in a way). I will respond to this better when 
I respond to Guido, so be sure to read that post.
 

>
> Regarding decimal literals: 
>
> Possible in 3.x, but it would require some investigation if 
> people really want arbitrary precision arithmetic by default 
> rather than e.g. IEEE Decimal64. 
>

       Good.  Dialogue is a good thing. Looking forward to it. 


Regarding decimal floats by default: 
>
> Perhaps in 4.0, but IMO only with *full* backing by the large SciPy 
> and NumPy communities. My guess is that it's not going to happen, not 
> in the least because the literature for floating point algorithms 
> (with proofs!) has an overwhelming focus on binary floating point. 
>

       I know.  Again, when I respond to Guido I'll address this a bit.

   Thanks Stefan, I appreciate your inputs, and your work... keep it up!

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/d5e20afd/attachment.html>

From greg.ewing at canterbury.ac.nz  Fri Mar  7 00:12:55 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Mar 2014 12:12:55 +1300
Subject: [Python-ideas] Please reconsider the Boolean
	evaluation	of	midnight
In-Reply-To: <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org> <loom.20140306T120303-396@post.gmane.org>
 <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com>
Message-ID: <531900F7.1090101@canterbury.ac.nz>

Andrew Barnert wrote:
> I agree with Stefan here. Treating midnight as zero is not at all baffling.
> There's a reason modular arithmetic is taught to kids as "clock arithmetic",

I still say this is confusing times with timedeltas. It
makes no conceptual sense to add two times of day together.
You can only make it work if you abuse one type to mean
two different things.

We keep datetimes and timedeltas clearly separated. Why
shouldn't we do the same for times of day?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar  7 00:48:06 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Mar 2014 12:48:06 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <lfaigr$8t4$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org>
Message-ID: <53190936.6030608@canterbury.ac.nz>

Terry Reedy wrote:
> What I find baffling and annoying is the endlessly 
> repeated false claim that there is nothing special about midnight,

I think what people are really saying is that it's not
special *enough* to be worth breaking people's intuition
about the truthiness of structured data types in Python.

Anyhow, it seems that the original motivation for the
current truthiness behaviour isn't really "midnight is
special", but rather "zero hours is special".

In other words, if you turn your head sideways and view
a time() object not as a time of day, but as a difference
between two times of day, then a time() representing
midnight morphs into a timedelta of zero hours, and it
kind of sort of makes sense to treat that as false.

If that's the intended interpretation of a time()
object, the docs do a poor job of conveying it. A
time() object is described as representing "a (local)
time of day", not "an amount of time since midnight".
The "local" even suggests that, if some midnight is to
special, it would be *local* midnight -- but this is
not the case for time() objects with a timezone.

It seems to me that those claiming "midnight is special"
are inventing a justification after the fact -- one
that is not supported by the actual behaviour.

Altogether, the whole thing seems to be a mess.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar  7 00:51:50 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Mar 2014 12:51:50 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <20140306152936.3c990c46@anarchist.wooz.org>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org>
Message-ID: <53190A16.4020201@canterbury.ac.nz>

Barry Warsaw wrote:
> On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote:
> 
>>Instances of existing code that would be broken by the change: 0
> 
> You can't know this because you can't audit all the Python code in the world.

I think he means instances that have been pointed out
so far in this discussion.

-- 
Greg

From cs at zip.com.au  Fri Mar  7 00:59:21 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 7 Mar 2014 10:59:21 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140305120410.GY28804@ando>
References: <20140305120410.GY28804@ando>
Message-ID: <20140306235921.GA20568@cskk.homeip.net>

Coming to this very late, but I was interested in the first thread...

On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:
> > I don't agree with Mark's proposal in this thread but I would like to
> > have decimal literals e.g. 1.23d, 
> 
> +1 on that. Although that will depend on how big a slowdown it causes to 
> Python's startup time.

+1 on 123d decimal literals.

> > and I would also use Fraction
> > literals if available e.g. 1/3F.
> 
> Out of curiosity, why F rather than f?

Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats
in play in Python right now.

If ever we get ambiguity on how numbers are done, this may be useful
in the transition or experiment phase.

So of course "F" is reasonable for fractions. But how hard does
that make the grammar? Because the parser now has to grab the entire
"1/3F" to construct the fraction. You can't just stick it in the lexer
at that point.

Aside: Unless you do something obtuse, like make "3F" be a regular
number with a special internal flag which is treated differently
on the right hand side of a division operation: if encoutered,
division now returns a Fraction instead of an int or float; this
feels like something that could easily have nasty side effects.

And, speaking personally, a big -1 on Mark's "abstract all the
numbers". Like other posters, I agree that a number's internal
implementation/representation affects its semantics. You can't just
wave a wand and say "it should all be abstract"; there will be
side-effects.

You can wave a wand and say "from here on in this code, unadorned
numbers make Decimals, not IEEE floats".

I'm +0.5 on something like:

  from __future__ import decimal_floats

to imply making Decimals from unadorned literals. "__future__" may
be the wrong pseudomodule name.

Not least, it makes testing the side-effects much easier because you
can tweak a nice big chunk of code without editing it internally.

> (By the way, I think it is somewhat amusing that Python not only has a 
> built-in complex type, but also *syntax* for creating complex numbers, 
> but no built-in support for exact rationals.)

Harder to parse (see above), smaller use case maybe. It is not a
bad idea, just not yet done...

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

The concept is interesting and well-formed, but in order to earn better than
a 'C,' the idea must be feasible.
      --A Yale University management professor in response to Fred
        Smith's paper proposing reliable overnight delivery service.
       (Smith went on to found  Federal Express Corp.)

From greg.ewing at canterbury.ac.nz  Fri Mar  7 01:04:56 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Mar 2014 13:04:56 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
Message-ID: <53190D28.50205@canterbury.ac.nz>

Chris Angelico wrote:
> Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00
> America/New_York?

If you think of times of day as modular, then it doesn't
make sense to ask whether one is earlier than another.

But to clarify my position: It's okay to attach a
timezone to a *time of day*. It's *not* okay to attach
a timezone to a *difference* between times of day.

This is one of the reasons it's imortant to keep the
two concepts clearly separated, as we do for datetimes
and timedeltas. Thinking that time(0, 0, 0) should be
false seems to be a result of conflating the two.

-- 
Greg

From tim.peters at gmail.com  Fri Mar  7 01:08:38 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 18:08:38 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53190936.6030608@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com>
 <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org>
 <53190936.6030608@canterbury.ac.nz>
Message-ID: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com>

[Greg Ewing]
> ...
> Altogether, the whole thing seems to be a mess.

Although a mess that fits in a thimble that successfully hid in a dark
corner unnoticed for a decade ;-)

From rosuav at gmail.com  Fri Mar  7 01:13:25 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 11:13:25 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53190D28.50205@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
 <53190D28.50205@canterbury.ac.nz>
Message-ID: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>

On Fri, Mar 7, 2014 at 11:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Chris Angelico wrote:
>>
>> Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00
>> America/New_York?
>
>
> If you think of times of day as modular, then it doesn't
> make sense to ask whether one is earlier than another.

It should be possible to ask if they're equal, at least. Are they?

> But to clarify my position: It's okay to attach a
> timezone to a *time of day*. It's *not* okay to attach
> a timezone to a *difference* between times of day.

What operations can you do on a time-of-day-with-timezone? Give me
some examples, and I'll show you, with the above two examples, how
quirky that can be.

ChrisA

From rosuav at gmail.com  Fri Mar  7 01:17:55 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 11:17:55 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140306235921.GA20568@cskk.homeip.net>
References: <20140305120410.GY28804@ando>
 <20140306235921.GA20568@cskk.homeip.net>
Message-ID: <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com>

On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote:
> Aside: Unless you do something obtuse, like make "3F" be a regular
> number with a special internal flag which is treated differently
> on the right hand side of a division operation: if encoutered,
> division now returns a Fraction instead of an int or float; this
> feels like something that could easily have nasty side effects.

Thanks, now I can take the paper bag off my head. I'm not the only one!

I thought exactly what you say here, that 3F would have to be a
magical type of integer. But actually, all it has to be is Fraction(3)
and everything will work perfectly.

>>> 1/Fraction(3)
Fraction(1, 3)

ChrisA

From harrismh777 at gmail.com  Fri Mar  7 01:17:55 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 16:17:55 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
Message-ID: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>



On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote:

What does your proposed hierarchy offer that numbers.Number doesn't? 
>
>  
I just had one more thought along this line;  consider this:

>>> from pdeclib import *
>>> 
>>> s1=sqrt(2.01)
>>> s1
Decimal('1.41774468787578244511883132198668766452744')
>>> s2=sqrt(d(2.01))
>>> s2
Decimal('1.41774468787578252029556185427085779261123')
>>> 
>>> s1**2
Decimal('2.00999999999999978683717927196994423866272')
>>> s2**2
Decimal('2.01000000000000000000000000000000000000000')
>>> 
>>> 

Which one is right,   s1 or s2 ?

Well, clearly s1 is wrong.  And yet, s1 was coded by giving the system a 
normal
human number, at a human level, and very innocently s1 is really badly 
broken
but not immediately noticed until we try to square it...  

s2 on the other hand is correct, but we had to go through some hoops
to make sure that the system received the correct type.  The system should 
be able to make this determination without the user having to determine 
types.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/146e8ea0/attachment.html>

From donald at stufft.io  Fri Mar  7 01:20:48 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 19:20:48 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
 <53190D28.50205@canterbury.ac.nz>
 <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>
Message-ID: <9F5908A8-8B2B-4320-8CEE-B61F99483222@stufft.io>



> On Mar 6, 2014, at 7:13 PM, Chris Angelico <rosuav at gmail.com> wrote:
> 
>> On Fri, Mar 7, 2014 at 11:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Chris Angelico wrote:
>>> 
>>> Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00
>>> America/New_York?
>> 
>> 
>> If you think of times of day as modular, then it doesn't
>> make sense to ask whether one is earlier than another.
> 
> It should be possible to ask if they're equal, at least. Are they?
> 
>> But to clarify my position: It's okay to attach a
>> timezone to a *time of day*. It's *not* okay to attach
>> a timezone to a *difference* between times of day.
> 
> What operations can you do on a time-of-day-with-timezone? Give me
> some examples, and I'll show you, with the above two examples, how
> quirky that can be.
> 

It's not an operation but a time object with a timezone is perfect for representing an event which occurs everyday at 11:30 EST

> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From rosuav at gmail.com  Fri Mar  7 01:29:34 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 11:29:34 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
Message-ID: <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>

On Fri, Mar 7, 2014 at 10:58 AM, Mark H. Harris <harrismh777 at gmail.com> wrote:
> But, it should not do this:
>
>>>> from pdeclib import *
>>>> x=1
>>>> y=Decimal(.1)
>>>> x+y
> Decimal('1.10000000000000000555111512312578270211816')
>>>>
>
> The policy hidden under the abstraction way down in the guts of the Class
> hierarchy should make the decision to do this:
>
>>>> ====== RESTART =====
>>>> from pdeclib import *
>>>> x=1
>>>> y=.1
>>>>
>>>> def add_(x, y):
> return d(x)+d(y)
>
>>>> add_(x, y)
> Decimal('1.1')
>>>>
>
> Please don't pick at this.  The point is not the code or the syntax. The
> point
> is that the policy in a correctly related Class hierarchy can handle
> somebody
> asking the system to add a  1.23b with a 1.23d , and if setup correctly,
> will
> just work.

If you had a literal syntax "0.1d" meaning Decimal("0.1"), this would
be solved. The only reason your code seems to work is that you're
rounding off your binary floats instead of using them with as much
accuracy as they have. You're deceiving yourself that
str(float('0.1')) seems to round-trip, and therefore that's the right
way to get a decimal value from a float. But with floats completely
out of the picture, there's no need to do any of this.

>>> x=1
>>> y=Decimal(".1") # y=.1d
>>> x+y
Decimal('1.1')

So... I'm +1 for adding a literal syntax for decimals. +1 for adding a
stating-the-default for floats (do it straight away, then code that
uses it can be backported all the way even when there's a change of
default). +0.5 for adding a syntax for fractions; support in principle
but the devil's in the details. -1 for trying to unify everything.

ChrisA

From python at mrabarnett.plus.com  Fri Mar  7 01:32:16 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 07 Mar 2014 00:32:16 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140306235921.GA20568@cskk.homeip.net>
References: <20140305120410.GY28804@ando>
 <20140306235921.GA20568@cskk.homeip.net>
Message-ID: <53191390.7050804@mrabarnett.plus.com>

On 2014-03-06 23:59, Cameron Simpson wrote:
> Coming to this very late, but I was interested in the first thread...
>
> On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:
>> > I don't agree with Mark's proposal in this thread but I would like to
>> > have decimal literals e.g. 1.23d,
>>
>> +1 on that. Although that will depend on how big a slowdown it causes to
>> Python's startup time.
>
> +1 on 123d decimal literals.
>
>> > and I would also use Fraction
>> > literals if available e.g. 1/3F.
>>
>> Out of curiosity, why F rather than f?
>
> Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats
> in play in Python right now.
>
> If ever we get ambiguity on how numbers are done, this may be useful
> in the transition or experiment phase.
>
> So of course "F" is reasonable for fractions. But how hard does
> that make the grammar? Because the parser now has to grab the entire
> "1/3F" to construct the fraction. You can't just stick it in the lexer
> at that point.
>
I think it would be confusing if there were "f" for "float" and "F" for
"fraction". How about "r" for "rationals"?

> Aside: Unless you do something obtuse, like make "3F" be a regular
> number with a special internal flag which is treated differently
> on the right hand side of a division operation: if encoutered,
> division now returns a Fraction instead of an int or float; this
> feels like something that could easily have nasty side effects.
>
> And, speaking personally, a big -1 on Mark's "abstract all the
> numbers". Like other posters, I agree that a number's internal
> implementation/representation affects its semantics. You can't just
> wave a wand and say "it should all be abstract"; there will be
> side-effects.
>
> You can wave a wand and say "from here on in this code, unadorned
> numbers make Decimals, not IEEE floats".
>
> I'm +0.5 on something like:
>
>    from __future__ import decimal_floats
>
> to imply making Decimals from unadorned literals. "__future__" may
> be the wrong pseudomodule name.
>
> Not least, it makes testing the side-effects much easier because you
> can tweak a nice big chunk of code without editing it internally.
>
>> (By the way, I think it is somewhat amusing that Python not only has a
>> built-in complex type, but also *syntax* for creating complex numbers,
>> but no built-in support for exact rationals.)
>
<pedantic>Python doesn't have a syntax for creating complex numbers; it
has a syntax for creating imaginary numbers.</pedantic>

> Harder to parse (see above), smaller use case maybe. It is not a
> bad idea, just not yet done...
>


From rosuav at gmail.com  Fri Mar  7 01:38:34 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 11:38:34 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
Message-ID: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>

On Fri, Mar 7, 2014 at 11:17 AM, Mark H. Harris <harrismh777 at gmail.com> wrote:
> Well, clearly s1 is wrong.  And yet, s1 was coded by giving the system a
> normal
> human number, at a human level, and very innocently s1 is really badly
> broken
> but not immediately noticed until we try to square it...
>
> s2 on the other hand is correct, but we had to go through some hoops
> to make sure that the system received the correct type.  The system should
> be able to make this determination without the user having to determine
> types.

Once again, the problem occurs only because you're using a float
literal, and 2.01 can't perfectly round-trip.

>>> Decimal(2.01)
Decimal('2.0099999999999997868371792719699442386627197265625')

That's pretty much the value you had (I have a few more digits there,
the square rooting and squaring probably cost some accuracy), so I'd
say Decimal correctly round-tripped. The problem is the conversion
from string (source code) to float to Decimal. You think that str()ing
a float gives you a better round-trip, but that works only because
you're using relatively small numbers.

Hmm. Is the rounding done by float.__str__() an attractive nuisance?
Would it be better to show exactly what's stored, if only to remove
the temptation to treat str(f) as "more correct" than f?

ChrisA

From rosuav at gmail.com  Fri Mar  7 01:45:33 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 11:45:33 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <53191390.7050804@mrabarnett.plus.com>
References: <20140305120410.GY28804@ando>
 <20140306235921.GA20568@cskk.homeip.net>
 <53191390.7050804@mrabarnett.plus.com>
Message-ID: <CAPTjJmrmDHe740bMS+s+oArnBL7a+zFoP4dnu9Rvri=LEL7wQg@mail.gmail.com>

On Fri, Mar 7, 2014 at 11:32 AM, MRAB <python at mrabarnett.plus.com> wrote:
> I think it would be confusing if there were "f" for "float" and "F" for
> "fraction". How about "r" for "rationals"?

Since the string prefixes are all case insensitive, I'd be very much
surprised if f and F did different things.

> <pedantic>Python doesn't have a syntax for creating complex numbers; it
> has a syntax for creating imaginary numbers.</pedantic>

<pedantic>

>>> type(1j)
<class 'complex'>

</pedantic>

ChrisA

From rob.cliffe at btinternet.com  Fri Mar  7 01:44:00 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Fri, 07 Mar 2014 00:44:00 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <53190A16.4020201@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
Message-ID: <53191650.5020208@btinternet.com>


On 06/03/2014 23:51, Greg Ewing wrote:
> Barry Warsaw wrote:
>> On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote:
>>
>>> Instances of existing code that would be broken by the change: 0
>>
>> You can't know this because you can't audit all the Python code in 
>> the world.
>
> I think he means instances that have been pointed out
> so far in this discussion.
>
Quite.  If you read my post, I was initialising totals.  I invited the 
universe to add to them.


From harrismh777 at gmail.com  Fri Mar  7 00:58:11 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 15:58:11 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
Message-ID: <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>



On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote:
 

>
> What does your proposed hierarchy offer that numbers.Number doesn't? 
>
>
hi Chris,  numbers.Number is the right concept, but not the correct 
relationship...
in sixty's vernacular, its the right hoo hoo but the wrong tah tah...

Here is the numbers.Number hierarchy:

CLASSES
    builtins.object
        Number
            Complex
                Real
                    Rational
                        Integral
    

Compare numbers.Number with my proposal. (I am not wanting to go into 
semantics at
this point, but the numbers.Number model is not unified and has problems. 
 If it were
ok, we would not be having this discussion. )  The age old question, "How's 
that working
for you?"   Well, not too well.  

Chris, its not just the virtual function table, its also operator 
overloading, and its policy 
handled by (AI) that intelligently decides what to do if someone tries to 
add a real_b with
a real_d.   But, it should not do this:

>>> from pdeclib import *
>>> x=1
>>> y=Decimal(.1)
>>> x+y
Decimal('1.10000000000000000555111512312578270211816')
>>> 

The policy hidden under the abstraction way down in the guts of the Class
hierarchy should make the decision to do this:

>>> ====== RESTART =====
>>> from pdeclib import *
>>> x=1
>>> y=.1
>>> 
>>> def add_(x, y):
return d(x)+d(y)

>>> add_(x, y)
Decimal('1.1')
>>> 

Please don't pick at this.  The point is not the code or the syntax. The 
point
is that the policy in a correctly related Class hierarchy can handle 
somebody
asking the system to add a  1.23b with a 1.23d , and if setup correctly, 
will 
just work. 

The idea is to unify numbers, so that efficient intelligent processing can 
occur
without "types" or "limits".

I'll say more later... after we see how folks respond.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/718b3c3d/attachment-0001.html>

From skip at pobox.com  Fri Mar  7 01:48:30 2014
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 6 Mar 2014 18:48:30 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz>
 <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com>
Message-ID: <CANc-5Uxve4i40+hsv2B-SeQBvxUpt7uP=M2=4rj7kRmwMZ6O=Q@mail.gmail.com>

> Although a mess that fits in a thimble that successfully hid in a dark
> corner unnoticed for a decade ;-)

+1 QOTW...

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/8d93a7df/attachment.html>

From anikom15 at gmail.com  Fri Mar  7 02:03:58 2014
From: anikom15 at gmail.com (=?iso-8859-1?Q?Westley_Mart=EDnez?=)
Date: Thu, 6 Mar 2014 17:03:58 -0800
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
 <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>
Message-ID: <003601cf39a1$21aa23b0$64fe6b10$@gmail.com>

Spaghetti?

Seriously though, I don't see how this is any different than a regular goto.
What's forcing users to use this only for errors?  I understand that error-
handling is what goto is most useful for in C, but if Python already has a
simple and robust method for handling errors, what is goto going to add?

> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+anikom15=gmail.com at python.org]
> On Behalf Of Yann Kaiser
> Sent: Thursday, March 06, 2014 1:42 PM
> To: Georg Brandl
> Cc: python-ideas
> Subject: Re: [Python-ideas] Add "goto fail" to Python?
> 
> What does this add that we can't do with control flow tools such as
> try..finally or context managers?
> 
> On 6 March 2014 22:36, Georg Brandl <g.brandl at gmx.net> wrote:
> > Am 06.03.2014 11:11, schrieb Nick Coghlan:
> >>
> >> On 6 Mar 2014 13:55, "Sturla Molden"
> >> <sturla.molden at gmail.com
> >> <mailto:sturla.molden at gmail.com>> wrote:
> >>>
> >>> "goto fail" is a well-known error handling mechanism in open source
> software,
> >> widely reputed for its robusteness:
> >>>
> >>>
> >> http://opensource.apple.com/source/Security/Security-
> 55471/libsecurity_ssl/lib/sslKeyExchange.c
> >>>
> >>>
> >>
> https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c
> 41d358a3b:lib/x509/verify.c
> >>>
> >>> I believe Python needs to add support for this superior paradigm.
> >>
> >> Unfortunately, we can't throw stones about those, since we currently have
> >> hostname matching off by default and expect developers to turn it on.
> >
> > And on another level, we use the same error handling method in C code
> > (which is not bad per se of course).  The literal code "goto fail;" actually
> > occurs about 200 times in the CPython sources :)
> >
> > Aaand... we also don't have a policy to always use braces...
> >
> > Georg
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From steve at pearwood.info  Fri Mar  7 02:12:29 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 12:12:29 +1100
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <003601cf39a1$21aa23b0$64fe6b10$@gmail.com>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
 <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>
 <003601cf39a1$21aa23b0$64fe6b10$@gmail.com>
Message-ID: <20140307011229.GM28804@ando>

Guys,

Sturla's original email was quite subtle, but you may have missed that 
he is proposing this as a joke PEP for April Fools Day, a bit like the 
introduction of GOTO or the retirement of Guido:

https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html

http://legacy.python.org/dev/peps/pep-0401/

As a joke, it pokes fun at Apple's recent "goto fail" security breach. 
Unfortunately the Python C source code apparently uses exactly the same 
"goto fail" idiom, although hopefully not as poorly as Apple did.



-- 
Steven

From tim.peters at gmail.com  Fri Mar  7 02:19:28 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 19:19:28 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5318F8F2.5010209@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
Message-ID: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>

[Tim]
>> The only difference between time and timedelta is in implementation
>> details.

[Greg]
> Only if you define it that way.

Sorry, don't know what you mean there.  But then I don't draw much
distinction between "defining" something and "implementation details".
 `time` and `timedelta` objects are both exact values with microsecond
resolution; the operations each supports is a matter of definition +
implementation.

> To my way of thinking, a time-of-day is like a mod-24-hour datetime,
> not a mod-24-hour timedelta.

Heh.  I'll ponder that and see if I can make sense of it tomorrow ;-)

> It seems you're thinking of it as a mod-24-hour timedelta.

No.  I'm saying: (1) it _can_ be thought of that way; and, (2) while
that's not how `time` was initially released, there was some effort
not to _preclude_ later extensions viewing it that way.

As is, the datetime module offers no obvious solution to problems like
"the clock reads 6:43 now; what will it read 57 minutes from now?".
`time` objects support no arithmetic of any kind.  The world wouldn't
really collapse if time(6, 43) + time(minute=57), or if time(6, 43) +
timedelta(minutes=57), didn't blow up but returned the obvious result
as a time object.

> Given that, I can see why you think that it's bizarre
> to attach a timezone to one. In fact, I'd say if that's
> really how a time object is meant to be interpreted,
> they shouldn't even be *allowed* to have a timezone,
> any more than a timedelta can.

No, it's bizarre to attach a timezone to a time object because most
tzinfo subclasses don't - and can't - know what to return for the UTC
offset in the absence of - at least - month and day info too.  Hour,
minute, second and microsecond aren't usually enough to determine
whether "daylight time" is in effect, and most tzinfo subclasses do
attempt to model daylight time.  And because daylight time is a
political conceit, most tzinfo subclasses also need year info.  That
has nothing to do with whether times are viewed abstractly,
concretely, etc - it has to do with that time zones generally aren't
_defined_ in terms of isolated time-of-day info.  It's 7:13 PM in US
Central.  Is that daylight (CDT) or standard (CST) time?  It's
impossible to answer.  What's the corresponding UTC time?  Ditto.

While time objects don't support arithmetic they do support all
comparison operators.  And this is another case where the docs explain
_how_ comparison of aware time objects works, but a close reading
reveals that the result generally makes no sense.  In effect, the time
objects are converted to UTC before comparison, but correct conversion
to UTC is generally impossible without month, day and year info too.

A user intimately familiar with how all this works can define tzinfo
subclasses that make time object conversions with the kinds of errors
they prefer. but that too is ... bizarre.

> In any case, I don't see how this interpretation
> justifies the truthiness behaviour with timezones.
> From my reading of the docs, the *intention* seems
> to be for midnight UTC to be false and all other
> times to be true. I would never have suspected the
> actual behaviour with negative offsets, which looks
> like an outright bug to me. Or is there some reason
> for it that I'm not seeing?

Truthiness would become useful if time() (i.e., the naive time object
with all 0 fields) acted as an additive identity in some extension
that allowed time object addition.  That's all.  I put more thought
into writing that sentence than into what truthiness of aware time
objects returned - and I bet it shows ;-)

If more thought had gone into it, and assuming it remained the case
that time objects had to support tzinfo members (despite that doing so
makes little sense), and assuming it remained the case that we wanted
to keep the _possibility_ of later time arithmetic open, I'd amend the
truthiness rules to that False was equivalent to that

    the naive time converted to minutes
    minus
    the UTC offset (or 0 if tzinfo=None)

is 0 mod 24*60.  Which would be a long way of saying "midnight UTC",
but a better way because: (1) again, it's not usually possible to do
_correct_ conversion of a time object to UTC; and, (2) it's not
"midnight" that's important to an additive identity, it's "0".

From donald at stufft.io  Fri Mar  7 02:25:10 2014
From: donald at stufft.io (Donald Stufft)
Date: Thu, 6 Mar 2014 20:25:10 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
Message-ID: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>


On Mar 6, 2014, at 8:19 PM, Tim Peters <tim.peters at gmail.com> wrote:

> No, it's bizarre to attach a timezone to a time object because most
> tzinfo subclasses don't - and can't - know what to return for the UTC
> offset in the absence of - at least - month and day info too.

It?s completely reasonable to attach a timezone to a time object. 

If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then
absolutely that should be represented as a date time. But what if I want
to schedule an event that occurs every day at 11:30AM EST? There?s
no date associated with it (except an infinite set, or the span of my life
I suppose!) it?s just 11:30AM EST.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/80dd4091/attachment-0001.sig>

From kaiser.yann at gmail.com  Fri Mar  7 02:32:10 2014
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Fri, 7 Mar 2014 02:32:10 +0100
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <20140307011229.GM28804@ando>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
 <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>
 <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> <20140307011229.GM28804@ando>
Message-ID: <CANUJvPUsy2khWzAhBX+d2XhABk7z3=L4dVr5opZu6oitjEhisg@mail.gmail.com>

Ah, my bad, I had received this message an hour before midnight in my
timezone so my brain shorted out. Isn't this more of an
insignificant-whitespace issue? We need more significant whitespace.

On 7 March 2014 02:12, Steven D'Aprano <steve at pearwood.info> wrote:
> Guys,
>
> Sturla's original email was quite subtle, but you may have missed that
> he is proposing this as a joke PEP for April Fools Day, a bit like the
> introduction of GOTO or the retirement of Guido:
>
> https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html
>
> http://legacy.python.org/dev/peps/pep-0401/
>
> As a joke, it pokes fun at Apple's recent "goto fail" security breach.
> Unfortunately the Python C source code apparently uses exactly the same
> "goto fail" idiom, although hopefully not as poorly as Apple did.
>
>
>
> --
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From rosuav at gmail.com  Fri Mar  7 02:34:37 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 12:34:37 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
Message-ID: <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>

On Fri, Mar 7, 2014 at 12:25 PM, Donald Stufft <donald at stufft.io> wrote:
> If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then
> absolutely that should be represented as a date time. But what if I want
> to schedule an event that occurs every day at 11:30AM EST? There?s
> no date associated with it (except an infinite set, or the span of my life
> I suppose!) it?s just 11:30AM EST.

Repeating events need their own handling. Maybe it's safe to use
11:30AM local time, but what if you specify 1:30AM America/New_York?
How can you render that onto a calendar? There'll be times when it
breaks - one day a year when it's ambiguous, and one day a year when
it doesn't exist. How can you work with a time that might not exist?

ChrisA

From abarnert at yahoo.com  Fri Mar  7 02:32:45 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 17:32:45 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
Message-ID: <1394155965.8575.YahooMailNeo@web181003.mail.ne1.yahoo.com>

From: Mark H. Harris <harrismh777 at gmail.com>
Sent: Thursday, March 6, 2014 2:23 PM


> Here is the Class hierarchy I am proposing:
>
>Number
>? ? ? ?PythonNumber
>? ? ? ? ? ? ? Decimal
>? ? ? ? ? ? ? ? ? ? ?Real ? ? ? ? <======= this is the ?default
>? ? ? ? ? ? ? BinaryIEEE85
>? ? ? ? ? ? ? ? ? ? ?Real_b
>? ? ? ? ? ? ? ?&c (...) ? omitted for clarity
>? ? ? ? ? ? ? ?Complex
>? ? ? ? ? ? ? ? ? ? ? ?Quaternion
>? ? ? ? ? ? ? ? Rational
>? ? ? ? ? ? ? ? ? ? ? ? Fraction
>? ? ? ? ? ? ? ? &c--------------------------------------------------


This kind of hierarchy doesn't work.?Most importantly,?you can have complex decimal floats and complex binary floats. This is why complex is a class template rather than a class in C++.?Also, this is ignoring the numerical tower?rationals are a specialization of reals, just as floats are.

>? ? ? ?Well, there it is. ? The idea is that any function or method that takes a ?Number?reference
>or a ?PythonNumber ?reference does not have to interrogate what type of Number?it is... it just
>works (why?) glad you asked, because of polymorphism and a three pointer hop down a virtual
>function table!

99% of the time you don't need these abstract base classes at all, because duck typing and operator overloading. But when you need it, it's already there, and does everything you're asking for. See below.


>? ? ? ?If a function or method gets a Number parm in as in ? ?method(num) ? ?then to get the pretty
>print representation of the number ?is simply ? ?num.pretty_print(), ? or to get an evaluation?
>of the number is ? num.eval(), or to get the string rep of the number is ?num.str()...

We already have the first and third as str() and repr(), and I'm not sure what the second is supposed to do. What does evaluating a number return other than the number itself?

>? ? ? ?Python at present is disjointed when it comes to number as a concept. In my view numbers?
>need to be objects, and the Class hierarchy (see my simplified version above) needs to unify numbers
>across the python boards (so to speak) so that "types" are only important to the implementors, and
>so that adding new "types" is easy, concise, and coherent.?

This is all just plainly false. Numbers are objects. There is a hierarchy of abstract base classes for them?done correctly?in the numbers module. And they handle everything you're asking for, except for one thing:?There are no ABCs to distinguish binary vs. decimal floats. But I doubt that's useful for anything. What kind of code can you imagine that does need to distinguish, say, float vs. decimal.Decimal, but doesn't need to distinguish decimal.Decimal from mylib.MyDecimal? They're both inexact floating-point representations of reals.

If the motivation for your whole idea is that you want a class hierarchy like the one in the numbers module but you didn't know that it already existed, then we're done.


From tim.peters at gmail.com  Fri Mar  7 02:36:29 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 19:36:29 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
Message-ID: <CAExdVNkjZB5rwV5c8KruVfV0c==i+uC_LLoFa=wkddWsVf+kBA@mail.gmail.com>

[Donald Stufft]
> It's completely reasonable to attach a timezone to a time object.
>
> If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then
> absolutely that should be represented as a date time. But what if I want
> to schedule an event that occurs every day at 11:30AM EST? There's
> no date associated with it (except an infinite set, or the span of my life
> I suppose!) it's just 11:30AM EST.

That's a stretch, Donald.  Can you present an example of such an
event?  For someone living on the US East Coast, some days that event
would occur when their clock said 11:30AM, and on other days when it
occurred at - figure it out! - 10:30AM or 12:30PM?  Depending on
whether, in the real world, daylight time wasn't or was in effect.

In any case, you keep ignoring the context of what's written to
present some caricature to shoot down.  Here:

[Tim]
> No, it's bizarre to attach a timezone to a time object because most
> tzinfo subclasses don't - and can't - know what to return for the UTC
> offset in the absence of - at least - month and day info too.

You didn't see the "most"?  Or the later repeated elaborations that
most tzinfo subclasses (in real life) _do_ attempt to model daylight
time?  An EST tzinfo subclass is a fixed-offset subclass, making no
attempt to model daylight time.  Those aren't the problem.  But
fixed-offset tzinfo subclasses are relatively rare.

From tim.peters at gmail.com  Fri Mar  7 02:51:48 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 19:51:48 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
 <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>
Message-ID: <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com>

[Donald Stufft]
>> If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then
>> absolutely that should be represented as a date time. But what if I want
>> to schedule an event that occurs every day at 11:30AM EST? There's
>> no date associated with it (except an infinite set, or the span of my life
>> I suppose!) it's just 11:30AM EST.

[Chris Angelico <rosuav at gmail.com>]
> Repeating events need their own handling. Maybe it's safe to use
> 11:30AM local time, but what if you specify 1:30AM America/New_York?
> How can you render that onto a calendar? There'll be times when it
> breaks - one day a year when it's ambiguous, and one day a year when
> it doesn't exist. How can you work with a time that might not exist?

Filling in the blanks, I presume that Donald's "EST" is what the docs
call a "fixed-offset" tzinfo subclass:  it represents a fixed offset
from UTC.  It's what a US East Coast clock would look like if the
idiotic ;-) "daylight saving" rules were repealed, and the East Coast
stayed on "standard" time forever.

Such classes are utterly problem-free.  No ambiguities, no clock
leaping ahead or leaping back, no "missing" hours, no "duplicate"
hours, they just march on forever in perfect lockstep with UTC.

Alas, such classes are also rare.

From abarnert at yahoo.com  Fri Mar  7 02:52:38 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 17:52:38 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
 <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>
Message-ID: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>

From: Chris Angelico <rosuav at gmail.com>

Sent: Thursday, March 6, 2014 4:29 PM


> So... I'm +1 for adding a literal syntax for decimals. +1 for adding a
> stating-the-default for floats (do it straight away, then code that
> uses it can be backported all the way even when there's a change of
> default).

This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more.

Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"?

> +0.5 for adding a syntax for fractions; support in principle
> but the devil's in the details.

What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right?


From breamoreboy at yahoo.co.uk  Fri Mar  7 02:51:04 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 07 Mar 2014 01:51:04 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
Message-ID: <lfb8m3$v6m$2@ger.gmane.org>

On 07/03/2014 01:25, Donald Stufft wrote:
>
> On Mar 6, 2014, at 8:19 PM, Tim Peters <tim.peters at gmail.com> wrote:
>
>> No, it's bizarre to attach a timezone to a time object because most
>> tzinfo subclasses don't - and can't - know what to return for the UTC
>> offset in the absence of - at least - month and day info too.
>
> It?s completely reasonable to attach a timezone to a time object.
>
> If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then
> absolutely that should be represented as a date time. But what if I want
> to schedule an event that occurs every day at 11:30AM EST? There?s
> no date associated with it (except an infinite set, or the span of my life
> I suppose!) it?s just 11:30AM EST.
>
> -----------------
> Donald Stufft

Hum, thinking out loud as I don't know the answer.  Assume that for your 
sins in a past life at 11:30AM EST every day you have to talk to me. How 
do I set my calendar up to have an infinite set of dates, I've never 
known an application let me do this?  How do we sync our calendars so 
that your EST or (I guess) EDT ties in with my GMT or BST?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From guido at python.org  Fri Mar  7 03:09:02 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 6 Mar 2014 18:09:02 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
Message-ID: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>

Mark, it feels like you do not understand Python well enough to be able to
make sweeping proposals about its reform. Hopefully the responses you are
getting will help you make more informed proposals in the future.


On Thu, Mar 6, 2014 at 2:23 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

>
>
> On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote:
> {snip}
>
>  hi Guido,  thanks for your responses, your candor, and your kindness;  I
> think
> dialogue will work, and I promise to keep the rhetoric to a bare minimum,
> if at all.    :)
>
> There really are only two questions I am going to try to answer here, and
> some of
> the answer goes back to Stefan's (and others) questions as well, so I hope
> its helpful.
>
>
>
>> So, by context you really seem to be referring to some specific state of
>> the program. Perhaps you are even talking about extending the existing
>> notion of numeric context used by Python's Decimal type. But it's not very
>> clear because "you change the context" is also a common term for changing
>> the topic of a discussion.
>>
>
>        This is what I mean by "context," for this entire discussion,
> mostly having to do with
> Decimal and the "context manager" and these two lines:
>
>        getcontext()
>
>        with localcontext(ctx=None) as context_manager
>
>        Allow me to back up a bit...  the pdeclib module was not difficult
> for me because of the
> mathematics (no brag) I can code transcendental functions in my sleep. The
> hard part of
> that project last week was coming to grips with the context manger, and
> the parameterization
> of context attributes in the Decimal class. Back in the day when I did
> this work for Rexx at IBM
> I used NUMERIC DIGITS and NUMERIC FUZZ  to emulate (in a crude way) what
> Decimal is
> doing correctly with its context, with the context manager. I credit Oscar
> Benjamin for helping
> me get up to speed with this concept (and I am so glad he did too, because
> its key for this
> entire discussion.
>
>        Before we go on, allow me to explain my italics of *Number  *and
> *PythonNumber.*  These are
> each an abstract base class.  They are a class that is never instantiated,
> but from which all
> other "numbers" are derived as objects.   (more below)
>
>
>> I'm sorry, but I am *still* unsure about what you mean by "context".
>> Python does not need a context object to switch between complex and real
>> number processing -- it's all done through operator overloading.
>>
>
>        (more on this below, but I am thinking beyond overloading/
>  actually, virtual functions and templates)
>
>
>> Actually, modern Python is considered object-oriented.
>>
>
>        Thank you.  This has everything to do with this discussion, but
> allow me to take a humorous
> break and relate an anecdote... I have had numerous discussions with T.R.
> and C.A and S.D. and
> others where in the end python was object based.  Ha!   Nope, you have
> seen it right here, the
> BDfL says its "object oriented," and that's the end of the argument!
> (ok, tiny rhetoric)
>
>>
>> Unifying the
>>> number system on a computer does not have to be grounded in paradigms
>>> that serviced the
>>> industry (and academy) for the past 40 years;
>>>
>>       {snip}
>
>>
>> Now you've got a specific proposal and we can discuss pros and cons. This
>> is a topic that comes up occasionally and it is indeed a pretty interesting
>> trade-off. I suspect the large (and fast-growing) SciPy community would
>> prefer to keep binary floating point, because that's what their libraries
>> written in Fortran or C++ want, and extra binary/decimal conversions at the
>> boundaries would just slow things down. But I'm really speculating here --
>> maybe they don't need high-performance conversion between binary and
>> decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe
>> it varies per application.
>>
>
>        Here is the main response for this discussion, and I will endeavor
> to keep the
> rhetoric down, I promise.
>        This may even be easier to accomplish in 3.x without breaking
> anyone, or any third party software either; depends
>
>        In my view numbers are objects (like circles, squares and
> triangles; in polymorphism discussions).
> Numbers are nothing more than objects which have "contexts" just like
> circles and squares and triangles
> are nothing more than objects that have "contexts". In the following
> discussion think in terms of object
> oriented (as in Grady Booch) rather than any specific language--- and
> certainly not python syntax.  um,
> think C++ for now, if you must think of any.  A  *Number  *is an abstract
> base class with "context" attributes
> and pure virtual functions--- it will never be instantiated---and  *PythonNumber
>  *will derive from that; also
> an abstract base class.   Here is the Class hierarchy I am proposing:
>
> *Number*
>
>        *PythonNumber*
>
>               Decimal
>
>                      Real         <======= this is the  default
>
>               BinaryIEEE85
>
>                      Real_b
>
>                &c (...)   omitted for clarity
>
>                Complex
>
>                        Quaternion
>
>                 Rational
>
>                         Fraction
>
>                 &c--------------------------------------------------
>
>        Well, there it is.   The idea is that any function or method that
> takes a  *Number* reference
> or a  *PythonNumber  *reference does not have to interrogate what type of
> *Number* it is... it just
> works (why?) glad you asked, because of polymorphism and a three pointer
> hop down a virtual
> function table!
>        If a function or method gets a *Number *parm in as in
>  method(num)    then to get the pretty
> print representation of the number  is simply    num.pretty_print(),   or
> to get an evaluation
> of the number is   num.eval(), or to get the string rep of the number is
>  num.str()...  and on and on.
> The point is that other parts of the system no longer need to know what
> "type" of number object
> it is,  the right code gets called because of virtual functions, operator
> overloading, and the beautiful
> virtual function table (all under the covers, not MAGIC, but polymorphism).
>        Python at present is disjointed when it comes to number as a
> concept. In my view numbers
> need to be objects, and the Class hierarchy (see my simplified version
> above) needs to unify numbers
> across the python boards (so to speak) so that "types" are only important
> to the implementors, and
> so that adding new "types" is easy, concise, and coherent.
>
>        I realize that this suggestion is over the top in terms of size and
> weight (Stefan's concern above).
> But not really, when you think of conceptualizing the unification of
> numbers  under *Number  *and
> *PythonNumber *.  There may be other kinds of numbers too, besides
> these...  like maybe *GmpyNumber*
> of *BC_Number*, or others.
>
>        The bottom line is that we have an object oriented language in
> front of us that has the
> software engineering advantages which would allow this kind of hierarchy
> and conceptualization
> of a unified number system.  Let's leverage our technology for mutual
> advantage across python
> communities?
>
> Thanks for your consideration,
>
> Sincerely,
>
> Kind regards,
>
> marcus
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3be4381e/attachment-0001.html>

From abarnert at yahoo.com  Fri Mar  7 03:09:23 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 18:09:23 -0800 (PST)
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
Message-ID: <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>

From: Tim Peters <tim.peters at gmail.com>

Sent: Thursday, March 6, 2014 5:19 PM


> No, it's bizarre to attach a timezone to a time object because most
> tzinfo subclasses don't - and can't - know what to return for the UTC
> offset in the absence of - at least - month and day info too.? Hour,
> minute, second and microsecond aren't usually enough to determine
> whether "daylight time" is in effect, and most tzinfo subclasses do
> attempt to model daylight time.? And because daylight time is a
> political conceit, most tzinfo subclasses also need year info.? That
> has nothing to do with whether times are viewed abstractly,
> concretely, etc - it has to do with that time zones generally aren't
> _defined_ in terms of isolated time-of-day info.? It's 7:13 PM in US
> Central.? Is that daylight (CDT) or standard (CST) time?? It's
> impossible to answer.? What's the corresponding UTC time?? Ditto.


Well, a time in Central is useless, but a time in CDT or CST is not, and you can design a library that's smart enough to give you a CDT or CST (as appropriate) time from a Central datetime.

However, I built something like that in a C++ datetime library, and we never found a use for it anywhere. Instead, we just carried the full datetime or recurring-datetime object around as late as possible, and rendered to naive times for display.

From greg.ewing at canterbury.ac.nz  Fri Mar  7 03:12:36 2014
From: greg.ewing at canterbury.ac.nz (Greg)
Date: Fri, 07 Mar 2014 15:12:36 +1300
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
 <53190D28.50205@canterbury.ac.nz>
 <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>
Message-ID: <53192B14.4070304@canterbury.ac.nz>

On 7/03/2014 1:13 p.m., Chris Angelico wrote:
> What operations can you do on a time-of-day-with-timezone? Give me
> some examples, and I'll show you, with the above two examples, how
> quirky that can be.

I'm not particularly attached to the idea of times of day with
timezones; I wouldn't mind if they didn't exist.

The main point is that they definitely don't make sense for
time differences. An hour in New York is the same length as
an hour in Melbourne, on any day of the year, as far as I know.

(At least in modern times. It seems the Romans divided daytime
and nighttime into 12 equal hours each, so that the length of
an hour varied with latitude and time of year, and daytime
hours were a different length from nighttime hours! I hate to
think what a Python datetime module would have looked like
back then.)

-- 
Greg


From python at mrabarnett.plus.com  Fri Mar  7 03:12:59 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 07 Mar 2014 02:12:59 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
 <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>
 <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>
Message-ID: <53192B2B.1010205@mrabarnett.plus.com>

On 2014-03-07 01:52, Andrew Barnert wrote:
> From: Chris Angelico <rosuav at gmail.com>
>
> Sent: Thursday, March 6, 2014 4:29 PM
>
>
>> So... I'm +1 for adding a literal syntax for decimals. +1 for
>> adding a stating-the-default for floats (do it straight away, then
>> code that uses it can be backported all the way even when there's a
>> change of default).
>
> This makes sense if you also add an optional suffix for binary floats
> at the same time. Otherwise, it would be confusing to talk about the
> "default" when there's no way to make it explicit, and it would delay
> any potential change of the default by at least one version, if not
> more.
>
> Of course there's a lot of room for bikeshedding the suffix. The "f"
> suffix from C implies 32-bit float as opposed to 64-bit double, which
> is obviously wrong. The "d" suffix from C might be confusing as
> distinguishing "binary, not decimal". Maybe "b"?
>
Calling the floating-point class "float" implies 32-bit float as
opposed to 64-bit double, doesn't it? :-)

>> +0.5 for adding a syntax for fractions; support in principle but
>> the devil's in the details.
>
> What details, other than bikeshedding the exact suffix? If 3r ==
> fraction.Fraction(3), we're done, right?
>

From greg.ewing at canterbury.ac.nz  Fri Mar  7 03:16:13 2014
From: greg.ewing at canterbury.ac.nz (Greg)
Date: Fri, 07 Mar 2014 15:16:13 +1300
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com>
References: <20140305120410.GY28804@ando>
 <20140306235921.GA20568@cskk.homeip.net>
 <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com>
Message-ID: <53192BED.6090909@canterbury.ac.nz>

On 7/03/2014 1:17 p.m., Chris Angelico wrote:
> On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote:
>> Aside: Unless you do something obtuse, like make "3F" be a regular
>> number with a special internal flag
>
> I thought exactly what you say here, that 3F would have to be a
> magical type of integer. But actually, all it has to be is Fraction(3)
> and everything will work perfectly.

I think Cameron was talking about using 'F' for both 'binary
floating point' *and* 'fraction', which would indeed lead to
madness.

-- 
Greg


From tim.peters at gmail.com  Fri Mar  7 03:16:53 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 6 Mar 2014 20:16:53 -0600
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
Message-ID: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>

[Tim]
>> No, it's bizarre to attach a timezone to a time object because most
>> tzinfo subclasses don't - and can't - know what to return for the UTC
>> offset in the absence of - at least - month and day info too.  Hour,
>> minute, second and microsecond aren't usually enough to determine
>> whether "daylight time" is in effect, and most tzinfo subclasses do
>> attempt to model daylight time.  And because daylight time is a
>> political conceit, most tzinfo subclasses also need year info.  That
>> has nothing to do with whether times are viewed abstractly,
>> concretely, etc - it has to do with that time zones generally aren't
>> _defined_ in terms of isolated time-of-day info.  It's 7:13 PM in US
>> Central.  Is that daylight (CDT) or standard (CST) time?  It's
>> impossible to answer.  What's the corresponding UTC time?  Ditto.

[Andrew Barnert <abarnert at yahoo.com>]
> Well, a time in Central is useless, but a time in CDT or CST is not, and you can
> design a library that's smart enough to give you a CDT or CST (as appropriate) time
> from a Central datetime.

Tell me how.  You have, in context, a Python `time` object with a
"Central" tzinfo member.  That's all you get from the user.  You're
given absolutely no information about day, month, or year.  What's
your algorithm for implementing picking CDT or CST "as appropriate"?
Note that we're NOT talking about datetime.datetime objects here.
These are datetime.time objects.

This isn't a problem for datetime.datetime objects.  A "Central"
tzinfo timeclass has no problem at all picking CDT or CST as
appropriate given all the info in a datetime.datetime object.

From harrismh777 at gmail.com  Fri Mar  7 03:08:28 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 18:08:28 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
Message-ID: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com>



On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote:

Once again, the problem occurs only because you're using a float 
> literal, and 2.01 can't perfectly round-trip. 
>
> >>> Decimal(2.01) 
> Decimal('2.0099999999999997868371792719699442386627197265625') 
>
> That's pretty much the value you had (I have a few more digits there, 
> the square rooting and squaring probably cost some accuracy), 
>

hi Chris,  yes,  you are completely missing the point. We are talking past
each other.  I  DO NOT need you to explain to me why the is working just
like its supposed to...      its broken and should not work this way !

If you write a number on paper do you write down   '2.01'   ?

Do you write down    d(2.01)  ?

Do you write down    2.01d   ?

(or)   do you write down   2.01     ?

When you punch a number into your TI89  do you punch in  {'} {2} {.} {0} 
{1} {'}   ?

(or)  do you punch in    {2} {.} {0} {1}   ?

When I want a square-root from python I want to enter    s1=sqrt(2.01)   
<======   this is normal
(why?) glad you asked, because that is the human normal thing to do.

Python numbers don't work correctly, because the underlying design is not
correct. Everyone sees it, nobody really likes it, but everyone wants to 
avoid
the problem like the proverbial ostrich (only worse , because this ostrich 
wants me to believe that everything is fine, " Don't pay any attention to 
that 
man behind the current...  I... am the great and powerful OZ"

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/61c380ba/attachment-0001.html>

From harrismh777 at gmail.com  Fri Mar  7 03:19:56 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 18:19:56 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
Message-ID: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>



On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:

Mark, it feels like you do not understand Python well enough to be able to 
> make sweeping proposals about its reform. 
>

hi Guido,  ouch.

   You can't be serious.  You have not commented yet...  are you happy with 
this:

 >>> from decimal import Decimal
>>> a=Decimal(1)
>>> b=Decimal(.1)
>>> a+b
Decimal('1.100000000000000005551115123')      <==== does this not bother 
you at all ?

    ... even though we both know why its doing this, doesn't it bother you 
a little?

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/047747b6/attachment.html>

From rymg19 at gmail.com  Fri Mar  7 03:21:13 2014
From: rymg19 at gmail.com (Ryan)
Date: Thu, 06 Mar 2014 20:21:13 -0600
Subject: [Python-ideas] Add "goto fail" to Python?
In-Reply-To: <20140307011229.GM28804@ando>
References: <lf8rh3$suv$1@ger.gmane.org>
 <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com>
 <lfapo9$t5i$1@ger.gmane.org>
 <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com>
 <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> <20140307011229.GM28804@ando>
Message-ID: <ab60b651-e71c-49bc-a4af-d190c04aaed5@email.android.com>

Wow, those links are hilarious.

Steven D'Aprano <steve at pearwood.info> wrote:
>Guys,
>
>Sturla's original email was quite subtle, but you may have missed that 
>he is proposing this as a joke PEP for April Fools Day, a bit like the 
>introduction of GOTO or the retirement of Guido:
>
>https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html
>
>http://legacy.python.org/dev/peps/pep-0401/
>
>As a joke, it pokes fun at Apple's recent "goto fail" security breach. 
>Unfortunately the Python C source code apparently uses exactly the same
>
>"goto fail" idiom, although hopefully not as poorly as Apple did.
>
>
>
>-- 
>Steven
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/10cf5d6b/attachment.html>

From abarnert at yahoo.com  Fri Mar  7 03:19:40 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 18:19:40 -0800 (PST)
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
 <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>
 <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com>
Message-ID: <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com>

From: Tim Peters <tim.peters at gmail.com>

Sent: Thursday, March 6, 2014 5:51 PM


> Filling in the blanks, I presume that Donald's "EST" is what the 
> docs
> call a "fixed-offset" tzinfo subclass:? it represents a fixed offset
> from UTC.? It's what a US East Coast clock would look like if the
> idiotic ;-) "daylight saving" rules were repealed, and the East Coast
> stayed on "standard" time forever.

Except that it still wouldn't work for dates before 2014. Or, of course, if DST were reinstated in the future.

If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm?


From abarnert at yahoo.com  Fri Mar  7 03:26:29 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 18:26:29 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <53192B2B.1010205@mrabarnett.plus.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
 <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>
 <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>
 <53192B2B.1010205@mrabarnett.plus.com>
Message-ID: <1394159189.4605.YahooMailNeo@web181003.mail.ne1.yahoo.com>

From: MRAB <python at mrabarnett.plus.com>

Sent: Thursday, March 6, 2014 6:12 PM


> On 2014-03-07 01:52, Andrew Barnert wrote:
>>  Of course there's a lot of room for bikeshedding the suffix. The 
>> "f"
>>  suffix from C implies 32-bit float as opposed to 64-bit double, which
>>  is obviously wrong. The "d" suffix from C might be confusing as
>>  distinguishing "binary, not decimal". Maybe "b"?
>> 
> Calling the floating-point class "float" implies 32-bit float as
> opposed to 64-bit double, doesn't it? :-)

Sure. It's not as strong an implication as an "f" suffix, because many languages (and applications) have types named float, while "0.3f" is a lot more C-family-specific. Which means "float" wasn't a perfect choice?but it was still probably the best choice. And it's certainly possible that "f" is the best choice for the suffix despite this problem.?If there were an obvious best answer that had no problems, there wouldn't be any room for bikeshedding.


From abarnert at yahoo.com  Fri Mar  7 03:35:31 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 18:35:31 -0800 (PST)
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
 <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
Message-ID: <1394159731.26118.YahooMailNeo@web181003.mail.ne1.yahoo.com>

From: Tim Peters <tim.peters at gmail.com>

Sent: Thursday, March 6, 2014 6:16 PM


>>  Well, a time in Central is useless, but a time in CDT or CST is not, and 
> you can
>>  design a library that's smart enough to give you a CDT or CST (as 
> appropriate) time
>>  from a Central datetime.
> 
> Tell me how.? You have, in context, a Python `time` object with a
> "Central" tzinfo member.? That's all you get from the user.? 
> You're
> given absolutely no information about day, month, or year.? What's
> your algorithm for implementing picking CDT or CST "as appropriate"?
> Note that we're NOT talking about datetime.datetime objects here.
> These are datetime.time objects.

Of course that's inherently ambiguous and therefore not solvable. If your app wants unambiguous times, it can't allow users to enter times in Central with no additional information.

But that's a separate problem from getting a CDT or CST time from a Central datetime (or from other problems, like handling a Central time that's known to be today). A datetime library could solve _that_ problem.

> This isn't a problem for datetime.datetime objects.? A "Central"
> tzinfo timeclass has no problem at all picking CDT or CST as
> appropriate given all the info in a datetime.datetime object.


Right, but if you get its tztime, you get a Central time, and no longer have the information that lets you use it as CDT or CST as appropriate. That is a problem?but, as I said before, it's probably not a problem in practice, and therefore probably not worth solving. (Again, I did solve it in a different timezone library, and never found a use for that feature.)

From harrismh777 at gmail.com  Fri Mar  7 03:51:18 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 18:51:18 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
Message-ID: <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>



On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:


Mark, it feels like you do not understand Python well enough to be able to 
> make sweeping proposals about its reform. 
>


hi Guido,  I'm sorry, but isn't that the whole point?  The person who 
notices that something
isn't working quite right (that would be me) comes to court, as it were, 
before those who
are supposed to know intimately how everything works (that would be you) 
and in good 
faith asks politely for the "people in the know" to correct the difficulty.

You know what, you're right--- I'm a dumb cluck who doesn't know anything 
about the 
intricacies of python yadda yadda yadda...  but you know what else ? >

I hate this:

>>> from decimal import Decimal
>>> a=Decimal(1)
>>> b=Decimal(.1)
>>> a+b
Decimal('1.100000000000000005551115123')
>>> 

... and I think the people who DO know intimately how python works, should 
do something to fix it.

I'd like to help, if you'd let me.

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5e698c43/attachment.html>

From cs at zip.com.au  Fri Mar  7 03:52:24 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 7 Mar 2014 13:52:24 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <53191390.7050804@mrabarnett.plus.com>
References: <53191390.7050804@mrabarnett.plus.com>
Message-ID: <20140307025224.GA17812@cskk.homeip.net>

On 07Mar2014 00:32, MRAB <python at mrabarnett.plus.com> wrote:
> On 2014-03-06 23:59, Cameron Simpson wrote:
> >Coming to this very late, but I was interested in the first thread...
> >
> >On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote:
> >>On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:
> >>> and I would also use Fraction
> >>> literals if available e.g. 1/3F.
> >>
> >>Out of curiosity, why F rather than f?
> >
> >Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats
> >in play in Python right now. [...]
> >So of course "F" is reasonable for fractions. But how hard does
> >that make the grammar? Because the parser now has to grab the entire
> >"1/3F" to construct the fraction. You can't just stick it in the lexer
> >at that point.
> >
> I think it would be confusing if there were "f" for "float" and "F" for
> "fraction". How about "r" for "rationals"?

I'm good with 'r'.
-- 
Cameron Simpson <cs at zip.com.au>

If you do not read the paper, you are uninformed. If you do read the
paper, you are misinformed. - Mark Twain

From abarnert at yahoo.com  Fri Mar  7 03:53:35 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 18:53:35 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
Message-ID: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>

From: Mark H. Harris <harrismh777 at gmail.com>
Sent: Thursday, March 6, 2014 6:19 PM


[snipping out of order]

>?>>> from decimal import Decimal
>>>> a=Decimal(1)
>>>> b=Decimal(.1)
>>>> a+b
>Decimal('1.100000000000000005551115123') ? ? ?<==== does this not bother you at all ?
>
>? ? ... even though we both know why its doing this, doesn't it bother you a little?


That's not a problem with Python's number system, it's that Decimal(.1) is not the right way to write what you want.

The only solution without changing Python is to train end-users to write something correct, like Decimal('.1').

The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. For example, a decimal suffix, as already proposed before this thread, would completely solve the problem:

? ? >>> a = 1d
? ? >>> b = .1d
? ? >>> a+b
? ? 1.1d

Of course the exact suffix (or other syntax) is up for bikeshedding, as is the possibility of one day changing the default from binary floats to decimal floats, but other than those trivial details, tada.?But there's no need for anything more radical, like some amorphous idea to "unify Python numbers".

>On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:
>
>>Mark, it feels like you do not understand Python well enough to be able to make sweeping proposals about its reform.?
>
>hi Guido, ?ouch.


You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python.

You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem.

You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading.

(And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to.?It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.)

There are some ideas in your posts that are worth responding to, but I think it's perfectly fair for Guido to decide it's not worth digging through the mass of ignorance about Python's basic design to find the nuggets that can be rejected for more specific reasons instead of just dismissed.


From ben+python at benfinney.id.au  Fri Mar  7 04:12:32 2014
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 07 Mar 2014 14:12:32 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
Message-ID: <85k3c6bt27.fsf@benfinney.id.au>

"Mark H. Harris" <harrismh777 at gmail.com>
writes:

> On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:
>
> > Mark, it feels like you do not understand Python well enough to be
> > able to make sweeping proposals about its reform.
>
> hi Guido,  I'm sorry, but isn't that the whole point?  The person who 
> notices that something
> isn't working quite right (that would be me) comes to court, as it were, 
> before those who
> are supposed to know intimately how everything works (that would be you) 
> and in good 
> faith asks politely for the "people in the know" to correct the difficulty.

No, that's not the point of this forum. The point of this forum is to
present ideas for improvement to Python and to *defend* those ideas
robustly against critique, in order to converge on concrete proposals.

To do that, you need to have a pretty good understanding of Python as it
currently is, especially in the domain of the change you're proposing,
in order to robustly defend an idea for improvement.

> I hate this:
[?]

If you want a forum for general discussion where complaints lacking a
good understanding of Python can be bounced around, this isn't it. You
can try the general discussion forum for Python
<URL:https://mail.python.org/mailman/listinfo/python-list>.

But beware that even there, complaints about how Python behaves aren't
going to be well received if you show a wilful ignorance of that
behaviour. Mere ignorance is fine, of course: we all start out ignorant
of any given topic. But wilful ignorance isn't a good foundation for
expecting change. It's better to take the complaint as motivation to
*learn* why Python behaves this way, to see if your assumptions may not
be flawed.

> ... and I think the people who DO know intimately how python works,
> should do something to fix it.

You'll need to form a much more concrete understanding of why it is the
way it is, and exactly what behavioural change you want, and an
understanding of the costs and benefits of that change, before such an
expectation would in any way imply an obligation on others to do what
you want. This forum isn't the place to begin with that.

-- 
 \      ?You say ?Carmina?, and I say ?Burana?, You say ?Fortuna?, and |
  `\    I say ?cantata?, Carmina, Burana, Fortuna, cantata, Let's Carl |
_o__)                                the whole thing Orff.? ?anonymous |
Ben Finney


From cs at zip.com.au  Fri Mar  7 03:58:21 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 7 Mar 2014 13:58:21 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <53192BED.6090909@canterbury.ac.nz>
References: <53192BED.6090909@canterbury.ac.nz>
Message-ID: <20140307025821.GA19433@cskk.homeip.net>

On 07Mar2014 15:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> On 7/03/2014 1:17 p.m., Chris Angelico wrote:
> >On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote:
> >>Aside: Unless you do something obtuse, like make "3F" be a regular
> >>number with a special internal flag
> >
> >I thought exactly what you say here, that 3F would have to be a
> >magical type of integer. But actually, all it has to be is Fraction(3)
> >and everything will work perfectly.
> 
> I think Cameron was talking about using 'F' for both 'binary
> floating point' *and* 'fraction', which would indeed lead to
> madness.

Well, only because I hadn't thought through to having it be a literal
Fraction, which it turns out already has exactly this cool semantic.
-- 
Cameron Simpson <cs at zip.com.au>

The reasonable man adapts himself to the world; the unreasonable one persists
in trying to adapt the world to himself.  Therefore all progress depends
on the unreasonable man.        - George Bernard Shaw

From cs at zip.com.au  Fri Mar  7 04:29:08 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 7 Mar 2014 14:29:08 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com>
References: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com>
Message-ID: <20140307032908.GA46124@cskk.homeip.net>

On 06Mar2014 18:08, Mark H. Harris <harrismh777 at gmail.com> wrote:
> On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote:
> Once again, the problem occurs only because you're using a float 
> > literal, and 2.01 can't perfectly round-trip. 
> >
> > >>> Decimal(2.01) 
> > Decimal('2.0099999999999997868371792719699442386627197265625') 
> >
> > That's pretty much the value you had (I have a few more digits there, 
> > the square rooting and squaring probably cost some accuracy), 
> 
> hi Chris,  yes,  you are completely missing the point. We are talking past
> each other.  I  DO NOT need you to explain to me why the is working just
> like its supposed to...      its broken and should not work this way !
> 
> If you write a number on paper do you write down   '2.01'   ?
> 
> Do you write down    d(2.01)  ?
> 
> Do you write down    2.01d   ?
> 
> (or)   do you write down   2.01     ?
[...]
> When I want a square-root from python I want to enter    s1=sqrt(2.01)   
> <======   this is normal

So essentially you are saying: when I write Python code I do not
want to have to put in all this special notation to get the actual
value which I naturally mean.

And if you write 2.01, you mean 2 + 1/100, without roundoff because
you do not want to be working in a system which would round that
off.  i.e. a Decimal internal representation because you're writing
a number in base 10.

To my mind you want a Python _parser_ mode, because inside python, given
the various numeric classes abounding, the actual _mechanisms_ already exist.

So really you want to have a way of saying to the parser: in the
code, 2.01 is not transformed into an IEEE float with some loss of
precision, it is transformed into a Decimal floating point number,
with no loss of precision.

There are, it seems to me, two basic ways to approach this.

The easy one is give Python something like:

  from __the_far_far_future__ import Decimal_numbers

and have decimal literals converted into those instead of Python "floats".

The alternative is the implement a Python class, let us call it
"AbtractNumber", which stores the literal text you put in the
programme code "2.01" and converts it, _when used_, to a suitable
python type with the right semantics.

Either would get you code where:

    x = 2.01

is stored without loss of precision, and may get you the behaviour
you're after.

Which of these more closely matches your desires?

Please try to be precise. Examples don't quite do it without some
rather precise accompanying lagunage which says what specific things
the example is meant to illustrate.

If one of my two suggestions above is very close to what you're
after, please present an example where one suggestion does what you
want and where the other would not, with an explaination of why.

Just hoping to get at things more precisely and clearly,
-- 
Cameron Simpson <cs at zip.com.au>

Careful and correct use of language is a powerful aid to straight thinking,
for putting into words precisely what we mean necessitates getting our own
minds quite clear on what we mean.      - W.I.B. Beveridge

From stephen at xemacs.org  Fri Mar  7 04:30:55 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Mar 2014 12:30:55 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <5318EA28.3040305@stoneleaf.us>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com>
 <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com>
 <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com>
 <5318EA28.3040305@stoneleaf.us>
Message-ID: <87eh2eptw0.fsf@uwakimon.sk.tsukuba.ac.jp>

Ethan Furman writes:
 > On 03/05/2014 07:42 AM, Paul Moore wrote:
 > > On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote:
 > >>   Let me show you an actual zero:
 > >>
 > >>      if event.num_attendants:
 > >>          prepare_cake()
 > >>      else:
 > >>          cancel_event()

 > > I'm sorry, are you really trying to say that the above code is better than
 > >
 > >      if event.num_attendants != 0:
 > >          prepare_cake()
 > >      else:
 > >          cancel_event()
 > >
 > > ?

 > Yes.  This is Python.  blah blah blah != 0 is unnecessary boiler-plate.

I wouldn't call that *better*, just a different style.  I agree I
prefer "if blah.count:" to "if blah.count != 0:", but I might very
well write "if blah.count > 0:".  I don't see anything "wrong" with
using an explicit comparison, but I don't have a problem reading the
abbreviated style in this situation.[1]

Personally I would *always* use the "implied boolean" style for
emptiness tests (I'm an old Lisper), but I might very well use
explicit comparisons against zero.  For example, I would be likely to
write the party example with a reversed test (to keep 'if' and 'else'
close together):

    if event.num_attendants == 0:    # nobody's coming
        cancel_event()
    else:
        # several-pages-long suite of preparations
        prepare_cake()

and I think that looks better than

    if not event.num_attendants:     # it's a count, not a boolean
    # etc

But actually I'd probably design the event class differently, and
write it as:

    if not event.list_of_attendants: # attendance list is empty
    # etc

or perhaps

    if event.list_of_attendants:     # attendance list is non-empty
    # reverse the suites

(depending on the phase of the moon and whether the trout are running
in Esopus Creek to choose between the last two :-).


Footnotes: 
[1]  I do have a real issue with the "if x:" to test for a sentinel
meaning "uninitialized" that happens to evaluate to false.  There's
objectively a probability that it would pass an error that would be
caught if the "is None" form were used, but that probability is
admittedly low.  Tastes may differ regarding "importance" here.



From stephen at xemacs.org  Fri Mar  7 04:37:02 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Mar 2014 12:37:02 +0900
Subject: [Python-ideas] Please reconsider the Boolean
	evaluation	of	midnight
In-Reply-To: <53190936.6030608@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz>
Message-ID: <87d2hyptlt.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:

 > It seems to me that those claiming "midnight is special"
 > are inventing a justification after the fact -- one
 > that is not supported by the actual behaviour.
 >
 > Altogether, the whole thing seems to be a mess.

Mirroring the real world, no?




From steve at pearwood.info  Fri Mar  7 04:45:47 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 14:45:47 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
Message-ID: <20140307034547.GO28804@ando>

On Thu, Mar 06, 2014 at 04:17:55PM -0800, Mark H. Harris wrote:

> I just had one more thought along this line;  consider this:
> 
> >>> from pdeclib import *
> >>> s1=sqrt(2.01)
> >>> s1
> Decimal('1.41774468787578244511883132198668766452744')
> >>> s2=sqrt(d(2.01))
> >>> s2
> Decimal('1.41774468787578252029556185427085779261123')
> >>> 
> >>> s1**2
> Decimal('2.00999999999999978683717927196994423866272')
> >>> s2**2
> Decimal('2.01000000000000000000000000000000000000000')

If you skip the conversion to Decimal, you actually get the right answer 
using floats:

py> (2.01**0.5)**2
2.01

So the problem here isn't the binary float, but that Decimal 
by default has *too much precision* and consequently it ends 
up keeping digits that the user doesn't care about:

py> from decimal import Decimal as D
py> (D.from_float(2.01)**D("0.5"))**2
Decimal('2.009999999999999786837179272')

Floats have about 14 significant base-10 figures of precision (more in 
base-2), so if we tell Decimal to use the same, we should get the same 
result:

py> import decimal
py> ct = decimal.getcontext()
py> ct.prec = 14
py> (D.from_float(2.01)**D("0.5"))**2
Decimal('2.0100000000000')


Decimal is not a panacea. Both Decimal and binary floats have the same 
limitations, they just occur in different places for different numbers. 
All floating point numbers have these same issues. Fixed point numbers 
have different issues, rationals have their own issues, and symbolic 
computations have a different set of issues.

Computer maths is a leaky abstraction. No matter what you do, how you 
implement it, the abstraction leaks. Not even Mathematica can entirely 
hide the fact that it is computing rather than performing a Platonic 
ideal of mathematics.


-- 
Steven

From harrismh777 at gmail.com  Fri Mar  7 04:53:15 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 19:53:15 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
Message-ID: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>



On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote:
>
>
> The only solution without changing Python is to train end-users to write 
> something correct, like Decimal('.1'). 
>

       hi Andrew,  yes, that is the problem.  And, to be fair, that example 
is not really the worst
because it might be expected that the user should know how to construct 
Decimals, and 
could be educated to construct them properly--> Decimal('0.1');  I concur.
       It is worse in these two scenarios (and others too):
            sqrt(.1)
            sin(.1)
       No one would expect that the user should know to quote the 
number---when is that ever done?
       QED   this is broken.  Again, we know perfectly well why its 
happening (I am not ignorant) but its not right. 

>
> The obvious solution for changing Python is to make it easier to create 
> Decimal numbers correctly and/or harder to create them incorrectly. For 
> example, a decimal suffix, as already proposed before this thread, would 
> completely solve the problem: 
>
>     >>> a = 1d 
>     >>> b = .1d 
>     >>> a+b 
>     1.1d 
>
  
       Yes, and at a bare minimum, that is the immediate change I have been 
asking for, for now; nothing more.

       I answered Guido's questions in hopes that he might be willing to 
dialogue --- not dismiss.
 

> You proposed that Python should handle numbers in an OO way, with numbers 
> being real objects, instances of classes, with a hierarchy including 
> abstract base classes; all of this is already there in Python. 
>

       Yes, it is... but its not designed to use decimal floating point by 
default... in order to do that the entire OO 
setup will have to be changed (what Guido called a sweeping reform).  Its 
like if we want to use floating point
decimals in the 21st century, using python, we have to duck tape modules on 
and then educate users in the
correct input of numbers.  Seems convoluted to me.  

>
> You went off on a long digression about how you could implement this using 
> the details of C++-style inheritance, when Python has a completely (and 
> more powerful) different solution to inheritance that has already been used 
> to solve this problem. 
>
  
        No, I did not.  I answered Guido's questions regarding context as 
clearly as I could. If python has a more powerful
way to handle this situation, gladly do it!  I will be happy as a clam to 
beta test or help with the coding.
 

>
> You proposed some complicated AI-based solution to solve the problem of 
> using separate number classes in a single expression, even though Python 
> (almost exactly like C++, in this case) has already solved that problem 
> with operator overloading. 
>

        No, I did not.  I suggested that unifying numbers in an (AI) way 
could solve this problem (conceptually) by 
regarding all numbers as *PythonNumbers. *Decimals should not only be 
default, they should be integrated, not
tacked on with duck tape. 

>
> (And note that Python is flexible enough that third-party libraries can 
> easily insert new types like quaternions, matrices, symbolic expressions, 
> etc. into the hierarchy in a way that's transparent to end users. I can 
> multiply a NumPy matrix of float64 values by the builtin in 2 just by 
> writing "m * 2", and it works exactly the way you'd want it to. It's hard 
> to imagine that would be even feasible with an AI-based solution, but with 
> the current design, that's the easiest part of NumPy.) 
>

       That's nice for you.  Because  sqrt(.23709)  does not behave as I 
expect, sadly, I have to train my users to enter  sqrt('0.23709'). 

>
> There are some ideas in your posts that are worth responding to, 
>

        Thank you.  If a user goes to the time and trouble to present an 
idea clearly, I would expect the responders 
to respect the effort and respond to the points that make sense.


      Andrew, I respect you for taking the time to dialogue, I appreciate 
it.  Thanks.

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3c21e039/attachment-0001.html>

From ethan at stoneleaf.us  Fri Mar  7 04:50:35 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 06 Mar 2014 19:50:35 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <20140306030002.GH28804@ando>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <20140306030002.GH28804@ando>
Message-ID: <5319420B.3000004@stoneleaf.us>

On 03/05/2014 07:00 PM, Steven D'Aprano wrote:
> On Wed, Mar 05, 2014 at 09:23:37PM -0500, Donald Stufft wrote:
>
>> It?s hard to do any sort of search for this, however in an informal poll where I?ve shown
>> people this code not a single person thought it made sense, and most of them responded
>> with ?wtf??.
>
> Well I don't know who these people are, what their background is, or
> exactly how you phrased the question. But in my experience, most
> programmers have extremely strongly held opinions about the sensibility
> of certain features that have little or nothing with any rational
> analysis of the pros and cons and far more likely to be "that's
> different from the first language I learned, therefore it's rubbish".

As part of learning Python, I learned one of its core tenets was the divide between something and nothing (some of that 
from you, I believe ;),  midnight is most certainly something, so I was very surprised to learn that bool(midnight) is 
False.

--
~Ethan~

From stephen at xemacs.org  Fri Mar  7 04:58:45 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Mar 2014 12:58:45 +0900
Subject: [Python-ideas] Please reconsider the Boolean
	evaluation	of	midnight
In-Reply-To: <53190A16.4020201@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
Message-ID: <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Barry Warsaw wrote:
 > > On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote:
 > > 
 > >>Instances of existing code that would be broken by the change: 0
 > > 
 > > You can't know this because you can't audit all the Python code
 > > in the world.
 > 
 > I think he means instances that have been pointed out
 > so far in this discussion.

Well, either way it's a pretty careless claim.

Alexander and Tim both have stated that they have used "if time:" to
test for a zero time (per the documentation) in actual applications.
I gather from the fact that Tim is "-0" on the change that his code
was never widely distributed, but I don't know about Alexander's.
Probably not, as he didn't post a URL, either.  AFAIK the code doesn't
have to be distributed for gratuitous code breakage to be considered a
Very Bad Thing, though.

One of Alexander's use cases (changing date in a simulation at
midnight using naive times) is particularly salient here.  Unlike any
of the other cases described so far by either side, it's clear that
his simulation would be completely broken by the proposed incompatible
change in behavior, as the date would never change.  (Granted, it's
not clear to me whether he actually implemented this in production,
but I also can't say from what I've read that he didn't.)


From rosuav at gmail.com  Fri Mar  7 05:16:27 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 15:16:27 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com>
 <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com>
 <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com>
Message-ID: <CAPTjJmpj-DNtzxbk8Ae-zQomRd65Jfe5CxXeUUpnvGm+RCcrVQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 12:52 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> From: Chris Angelico <rosuav at gmail.com>
>
> Sent: Thursday, March 6, 2014 4:29 PM
>
>
>> So... I'm +1 for adding a literal syntax for decimals. +1 for adding a
>> stating-the-default for floats (do it straight away, then code that
>> uses it can be backported all the way even when there's a change of
>> default).
>
> This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more.
>
> Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"?
>

Yep. That's what I mean (when I  said "float"s up there, I meant the
current type 'float', aka binary floating point). The suffix won't
have any effect; it'll be like u"string" in Py3, explicitly stating
the default, and added for the exact same reason.

>> +0.5 for adding a syntax for fractions; support in principle
>> but the devil's in the details.
>
> What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right?
>
Precisely that detail. Is it 3r? 3F? Something else? Would it look
tidier as a prefix instead of a suffix? But if that can be resolved,
it'd be good to have a syntax for fractions.

I'm only +0.5 on that, as rationals aren't as big an advantage over
the status quo as decimal is. It's a lot less common to see:

>>> Fraction(1/3)
Fraction(6004799503160661, 18014398509481984)

than the oddities of decimal.Decimal construction that we're seeing
here. Having a numeric suffix for decimal literals will be much more
beneficial to the language, imo; if the bikeshedding of Fraction
literals is problematic, I'd not be against doing decimal (and binary
float) tags one version, and leaving Fraction for another version.
('Course, it might all work out perfectly, in which case great! Add
'em all at once.)

ChrisA

From harrismh777 at gmail.com  Fri Mar  7 05:17:33 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 20:17:33 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <85k3c6bt27.fsf@benfinney.id.au>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
 <85k3c6bt27.fsf@benfinney.id.au>
Message-ID: <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com>



On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote:
 

> But beware that even there, complaints about how Python behaves aren't
> going to be well received if you show a wilful ignorance of that
> behaviour. Mere ignorance is fine, of course: we all start out ignorant
> of any given topic. But wilful ignorance isn't a good foundation for
> expecting change. 
>
       Ben,  excuse me,  but you are out of line here.  I have not shown 
will full ignorance in any of this discussion (None).  I am not ignorant of 
the system, nor am I ignorant of the underlying caveats. I have studied
here, and I have a pretty good handle on how things are working. When
I have been confused I asked questions, read, experimented, and researched.
I will not have it here said that I have displayed will full ignorance, nor 
should
anyone believe it.
 
       I have a good understanding of why it is... I just don't like why it 
is... those
are two very different things. I am fully aware of the issues created by 
IEEE 754 1985 floats and doubles. This is 2014. I would expect that python
would be using IEEE 754 2008 not only for specification, but also for moving
towards decimal floating point arithmetic by default. Continuing to stick 
with
a 1985 standard, or even an IEEE 854 1987 standard, is putting all python
communities behind the eight ball going forward into the 21st Century. This 
is 
not will full ignorance. Its eyes-wide-open thoroughly knowledgable concern
with a genuine "will" to see the right thing done... because its the right 
thing to do.

marcus

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/d172b262/attachment.html>

From ben+python at benfinney.id.au  Fri Mar  7 05:27:39 2014
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 07 Mar 2014 15:27:39 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
 <85k3c6bt27.fsf@benfinney.id.au>
 <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com>
Message-ID: <85d2hybpl0.fsf@benfinney.id.au>

"Mark H. Harris" <harrismh777 at gmail.com>
writes:

> On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote:
>
> > But beware that even there, complaints about how Python behaves aren't
> > going to be well received if you show a wilful ignorance of that
> > behaviour. Mere ignorance is fine, of course: we all start out ignorant
> > of any given topic. But wilful ignorance isn't a good foundation for
> > expecting change. 

>        Ben,  excuse me,  but you are out of line here.  I have not shown 
> will full ignorance in any of this discussion (None).

I didn't mean to imply that, though I can see how the above can be read
that way. My apologies for giving that impression.

Nevertheless, the warning stands for those who wish to bring a complaint
in this forum or the ?python-list? forum.

-- 
 \       ?Probably the toughest time in anyone's life is when you have |
  `\    to murder a loved one because they're the devil.? ?Emo Philips |
_o__)                                                                  |
Ben Finney


From rosuav at gmail.com  Fri Mar  7 05:29:15 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 15:29:15 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
 <85k3c6bt27.fsf@benfinney.id.au>
 <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com>
Message-ID: <CAPTjJmp+UEOho5-0Ku22orMvw9bKmnsjdccQ2==xS3u5-h9jjQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 3:17 PM, Mark H. Harris <harrismh777 at gmail.com> wrote:
> On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote:
>
>>
>> But beware that even there, complaints about how Python behaves aren't
>> going to be well received if you show a wilful ignorance of that
>> behaviour. Mere ignorance is fine, of course: we all start out ignorant
>> of any given topic. But wilful ignorance isn't a good foundation for
>> expecting change.
>
>        Ben,  excuse me,  but you are out of line here.  I have not shown
> will full ignorance in any of this discussion (None).

Hey hey, I'd be careful of calling someone out like that... might want
to be a little humble and respectful here :)

The main problem here is that you and the Python interpreter have
different expectations about a string of digits in the source code.
You expect them to be represented exactly; the language specifies that
they be represented with the float type. Changing how the language
interprets something as fundamental as "number with decimal point in
it", even if unassailably correct, is a backward-incompatible change.
(The full unification of the int and long types wasn't effected until
Python 3, even though that change is unlikely to break much code.)

If someone wants to push for it, write the PEP, go through all the
details, then it'd be possible to have this for Python 3.5:

>>> 0.1d == Decimal("0.1")
True

But you're not going to have that effect for untagged numbers without
a long deprecation period; probably we're talking Python 4000, which
may not ever even exist. It'd be possible to have a __future__
directive that switches around the two types, so decimal is the
default and binary floats have to be tagged, but that could cause so
much confusion that it'd itself be the subject of massive controversy.
(Although there would be precedent for it. "from __future__ import
unicode_literals" didn't destroy Py2 code readability.)

ChrisA

From cs at zip.com.au  Fri Mar  7 05:32:33 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 7 Mar 2014 15:32:33 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
References: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
Message-ID: <20140307043233.GA14220@cskk.homeip.net>

On 06Mar2014 19:53, Mark H. Harris <harrismh777 at gmail.com> wrote:
> On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote:
[... lots of stuff, snipped along with Mark's replies ...]
> > You proposed some complicated AI-based solution to solve the problem of 
> > using separate number classes in a single expression, even though Python 
> > (almost exactly like C++, in this case) has already solved that problem 
> > with operator overloading. 
> 
>         No, I did not.  I suggested that unifying numbers in an (AI) way 
> could solve this problem (conceptually) by 
> regarding all numbers as *PythonNumbers. *Decimals should not only be 
> default, they should be integrated, not
> tacked on with duck tape. 

This sounds to me like keeping the original numeric text around and
only turning it into some efficient-for-a-machine representation
when it comes time to work on it.

One issue with that is that when the work occurs, it can be far
from where the number was defined, and the correct internal
representation might be poorly chosen.

> > (And note that Python is flexible enough that third-party libraries can 
> > easily insert new types like quaternions, matrices, symbolic expressions, 
> > etc. into the hierarchy in a way that's transparent to end users. I can 
> > multiply a NumPy matrix of float64 values by the builtin in 2 just by 
> > writing "m * 2", and it works exactly the way you'd want it to. It's hard 
> > to imagine that would be even feasible with an AI-based solution, but with 
> > the current design, that's the easiest part of NumPy.) 
> 
>        That's nice for you.  Because  sqrt(.23709)  does not behave as I 
> expect, sadly, I have to train my users to enter  sqrt('0.23709'). 

That's partly because .23709, in current python, becomes an IEEE float with some loss of precision
because binary fractions and base-10 fractions do not round trip.

But also partly because sqrt() (in the pure mathematical sense)
often produces transcendental numbers,
which are not representation by any fixed precision intergal base
notation - effectively IEEE floats and Decimal floats are
fractions/rationals.

So sqrt() will, for almost all numbers, involve loss of precision
no matter what base your backing storage is: base 2 as in IEEE float
or base 10 as in a Decimal. Steven (or Stephen) has already pointed
this out to you; perhaps it has been missed.

And going back (eg sqrt(2.01)*sqrt(2.01) ==> not-quite-2.01) just
extends this loss of precision.

This is an inherent problem unless sqrt() doesn't convert to a
"concrete" type, and instead just lurks as some symbolic representation,
so that any expression involving it becomes a progressively more
elaboration representation of an algebraic expression. Which may
be correct, but only until you try to evaluate it to get a concrete
number again.

> > There are some ideas in your posts that are worth responding to, 
> 
>         Thank you.  If a user goes to the time and trouble to present an 
> idea clearly, I would expect the responders 
> to respect the effort and respond to the points that make sense.

Many people have. You should see the short shrift some ideas receive.

I think part of the problem is not your wish for a "number" but that lack
of a concrete proposal to implement it.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

The Usenet is not the real world. The Usenet usually does not even resemble
the real world. - spaf at cs.purdue.edu

From rosuav at gmail.com  Fri Mar  7 05:38:02 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 15:38:02 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
 <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>
 <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com>
 <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com>
Message-ID: <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com>

On Fri, Mar 7, 2014 at 1:19 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm?
>

Fortunately, you don't have to be dictator of the whole world. As
admin of Minstrel Hall (minstrelhall.com), I simply made the
declaration that all campaign times would be listed in UTC. That
effectively bans DST and timezones. The in-game clock uses 24-hour
time, so a lot of people follow that. Most things happen on a weekly
cycle, so I haven't done anything about MDY (the in-game clock
currently reads "Fri 04:36:05"), but again, I would personally use YMD
most likely, and a lot of people will just follow that.

Let's start creating pockets like that all over the place. Train
people to manage international events in UTC. Eventually the benefits
will start to be more visible :)

ChrisA

From rosuav at gmail.com  Fri Mar  7 05:43:26 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 15:43:26 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <53192B14.4070304@canterbury.ac.nz>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com>
 <53190D28.50205@canterbury.ac.nz>
 <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com>
 <53192B14.4070304@canterbury.ac.nz>
Message-ID: <CAPTjJmoFyF8qbbX7VWRWFmBRFk0EzACNtgNZOCOkhKbwKuY=1g@mail.gmail.com>

On Fri, Mar 7, 2014 at 1:12 PM, Greg <greg.ewing at canterbury.ac.nz> wrote:
> On 7/03/2014 1:13 p.m., Chris Angelico wrote:
>>
>> What operations can you do on a time-of-day-with-timezone? Give me
>> some examples, and I'll show you, with the above two examples, how
>> quirky that can be.
>
>
> I'm not particularly attached to the idea of times of day with
> timezones; I wouldn't mind if they didn't exist.
>
> The main point is that they definitely don't make sense for
> time differences. An hour in New York is the same length as
> an hour in Melbourne, on any day of the year, as far as I know.

Sure. So a timedelta with timezone makes no sense. But we're not
talking about timedelta here, we're talking about time.

(And we're also ostriching leap seconds away. Don't wanna know! Don't
wanna know!)

A calendar that supports repeating events is going to need some kind
of rule system anyway. I don't think a "time with timezone" is
sufficient for that. I haven't used that kind of system (ever!), but
from Lennart Regebro's PyCon 2013 talk "Blame it on Caesar", I gather
that there is such a thing in Python.

So... from this thread, it seems like there's only one real use for
time-with-timezone: fracturing a datetime-with-timezone into two
parts, date and time, and recombining losslessly. Anyone have other
uses?

ChrisA

From harrismh777 at gmail.com  Fri Mar  7 05:51:16 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 20:51:16 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <85d2hybpl0.fsf@benfinney.id.au>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
 <85k3c6bt27.fsf@benfinney.id.au>
 <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com>
 <85d2hybpl0.fsf@benfinney.id.au>
Message-ID: <c6e1f8ff-a95d-4d2c-a9ff-49b5c4ee55e3@googlegroups.com>



On Thursday, March 6, 2014 10:27:39 PM UTC-6, Ben Finney wrote:
>
> >        Ben,  excuse me,  but you are out of line here.  I have not shown 
> > will full ignorance in any of this discussion (None).
>
> I didn't mean to imply that, though I can see how the above can be read
> that way. My apologies for giving that impression.
>
> Nevertheless, the warning stands for those who wish to bring a complaint
> in this forum or the ?python-list? forum.
>

hi Ben,  forgive me,  I misunderstood you.  No problem, nor hard feelings. 
Chris is 
absolutely correct when he asks for humble respect.  I absolutely respect 
you
guys, and I'm here to learn as much as anything. Thank you for your 
insights.

Kind regards,
marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/a67aaf91/attachment-0001.html>

From ethan at stoneleaf.us  Fri Mar  7 06:02:54 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 06 Mar 2014 21:02:54 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com>
 <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com>
Message-ID: <531952FE.5050205@stoneleaf.us>

On 03/06/2014 08:13 AM, Alexander Belopolsky wrote:
>
> On Thu, Mar 6, 2014 at 8:21 AM, M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>> wrote:
>
>     > Are they all False? No, no they're not (unless your local timezone is UTC):
>     >
>     >>>> bool(utcmidnight)
>     > False
>     >>>> bool(naivemidnight)
>     > False
>     >>>> bool(localmidnight)
>     > True
>
>     Now this is a what I consider a valid argument for making a change.
>
>
> FWIW, I would be +0 for making a change so that t.tzinfo is not None
> implies bool(t.timetz()) is True.
>
> My intuition goes like this.  Given
>
>>>> from datetime import *
>>>> t1 = datetime(2014, 1, 1)
>>>> t2 = datetime(2014, 1, 1, 12)
>>>> t3 = datetime(2014, 1, 1, tzinfo=timezone.utc)
>
> Instance t1 has no time information, so t1.time() is false-y.

If datetime actually worked like that I would agree with you; but it doesn't -- it says you have the time of midnight on 
2014, 1, 1, not that you don't have a time at all.

--
~Ethan~

From steve at pearwood.info  Fri Mar  7 06:25:21 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 16:25:21 +1100
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com>
References: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io>
 <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com>
 <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com>
 <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com>
Message-ID: <20140307052521.GP28804@ando>

On Fri, Mar 07, 2014 at 03:38:02PM +1100, Chris Angelico wrote:
> On Fri, Mar 7, 2014 at 1:19 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> > If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm?
> >
> 
> Fortunately, you don't have to be dictator of the whole world. As
> admin of Minstrel Hall (minstrelhall.com), I simply made the
> declaration that all campaign times would be listed in UTC. That
> effectively bans DST and timezones. 

No it doesn't. People still have to convert from the listed timezone to 
their own local timezone. The only difference is that instead of 
privileging some semi-arbitrary timezone on the basis of where your 
servers are, you're privileging an even more arbitrary timezone in 
Europe.

Depending on the distribution of your users, chances are good that 
you're actually inconveniencing more people by your decision than if you 
used the local timezone.

[...]
> Let's start creating pockets like that all over the place. Train
> people to manage international events in UTC. Eventually the benefits
> will start to be more visible :)

There are no benefits, not even to programmers, since they will still 
have to convert a local time to UTC. The only differences will be:

- people whose local time happens to be UTC won't have to do anything;

- everyone else will suffer more, as even local (to them) events will 
  need to be converted from UTC.


Really, all these (hopefully tongue-in-cheek) suggestions to ban 
timezones are as useful as those (unfortunately serious) suggestions to 
avoid all the issues with Unicode by going back to "good ol' ASCII" and 
insist that everybody write in American English.

(Actually a subset of American English, since ASCII is missing some 
common American symbols.)

-- 
Steven

From abarnert at yahoo.com  Fri Mar  7 06:30:47 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 6 Mar 2014 21:30:47 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
Message-ID: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>

On Mar 6, 2014, at 19:53, "Mark H. Harris" <harrismh777 at gmail.com> wrote:

> On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote:
>> 
>> 
>> The only solution without changing Python is to train end-users to write something correct, like Decimal('.1'). 
> 
>        hi Andrew,  yes, that is the problem.  And, to be fair, that example is not really the worst
> because it might be expected that the user should know how to construct Decimals, and 
> could be educated to construct them properly--> Decimal('0.1');  I concur.
>        It is worse in these two scenarios (and others too):
>             sqrt(.1)
>             sin(.1)

No, that's not the same problem but worse, it's a completely different problem.

In the first case, the user is trying to specify 0.1 as a decimal number, which is exactly representable, just not the way he's entered it.

In these cases, the numbers are irrational, and therefore inherently impossible to represent exactly. It doesn't matter whether you use decimal floats or binary floats.

>        No one would expect that the user should know to quote the number---when is that ever done?

What does quoting have to do with anything? Do you not understand why Decimal('.1') works? It's not because Python is a weakly-typed language that allows you to use strings as numbers, but because Decimal has a constructor that takes strings, for a specific and well-documented reason. Quoting here would just give you a TypeError. And nothing in your proposal(s) would change that. 

>        QED   this is broken.  Again, we know perfectly well why its happening (I am not ignorant) but its not right. 

The only way to fix this second problem is to use a symbolic representation instead of a numeric one.

The good news is that Python's design makes that pretty easy to add on--as SymPy demonstrates. Your proposal would actually make this kind of add-on much harder.

>> The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. 
>   
>        Yes, and at a bare minimum, that is the immediate change I have been asking for, for now; nothing more.

And people have agreed with that, and proposed feasible extensions to it. Presenting it as the first step toward some radical and ill-formed transformation of the whole language weakens the case for this suggestion. Implying that it would solve problems (like handling irrational numbers) that it obviously can't also weakens the case.

>        I answered Guido's questions in hopes that he might be willing to dialogue --- not dismiss.

He was willing to dialogue, as evidenced by his initial replies. It was only after you demonstrated your ignorance of basics fundamentals of Python (as a user, not even about its implementation) and math/numerics, and implied that you had no interest in correcting that ignorance, that he dismissed you. And he has every right to do so. He's the one donating his free time to make a great language for you (and many others) to use, not the other way around.

>> You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python.
> 
>        Yes, it is... but its not designed to use decimal floating point by default... in order to do that the entire OO 
> setup will have to be changed (what Guido called a sweeping reform).

Nonsense. Python already has classes for both binary and decimal floats. They both fit into the hierarchy properly. They both interact with other types the way they should. Changing which one you get from the literal "0.1" would be a simple change to the parser, and have no effect whatsoever to the OO setup.

>> You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem.
>   
>         No, I did not.  I answered Guido's questions regarding context as clearly as I could. If python has a more powerful
> way to handle this situation, gladly do it!  I will be happy as a clam to beta test or help with the coding.

It's already been done, years ago, so nobody has to do it. There's already an OO system that works, with abstract and concrete classes, and with a rich operator overloading system. And it's already been used to give you all of the abstract and concrete numeric types you want, and they do the things you asked for in this section, all without having to do any jumps through virtual pointer tables.

>> You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading.
> 
>         No, I did not.  I suggested that unifying numbers in an (AI) way could solve this problem (conceptually) by 
> regarding all numbers as PythonNumbers.

So you didn't suggest a complicated AI-based solution, you suggested a complicated AI-based solution?

> Decimals should not only be default, they should be integrated, not
> tacked on with duck tape. 

How does that have anything to do with the first half of this paragraph?

And in what way are Decimals "tacked on with duck tape"? They're instances of Number, and of Real. They act like numbers of other types, including interacting properly with other types like int. What is missing from the Decimal type and the ABCs in Number that makes you think we need a radical change?

If all you're suggesting is moving Decimal from the decimal module to builtins and/or adding parser support for decimal literals, those are not sweeping changes, they're both very simple changes. (That doesn't necessarily mean they're _desirable_ changes, but that's another argument--an argument nobody can actually begin until you make it clear whether or not that's what you're suggesting, which can't happen until you learn enough about using Python to know what you're suggesting.)

>> (And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to. It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.)
> 
>        That's nice for you.  Because  sqrt(.23709)  does not behave as I expect, sadly, I have to train my users to enter  sqrt('0.23709'). 

How do you expect it to behave? Again, using decimal floats here would make no difference. Instead of needing an infinite number of binary digits, you need an infinite number of decimal digits. Dividing infinite by log2(10) still leaves infinity. If you're trying to fix that, you're not trying to fix Python, you're trying to fix irrational numbers. This list cannot help you with that. Prayer is the only option, but medieval mathematicians tried that and didn't get very far, and eventually we had to accept that there are numbers that cannot be represented finitely.

>> There are some ideas in your posts that are worth responding to,
> 
>         Thank you.  If a user goes to the time and trouble to present an idea clearly, I would expect the responders 
> to respect the effort and respond to the points that make sense.
> 
> 
>       Andrew, I respect you for taking the time to dialogue, I appreciate it.  Thanks.
> 
> marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/2c86a9b0/attachment-0001.html>

From ethan at stoneleaf.us  Fri Mar  7 06:11:55 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 06 Mar 2014 21:11:55 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz>
 <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com>
Message-ID: <5319551B.6000000@stoneleaf.us>

On 03/06/2014 04:08 PM, Tim Peters wrote:
> [Greg Ewing]
>> ...
>> Altogether, the whole thing seems to be a mess.
>
> Although a mess that fits in a thimble that successfully hid in a dark
> corner unnoticed for a decade ;-)

If the decade is now, it didn't go that long.  I discovered this irritation within a year of learning Python, and at 
least three years ago brought it up on one of the mailing lists.  I ended up making my own Date, Time, and DateTime classes.

--
~Ethan~

From harrismh777 at gmail.com  Fri Mar  7 05:05:02 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 20:05:02 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307034547.GO28804@ando>
References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <20140307034547.GO28804@ando>
Message-ID: <5a9524af-1d00-4663-9bca-1d04bbbc4fea@googlegroups.com>



On Thursday, March 6, 2014 9:45:47 PM UTC-6, Steven D'Aprano wrote:
 

> Decimal is not a panacea. Both Decimal and binary floats have the same 
> limitations, they just occur in different places for different numbers. 
> All floating point numbers have these same issues. Fixed point numbers 
> have different issues, rationals have their own issues, and symbolic 
> computations have a different set of issues. 
>

       hi Steven,  yes, I concur.  My primary objection (for immediate 
concern)
is that we have now a very fast module for doing decimal floating point 
 math
with less of the issues you speak of. Doing decimal floating point math by
default, or at a minimum, providing a way to enter decimal numbers 1.234d
would do a lot to alleviate  much of the perceived difficulty with float 
issues. 

>
> Computer maths is a leaky abstraction. No matter what you do, how you 
> implement it, the abstraction leaks. Not even Mathematica can entirely 
> hide the fact that it is computing rather than performing a Platonic 
> ideal of mathematics. 
>

      Again, I concur.  Leaky is one thing... but being mired in 40-year-old
paradigms because IEEE 754 1985 floats |doubles are so entrenched just
doesn't make sense to me, and should be corrected.  

       I am not insisting on Guido's "sweeping" reform.  I'm just trying to 
make
like a little easier for number munching and get folks to be forward 
thinking 
about moving to a decimal based floating point system of number for python
by default---sometime.

      Thank you for your response, Steven.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/59f344cc/attachment.html>

From ethan at stoneleaf.us  Fri Mar  7 05:47:26 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 06 Mar 2014 20:47:26 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
 <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <lfa1as$8bm$1@ger.gmane.org> <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <53194F5E.8070103@stoneleaf.us>

On 03/06/2014 09:40 AM, Stephen J. Turnbull wrote:
> Antoine Pitrou writes:
>   > Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit :
>
>   > > never get fixed, and that "if var:" which is actually testing for
>   > > identity with a false-y sentinel is still strongly discouraged.
>   >
>   > Why "strongly discouraged"? This is more of a style issue than anything
>   > else?
>
> No, it's not just a style issue.  As Skip points out, it's a
> *semantic* issue as well.

Indeed, I have been bitten by using the lazy method of `if var` instead of `if var is not None`, and then been briefly 
puzzled as why my empty string | container | whatever that I passed in did not give me the correct results.

I'm still all for having truthy midnights.  :)

--
~Ethan~

From steve at pearwood.info  Fri Mar  7 07:11:21 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 17:11:21 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
Message-ID: <20140307061121.GQ28804@ando>

On Thu, Mar 06, 2014 at 06:19:56PM -0800, Mark H. Harris wrote:

>  >>> from decimal import Decimal
> >>> a=Decimal(1)
> >>> b=Decimal(.1)
> >>> a+b
> Decimal('1.100000000000000005551115123')      <==== does this not bother 
> you at all ?

Of course it bothers people a little. It bothers me. It also bothers me 
when I'm too hot wearing a coat and too cold when I take it off, but 
sometimes that's how the universe works.

For the first few releases of Decimal, it prohibited direct conversion 
of floats specifically to avoid that issue:

# Python 2.5
py> decimal.Decimal(2.01)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.5/decimal.py", line 648, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string


But that turned out to be more of a nuisance than what it was 
trying to protect from, so starting in Python 2.7 Decimal now 
supports direct and exact conversion from float.

The problem is, you are focused on one application for numeric computing 
to the exclusion of all else, specifically using Python as an 
interactive calculator. But in practice, for many uses, nobody typed in 
2.01 and nobody has any expectation that it will be exactly the value 
2.01. Rather, the value will be the result of some calculation, and 
there is *absolutely no reason to think* that the decimal number 2.01 
will be more accurate than the binary number 
10.101000111101011100001010001111010111000010100, or for that matter, 
the base-7 number 2.0462046204620462. The difference between those three 
representations is usually minor compared to the actual measurement 
errors of the initial data and the rounding errors from the calculation.



-- 
Steven

From harrismh777 at gmail.com  Fri Mar  7 07:24:47 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 22:24:47 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
Message-ID: <b63af00a-ebea-4f2e-89bb-59f74b98b506@googlegroups.com>



On Thursday, March 6, 2014 11:30:47 PM UTC-6, Andrew Barnert wrote:
>
>        I answered Guido's questions in hopes that he might be willing to 
> dialogue --- not dismiss.
>
>
> He was willing to dialogue, as evidenced by his initial replies. It was 
> only after you demonstrated your ignorance of basics fundamentals of Python 
> (as a user, not even about its implementation) and math/numerics, and 
> implied that you had no interest in correcting that ignorance, that he 
> dismissed you. And he has every right to do so. He's the one donating his 
> free time to make a great language for you (and many others) to use, not 
> the other way around.
>

    Andrew,  I demonstrated no such thing.   This is the second time you 
have accused me
of being ignorant of basics and fundamentals.  My answers in no way 
(implied or otherwise)
ignorant, they were well defined, carefully articulated, and implied an 
ernest  interest in understanding
and cooperation.  Your *ad hominem* comments about my so-called ignorance 
do not become you
and are beginning to annoy me...   I don't see any reason for it. 

    I know how the system works... and I think it can be improved.  My 
ideas are unpopular, I realize that,
but that does not mean that I am ignorant, nor does it mean that I don't 
understand python fundamentals.
I had a professor one time who told us, "You know when you have won the 
argument when they 
attack you personally  (or call you names)". 

    Why don't we keep the discussion on the details relevant to the topic 
and leave the personal attacks 
alone?  



marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/75b55e08/attachment-0001.html>

From harrismh777 at gmail.com  Fri Mar  7 07:39:35 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 22:39:35 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307061121.GQ28804@ando>
References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <20140307061121.GQ28804@ando>
Message-ID: <31615f5e-5429-4027-915d-8ceea251465e@googlegroups.com>



On Friday, March 7, 2014 12:11:21 AM UTC-6, Steven D'Aprano wrote:
 

> For the first few releases of Decimal, it prohibited direct conversion 
> of floats specifically to avoid that issue: 
>
> # Python 2.5 
> py> decimal.Decimal(2.01) 
> Traceback (most recent call last): 
>   File "<stdin>", line 1, in <module> 
>   File "/usr/local/lib/python2.5/decimal.py", line 648, in __new__ 
>     "First convert the float to a string") 
> TypeError: Cannot convert float to Decimal.  First convert the float to a 
> string 
>
>
> But that turned out to be more of a nuisance than what it was 
> trying to protect from, so starting in Python 2.7 Decimal now 
> supports direct and exact conversion from float. 


hey Steven,   whoa...!   that is useful, and very interesting news. That 
almost makes me want to cry.... 
Wait a minute, I'm going to check it out, ...  son of a gun, you're right. 
My Py2.6.1 does not
allow me to enter this   sqrt( .789 ).   What-da-ya-know........ Here is 
the output:

>>> from pdeclib import *
>>> sqrt(.789)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pdeclib.py", line 243, in sqrt
    sqr=Decimal(x).sqrt()
  File 
"/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/decimal.py", 
line 652, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a 
string
>>> 
 
So, "ignorant boy" finds out that the community has come to court with 
unclean hands !  

Everyone (even the core devs) know that this little problem sucks, and 
they've tried to
fix it in different ways, and yet when I bring it up its like (hypocrisy) 
I'm the one with the
problem--- when in fact all along everyone knows that this little IED was 
going to go off and
make a mess.   ~nice.

<sigh>   Steven,  I honor you and thank you for being honest and coming 
clean on this. 

Geeze... I'm going to bed.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3e251deb/attachment.html>

From harrismh777 at gmail.com  Fri Mar  7 07:01:51 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Thu, 6 Mar 2014 22:01:51 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
Message-ID: <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>



On Thursday, March 6, 2014 11:30:47 PM UTC-6, Andrew Barnert wrote:
>
>        No one would expect that the user should know to quote the 
> number---when is that ever done?
>
>
> What does quoting have to do with anything? Do you not understand why 
> Decimal('.1') works? It's not because Python is a weakly-typed language 
> that allows you to use strings as numbers, but because Decimal has a 
> constructor that takes strings, for a specific and well-documented reason. 
> Quoting here would just give you a TypeError. And nothing in your 
> proposal(s) would change that. 
>
>
I understand precisely what it happening.  It is just the opposite of what 
you 
are implying (that I'm ignorant) which you actually state later. I full 
well 
know that the quoted string is forcing the right constructor to be called 
for the 
Decimal object. I am an experienced OO programmer and have many years 
under my belt with C++, Smalltalk, and Java.  Please don't be pedantic with 
me, nor imply that I am ignorant.  It does not become you, and it irritates 
me... 
besides, it gets in the way of discussion. My point is not about loose 
typing, 
nor about Decimal constructors, ...  my point is
that an average user entering a number into a sqrt() function is not going 
to 
realize that they need to enter a quoted string in order to force the string
constructor on the Decimal Class !!   They are going to want to enter this 
-->  sqrt( .789 ) not this  sqrt( '0.789' )


I know how it works.   I'm asking for a reasonable response to the need for 
normal usability.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/9e3a2147/attachment.html>

From stefan at bytereef.org  Fri Mar  7 09:11:44 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Fri, 7 Mar 2014 09:11:44 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
Message-ID: <20140307081144.GA6899@sleipnir.bytereef.org>

Mark H. Harris <harrismh777 at gmail.com> wrote:
>    You can't be serious.  You have not commented yet...  are you happy with
> this:
> 
>  >>> from decimal import Decimal
> >>> a=Decimal(1)
> >>> b=Decimal(.1)
> >>> a+b
> Decimal('1.100000000000000005551115123')      <==== does this not bother you at
> all ?


You can set a trap:

>>> getcontext().traps[FloatOperation] = True
>>> b = Decimal(.1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
decimal.FloatOperation: [<class 'decimal.FloatOperation'>]
>>> 


Stefan Krah




From mertz at gnosis.cx  Fri Mar  7 10:17:26 2014
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 7 Mar 2014 01:17:26 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
Message-ID: <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com>

On Thu, Mar 6, 2014 at 6:51 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

> I hate this:
> >>> from decimal import Decimal
> >>> a=Decimal(1)
> >>> b=Decimal(.1)
> >>> a+b
> Decimal('1.100000000000000005551115123')
> >>>
> ... and I think the people who DO know intimately how python works, should
> do something to fix it.
>

The problem here--and the problem with the "default number type is decimal"
is that what you want fixed is "unfixable."  As a number of other people
have noted, what you are demanding is that numbers have finite sized and
exact representations, which they simply don't and can't in computers with
finite memory.

Your example, Mark, is merely one where you have misspelled your Python
code:

>>> from decimal import Decimal as D
>>> D('1') + D('.1')
Decimal('1.1')

Fix your spelling error, and in this case you get what you want.  However,
what about this:

>>> D(1/3) * 3
Decimal('0.9999999999999999444888487687')

There's no way to fix that spelling.  Yes, you might be able to play with a
decimal.Context in a way that gets this one example to happen to round the
way you want it to, but no context will do so generically for every similar
expected identity under reciprocal operators.

By coincidence, this works under binary FB *because* of the limited
precision of floats:

>>> 1/3 * 3
1.0

But that's just luck, basically, it's not because we've managed to
precisely represent 1/3 as a binary float.  We don't have to look all that
far for a problem case:

>>> 1/49 * 49
0.9999999999999999

Of course, we do have Fractions, which will always do the right thing under
the reciprocal division and multiplication operators:

>>> F('1/3') * 3
Fraction(1, 1)

Don't misspell this either, of course:

>>> F(1/3) * 3
Fraction(18014398509481983, 18014398509481984)

However, even if Fraction were to become the default numeric type for
Python, and preserve the reciprocity of div/mul, that wouldn't help us any
under the hoped reciprocals of sqrt() and square. Similarly, of course, for
trigonometric identities and various other numeric functions that produce
irrational answers. E.g.:

>>> from fractions import Fraction as F
>>> sqrt(F('1/3'))**2
0.3333333333333333
>>> F(sqrt(F('1/3'))**2)
Fraction(6004799503160661, 18014398509481984)

Well, that answer is a binary floating point; we leave Fraction land pretty
easily in Python.  The float we get, of course, is something that's not
exactly 1/3 (but is pretty close).

Even if we imagined some future version of Python that did sqrt() purely as
a Fraction without converting to IEEE 754, we STILL couldn't ever have an
exact identity.  There simply does not exist any Fraction that can
accurately represent sqrt(F('1/3')); there are only rational numbers that
are "pretty close."  Maybe this one, for example:

>>> F(sqrt(F('1/3')))
Fraction(1300077228592327, 2251799813685248)

Maybe this hypothetical Python 4000 that didn't do the conversion to
floating point would produce a different approximation, but it would always
be an approximation, not the Real (irrational) answer.



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/f5546c95/attachment-0001.html>

From saghul at gmail.com  Fri Mar  7 10:37:58 2014
From: saghul at gmail.com (=?ISO-8859-1?Q?Sa=FAl_Ibarra_Corretg=E9?=)
Date: Fri, 07 Mar 2014 10:37:58 +0100
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
Message-ID: <53199376.40007@gmail.com>

Hi,

(I hope this list is the right place for this)

I was browsing around for accept4 usage in Python and I saw it's already 
used internally in socket.accept (on Linux only), albeit it's not 
exposed, so accepting an incoming connection and setting it to be 
non-blocking takes 3 syscalls.

accept4 allows us to do this with a single syscall, and it made it into 
FreeBSD 10, so that's another system that could benefit from this 
optimization.

Would it be desirable to extend socket.accept to something like:

socket.accept(set_non_blocking=False)

which would use accept4's flags on supported systems and fallback to 
regular fcntl on others?

The idea is to accept an incoming connection and make it non-blocking at 
the same time, asyncio and other frameworks would benefit from this by 
doing one function call and 2 syscalls less (on supported systems, that is).


Cheers,

-- 
Sa?l Ibarra Corretg?
bettercallsaghul.com

From steve at pearwood.info  Fri Mar  7 11:12:35 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 21:12:35 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
Message-ID: <20140307101235.GR28804@ando>

On Thu, Mar 06, 2014 at 07:53:15PM -0800, Mark H. Harris wrote:
[...]
> because it might be expected that the user should know how to construct 
> Decimals, and 
> could be educated to construct them properly--> Decimal('0.1');  I concur.
>        It is worse in these two scenarios (and others too):
>             sqrt(.1)
>             sin(.1)

Decimal won't save you in those two cases.

For square roots, the result is usually a surd, an irrational number. 
The only way to store surds exactly is to store it symbolically. 
Should we try to turn Python into Mathematica?

In the case of sines, the same applies. While it is possible to 
calculate exact rational values for some angles, but for most the 
calculation rapidly becomes untenable, and in general trigonometric 
values are irrational, like surds.


>        No one would expect that the user should know to quote the 
> number---when is that ever done?

OF COURSE WE DO. This is a programming language, not a calculator.

We expect users -- programmers -- to know how to write functions, define 
classes, import modules, use regular expressions, call external 
processes, and so on. In other words, we expect them to know how to 
program.

>        QED   this is broken. 

No it is not.

Another factor you are not taking into account is that there is a reason 
that the computer industry has standardised on base-2 floats rather than 
10. 30 years ago, it was quite common to find applications using some 
variant of Decimal, perhaps using BCD (binary coded decimal) as their 
native numeric format. Going back even further, computers were built 
using decimal-based hardware:

http://en.wikipedia.org/wiki/Decimal_computer

But people moved to binary for multiple reasons:

- it's cheaper;

- it's faster;

- it's less surprising.


For example, between 1 and 100000, about 12% of integer-valued floats 
fail the "1/(1/n) == n" test, but over 51% of the integer-valued 
Decimals:

py> from decimal import Decimal as D
py> sum(1/(1/n) != n for n in range(1, 100001))
11846
py> sum(1/(1/D(n)) != n for n in range(1, 100001))
51665


Likewise we can test how many fail the "(1/n)*n == 1" test:

py> sum((1/n)*n != 1 for n in range(1, 100001))
13117
py> sum((1/D(n))*n != 1 for n in range(1, 100001))
36806

13% for floats versus 36% for Decimals.


One more to prove it isn't a fluke: the "sqrt(n)**2 == n" test:

py> sum((n**0.5)**2 != n for n in range(1, 100001))
49544
py> sum((n**D("0.5"))**2 != n for n in range(1, 100001))
71303


That's three for three in favour of binary floats.


[...]
>        Yes, and at a bare minimum, that is the immediate change I have been 
> asking for, for now; nothing more.

You're not going to get an immediate change, no matter how solid the 
case you ask for, no matter how logical the argument. Python 3.4 is not 
receiving any new features. The earliest this could come out is 3.5, in 
about 18 months or so.


>        I answered Guido's questions in hopes that he might be willing to 
> dialogue --- not dismiss.

You've had at least three emails from Guido. That's three more than most 
radical or wild ideas receive. The fact that so many people are 
continuing to discuss this with you is proof that we recognise that 
computer maths is hard and unintuitive. But just because you have 
identified a problem doesn't mean that your answer is a solution.


[...]
> > You proposed some complicated AI-based solution to solve the problem of 
> > using separate number classes in a single expression, even though Python 
> > (almost exactly like C++, in this case) has already solved that problem 
> > with operator overloading. 
> 
>         No, I did not.  I suggested that unifying numbers in an (AI) way 
> could solve this problem (conceptually) by 
> regarding all numbers as *PythonNumbers. *Decimals should not only be 
> default, they should be integrated, not
> tacked on with duck tape. 

It's statements like this that strongly suggest that you don't really 
understand how numbers work in computer programming. Or the practical 
limitations of what is *possible*. And as soon as you start talking 
about using an AI to decide what value the user intended, you've entered 
crackpot territory.

Speaking of AIs, perhaps you ought to see what Wolfram Alpha is capable 
of:

https://www.wolframalpha.com/input/?i=%28square+root+of+2.01%29+to+the+power+of+2

Not surprisingly for something with Mathematica behind it, it gets the 
exact right answer. Do you understand why Python the language cannot 
duplicate Mathematica's abilities with symbolic maths? Hint: it's not 
because it isn't technically possible. See, for example, libraries 
like sympy.



-- 
Steven

From steve at pearwood.info  Fri Mar  7 11:16:22 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 21:16:22 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307061121.GQ28804@ando>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <20140307061121.GQ28804@ando>
Message-ID: <20140307101622.GS28804@ando>

On Fri, Mar 07, 2014 at 05:11:21PM +1100, Steven D'Aprano wrote:
[...]
> there is *absolutely no reason to think* that the decimal number 2.01 
> will be more accurate than the binary number 
> 10.101000111101011100001010001111010111000010100, or for that matter, 
> the base-7 number 2.0462046204620462. The difference between those three 
> representations is usually minor compared to the actual measurement 
> errors of the initial data and the rounding errors from the calculation.

Er, apparently I cannot convert fractions into base 7 even using Python. 
That should be 2.003300330033 in base 7. The number I gave was actually 
2.1.


-- 
Steven

From steve at pearwood.info  Fri Mar  7 11:39:56 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 21:39:56 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
Message-ID: <20140307103956.GT28804@ando>

On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote:

> Hmm. Is the rounding done by float.__str__() an attractive nuisance?

It's not *rounding* precisely. Starting in Python 3.1, the float 
__repr__ will display the shortest decimal string that doesn't change 
the float's value.

Reading the issue tracker history for this change is informative:

http://bugs.python.org/issue1580

To the guys who worked on that, I take my hat off to you all.


-- 
Steven

From solipsis at pitrou.net  Fri Mar  7 11:40:39 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 7 Mar 2014 11:40:39 +0100
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
References: <53199376.40007@gmail.com>
Message-ID: <20140307114039.720efff9@fsol>

On Fri, 07 Mar 2014 10:37:58 +0100
Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
wrote:
> 
> accept4 allows us to do this with a single syscall, and it made it into 
> FreeBSD 10, so that's another system that could benefit from this 
> optimization.

If it doesn't, then perhaps the configure script needs to be fixed.

> The idea is to accept an incoming connection and make it non-blocking at 
> the same time, asyncio and other frameworks would benefit from this by 
> doing one function call and 2 syscalls less (on supported systems, that is).

That's not likely to do a significant difference (benchmarks welcome).

Regards

Antoine.



From rob.cliffe at btinternet.com  Fri Mar  7 11:51:12 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Fri, 07 Mar 2014 10:51:12 +0000
Subject: [Python-ideas] Please reconsider the Boolean
	evaluation	of	midnight
In-Reply-To: <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
 <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5319A4A0.10905@btinternet.com>


On 07/03/2014 03:58, Stephen J. Turnbull wrote:
> Greg Ewing writes:
>   > Barry Warsaw wrote:
>   > > On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote:
>   > >
>   > >>Instances of existing code that would be broken by the change: 0
>   > >
>   > > You can't know this because you can't audit all the Python code
>   > > in the world.
>   >
>   > I think he means instances that have been pointed out
>   > so far in this discussion.
>
> Well, either way it's a pretty careless claim.
>
>
Please read what I actually said:

     Let's _start_ a survey.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/bfe69403/attachment-0001.html>

From steve at pearwood.info  Fri Mar  7 11:57:32 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Mar 2014 21:57:32 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
 <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com>
Message-ID: <20140307105732.GU28804@ando>

On Thu, Mar 06, 2014 at 06:08:28PM -0800, Mark H. Harris wrote:

> If you write a number on paper do you write down   '2.01'   ?
> Do you write down    d(2.01)  ?
> Do you write down    2.01d   ?

Sometimes I write down 2.01 subscript 10. That's just a different way of 
spelling 2.01d.

(That's the trouble with rhetorical questions. Sometimes people will 
give an answer other than what you were expecting.)


> (or)   do you write down   2.01     ?
> 
> When you punch a number into your TI89  do you punch in  {'} {2} {.} {0} 
> {1} {'}   ?
> (or)  do you punch in    {2} {.} {0} {1}   ?

For what it's worth, the TI-89 stores numbers using a decimal floating 
point format.



-- 
Steven

From rosuav at gmail.com  Fri Mar  7 12:05:09 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 22:05:09 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307103956.GT28804@ando>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
 <20140307103956.GT28804@ando>
Message-ID: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>

On Fri, Mar 7, 2014 at 9:39 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote:
>
>> Hmm. Is the rounding done by float.__str__() an attractive nuisance?
>
> It's not *rounding* precisely. Starting in Python 3.1, the float
> __repr__ will display the shortest decimal string that doesn't change
> the float's value.
>
> Reading the issue tracker history for this change is informative:
>
> http://bugs.python.org/issue1580
>
> To the guys who worked on that, I take my hat off to you all.

That effect, yes. Let's call it "the magic of float.__str__", because
it really is pretty amazing.

But it's still post-processing magic. It means that strings appear to
round-trip through floats, as long as you're a long way within the
available precision; but as soon as you do operations, that ceases to
be the case. I think it's great for display, but is putting that into
__repr__ (at least, they do appear to be the same) an attractive
nuisance, in that it encourages people to treat float("...") as a true
representation?

Don't get me wrong, it's a really *awesome* feature. It just happens
to have been partially responsible for this thread, which I think is
approaching 300 posts.

ChrisA

From rosuav at gmail.com  Fri Mar  7 12:06:03 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 22:06:03 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
 <20140307103956.GT28804@ando>
 <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>
Message-ID: <CAPTjJmqhDea4Qj=aUDb1Po6i2GQYsr-Sby6MmQ2kc3EiNA7H8Q@mail.gmail.com>

On Fri, Mar 7, 2014 at 10:05 PM, Chris Angelico <rosuav at gmail.com> wrote:
> Don't get me wrong, it's a really *awesome* feature. It just happens
> to have been partially responsible for this thread, which I think is
> approaching 300 posts.

Whoops, no. This one's only approaching 100. It's the "should midnight
be true or false" that's approaching 300. Sorry!

ChrisA

From rosuav at gmail.com  Fri Mar  7 12:08:23 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Mar 2014 22:08:23 +1100
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
In-Reply-To: <53199376.40007@gmail.com>
References: <53199376.40007@gmail.com>
Message-ID: <CAPTjJmrcQQPL46G8urY1B-nweoJz0XkbhiUpXcOq5tHA2LdVhA@mail.gmail.com>

On Fri, Mar 7, 2014 at 8:37 PM, Sa?l Ibarra Corretg? <saghul at gmail.com> wrote:
> The idea is to accept an incoming connection and make it non-blocking at the
> same time, asyncio and other frameworks would benefit from this by doing one
> function call and 2 syscalls less (on supported systems, that is).

Is the benefit that it's simply more efficient, or is there a specific
benefit from atomicity? The latter is a pretty strong argument (look
at close-on-exec for instance), if it applies.

ChrisA

From saghul at gmail.com  Fri Mar  7 12:14:04 2014
From: saghul at gmail.com (=?UTF-8?B?U2HDumwgSWJhcnJhIENvcnJldGfDqQ==?=)
Date: Fri, 07 Mar 2014 12:14:04 +0100
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
In-Reply-To: <20140307114039.720efff9@fsol>
References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol>
Message-ID: <5319A9FC.1010609@gmail.com>

On 03/07/2014 11:40 AM, Antoine Pitrou wrote:
> On Fri, 07 Mar 2014 10:37:58 +0100
> Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
> wrote:
>>
>> accept4 allows us to do this with a single syscall, and it made it into
>> FreeBSD 10, so that's another system that could benefit from this
>> optimization.
>
> If it doesn't, then perhaps the configure script needs to be fixed.
>

Probably not, I'll have a look.

>> The idea is to accept an incoming connection and make it non-blocking at
>> the same time, asyncio and other frameworks would benefit from this by
>> doing one function call and 2 syscalls less (on supported systems, that is).
>
> That's not likely to do a significant difference (benchmarks welcome).
>

Actually, after http://hg.python.org/cpython/rev/5f0d1aad7322/ it's 2 
function calls (accept + set_blocking)+ 2 syscalls (accept + ioctl 
FIONBIO) vs 1 function call (accept) + 1 syscall (accept4).

-- 
Sa?l Ibarra Corretg?
bettercallsaghul.com

From rob.cliffe at btinternet.com  Fri Mar  7 12:13:29 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Fri, 07 Mar 2014 11:13:29 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
 <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
Message-ID: <5319A9D9.8000302@btinternet.com>


On 07/03/2014 02:16, Tim Peters wrote:
> [Tim]
>>> No, it's bizarre to attach a timezone to a time object because most
>>> tzinfo subclasses don't - and can't - know what to return for the UTC
>>> offset in the absence of - at least - month and day info too.  Hour,
>>> minute, second and microsecond aren't usually enough to determine
>>> whether "daylight time" is in effect, and most tzinfo subclasses do
>>> attempt to model daylight time.  And because daylight time is a
>>> political conceit, most tzinfo subclasses also need year info.  That
>>> has nothing to do with whether times are viewed abstractly,
>>> concretely, etc - it has to do with that time zones generally aren't
>>> _defined_ in terms of isolated time-of-day info.  It's 7:13 PM in US
>>> Central.  Is that daylight (CDT) or standard (CST) time?  It's
>>> impossible to answer.  What's the corresponding UTC time?  Ditto.
> [Andrew Barnert <abarnert at yahoo.com>]
>> Well, a time in Central is useless, but a time in CDT or CST is not, and you can
>> design a library that's smart enough to give you a CDT or CST (as appropriate) time
>> from a Central datetime.
> Tell me how.  You have, in context, a Python `time` object with a
> "Central" tzinfo member.  That's all you get from the user.  You're
> given absolutely no information about day, month, or year.  What's
> your algorithm for implementing picking CDT or CST "as appropriate"?
> Note that we're NOT talking about datetime.datetime objects here.
> These are datetime.time objects.
>
> This isn't a problem for datetime.datetime objects.  A "Central"
> tzinfo timeclass has no problem at all picking CDT or CST as
> appropriate given all the info in a datetime.datetime object.
> _______________________________________________
>
Guys, please.  This discussion is getting increasingly esoteric and has 
become irrelevant to the original proposition.
What it does show is that the current truthy behaviour of time objects 
is incomprehensible to the average programmer.  And therefore useless to 
her.
Rob Cliffe

From p.f.moore at gmail.com  Fri Mar  7 14:00:10 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 7 Mar 2014 13:00:10 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5319A9D9.8000302@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
 <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
 <5319A9D9.8000302@btinternet.com>
Message-ID: <CACac1F_DLxJSPF=j9Nr3DW+7cg4kA1+Dv28kN9ch35G6a6rZ_A@mail.gmail.com>

On 7 March 2014 11:13, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> Guys, please.  This discussion is getting increasingly esoteric and has
> become irrelevant to the original proposition.

Everything here is irrelevant to the original proposition. The bug
report has been reopened. The next (relevant) step is to create a
patch and post it to the bug report. Everything else is just people
having a chat...

Paul

From dickinsm at gmail.com  Fri Mar  7 14:07:15 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 7 Mar 2014 13:07:15 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io>
 <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org>
 <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org>
 <53185658.5080507@egenix.com>
 <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com>
Message-ID: <CAAu3qLXgOt0FNrigGjHMW3+S0hbd8kkLUJUrguYDsQmhmW8kAw@mail.gmail.com>

On Thu, Mar 6, 2014 at 12:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Now, let's consider the time with the *best* possible claim to being
> false: timestamp zero. How does that behave?
>
> [... snipped ...]


> So, unless I happen to live in UTC, it's highly unlikely that I'm
> going to infer from Python's *behaviour* that datetime.time() (unlike
> datetime.date() and datetime.datetime()) belong in the "number"
> category, rather than the "arbitrary object" category.
>

This one's even *more* fun if you live in the UK.  I live on the Greenwich
Meridian (and I mean that quite literally: it goes through the middle of my
living room), and since the epoch was in January, well away from British
Summer Time, I shouldn't see any of those pesky timezone-related problems
that those poor underprivileged people living in non-UTC timezones see.
 Right?

Here's Python 2.7 on my Mac laptop, whose timezone is set to London time:

>>> import datetime as dt
>>> bool(dt.datetime.fromtimestamp(0).time())
True


And on Windows (again with timezone set to the UK):

>>> import datetime as dt
>>> bool(dt.datetime.fromtimestamp(0).time())
False


Wait, what?  It turns out that in January 1970, the UK government was in
the middle of an experimental change to a year-round GMT+1 timezone.  Some
operating systems seem to be aware of that fact;  some aren't.

Count me in the bool(time(0)) should be True camp, by the way.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/5179e860/attachment.html>

From breamoreboy at yahoo.co.uk  Fri Mar  7 15:41:18 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 07 Mar 2014 14:41:18 +0000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <5319A9D9.8000302@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <loom.20140305T220007-633@post.gmane.org>
 <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com>
 <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au>
 <5317A696.8040605@egenix.com>
 <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com>
 <53181ADA.2030401@canterbury.ac.nz>
 <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com>
 <5318F8F2.5010209@canterbury.ac.nz>
 <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com>
 <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com>
 <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com>
 <5319A9D9.8000302@btinternet.com>
Message-ID: <lfclqa$jhd$1@ger.gmane.org>

On 07/03/2014 11:13, Rob Cliffe wrote:
>
> On 07/03/2014 02:16, Tim Peters wrote:
>> [Tim]
>>>> No, it's bizarre to attach a timezone to a time object because most
>>>> tzinfo subclasses don't - and can't - know what to return for the UTC
>>>> offset in the absence of - at least - month and day info too.  Hour,
>>>> minute, second and microsecond aren't usually enough to determine
>>>> whether "daylight time" is in effect, and most tzinfo subclasses do
>>>> attempt to model daylight time.  And because daylight time is a
>>>> political conceit, most tzinfo subclasses also need year info.  That
>>>> has nothing to do with whether times are viewed abstractly,
>>>> concretely, etc - it has to do with that time zones generally aren't
>>>> _defined_ in terms of isolated time-of-day info.  It's 7:13 PM in US
>>>> Central.  Is that daylight (CDT) or standard (CST) time?  It's
>>>> impossible to answer.  What's the corresponding UTC time?  Ditto.
>> [Andrew Barnert <abarnert at yahoo.com>]
>>> Well, a time in Central is useless, but a time in CDT or CST is not,
>>> and you can
>>> design a library that's smart enough to give you a CDT or CST (as
>>> appropriate) time
>>> from a Central datetime.
>> Tell me how.  You have, in context, a Python `time` object with a
>> "Central" tzinfo member.  That's all you get from the user.  You're
>> given absolutely no information about day, month, or year.  What's
>> your algorithm for implementing picking CDT or CST "as appropriate"?
>> Note that we're NOT talking about datetime.datetime objects here.
>> These are datetime.time objects.
>>
>> This isn't a problem for datetime.datetime objects.  A "Central"
>> tzinfo timeclass has no problem at all picking CDT or CST as
>> appropriate given all the info in a datetime.datetime object.
>> _______________________________________________
>>
> Guys, please.  This discussion is getting increasingly esoteric and has
> become irrelevant to the original proposition.
> What it does show is that the current truthy behaviour of time objects
> is incomprehensible to the average programmer.  And therefore useless to
> her.
> Rob Cliffe

I disagree.  People here have stated that they use it.  I could use it 
if I wanted to, having been raised to do some strange things, as in read 
docs, so I'm firmly in favour of sticking with the status quo.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From stephen at xemacs.org  Fri Mar  7 15:59:38 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Mar 2014 23:59:38 +0900
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
 <20140307103956.GT28804@ando>
 <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>
Message-ID: <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > That effect, yes. Let's call it "the magic of float.__str__", because
 > it really is pretty amazing.
 > 
 > But it's still post-processing magic. It means that strings appear to
 > round-trip through floats, as long as you're a long way within the
 > available precision; but as soon as you do operations, that ceases to
 > be the case. I think it's great for display, but is putting that into
 > __repr__ (at least, they do appear to be the same) an attractive
 > nuisance, in that it encourages people to treat float("...") as a true
 > representation?

What makes you think they need more encouragment?

Seriously, as one data point, I don't think having more "human"
representations encourages me the think of floating point results as
the product of arithmetic on real numbers.  I don't think anybody who
knows how tricky "floating point" arithmetic can be is going to be
fooled by the "pretty eyes" of a number represented as "2.0" rather
than "1.99999999999999743591".

From rosuav at gmail.com  Fri Mar  7 16:09:12 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Mar 2014 02:09:12 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com>
 <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com>
 <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com>
 <20140307103956.GT28804@ando>
 <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com>
 <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmp4YhMrix5p6B9=4o6sWQW5wdMmGM0RNz2i6E0D__ZckQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 1:59 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Seriously, as one data point, I don't think having more "human"
> representations encourages me the think of floating point results as
> the product of arithmetic on real numbers.  I don't think anybody who
> knows how tricky "floating point" arithmetic can be is going to be
> fooled by the "pretty eyes" of a number represented as "2.0" rather
> than "1.99999999999999743591".

Fair enough. I just remember reading, back in my really REALLY early
days with GW-BASIC, an explanation of why 3.2# (the hash made it
double-precision) came out as whatever-it-did. Went into a full
explanation of the nature of binary floating point, and the issue was
forced to your attention because just about _any_ value that wasn't a
neat multiple of a (negative) power of two would do that.

You can lead a programmer to docs, but you can't make him understand.

ChrisA

From stephen at xemacs.org  Fri Mar  7 16:53:55 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 08 Mar 2014 00:53:55 +0900
Subject: [Python-ideas] Please reconsider the
	Boolean	evaluation	of	midnight
In-Reply-To: <5319A4A0.10905@btinternet.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
 <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5319A4A0.10905@btinternet.com>
Message-ID: <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>

Rob Cliffe writes:

 > Please read what I actually said:
 > ??? Let's start a survey. 

Ah, OK.  You started a survey, but don't care if it is accurate.

I remain deeply unimpressed with your methodology.


From guido at python.org  Fri Mar  7 18:17:45 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 09:17:45 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>

If I had to do it over again I would *definitely* never make a time value
"falsy". The datetime module was conceived long ago, when the dangers of
falsy objects that weren't clearly trying to be "like numbers" or "like
collections" weren't so clear. Today I would argue that times aren't enough
"like numbers" to qualify.

The only question left in my mind is how safe it is to change the current
behavior, given that it's been documented and implemented for over a
decade. If this was for Python 3.0 (or even 3.1) I would definitely vote
for fixing it, damn the torpedoes. But given that we're talking about 3.5,
I'm more hesitant. It's also not something for which a deprecation warning
would work well. (When would it be issued? Whenever bool(<time>) is called?
Or only when it's about to return False?)

Still, my intuition tells me that it's pretty much a nuisance feature and
few people have actually relied on it, so I'd be okay with fixing (and
breaking!) this in 3.5, perhaps after a thorough search for how much the
feature is actually relied upon and how legitimate or important those uses
are. A search should *also* try to unearth code that is technically
*broken* by the current behavior. I would expect there to be quite a lot of
the latter kind, and very few of the former.

Fearing I have said nothing new,

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/ce710a46/attachment.html>

From ncoghlan at gmail.com  Fri Mar  7 18:30:52 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 8 Mar 2014 03:30:52 +1000
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
 <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5319A4A0.10905@btinternet.com>
 <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
Message-ID: <CADiSq7d3FrsFzk=DW=fta6EFh7S_7oX3=-efukc0O0edva=rPQ@mail.gmail.com>

On 8 Mar 2014 03:19, "Guido van Rossum" <guido at python.org> wrote:
>
> If I had to do it over again I would *definitely* never make a time value
"falsy". The datetime module was conceived long ago, when the dangers of
falsy objects that weren't clearly trying to be "like numbers" or "like
collections" weren't so clear. Today I would argue that times aren't enough
"like numbers" to qualify.
>
> The only question left in my mind is how safe it is to change the current
behavior, given that it's been documented and implemented for over a
decade. If this was for Python 3.0 (or even 3.1) I would definitely vote
for fixing it, damn the torpedoes. But given that we're talking about 3.5,
I'm more hesitant. It's also not something for which a deprecation warning
would work well. (When would it be issued? Whenever bool(<time>) is called?
Or only when it's about to return False?)

I was proposing a deprecation warning when it would be about to return
False, advising the use of an explicit comparison instead. My rationale was
that people that actually rely on it are at least somewhat likely to be
testing it and would trigger the warning and have a chance to change their
code.

However, on reflection, just changing with a note in the porting guide
would actually have a similar consequence.

Regards,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/b4b68e06/attachment.html>

From stephen at xemacs.org  Fri Mar  7 18:50:26 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 08 Mar 2014 02:50:26 +0900
Subject: [Python-ideas] Please reconsider the Boolean evaluation
	of	midnight
In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
 <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5319A4A0.10905@btinternet.com>
 <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
Message-ID: <874n39q4nx.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > It's also not something for which a deprecation warning would work
 > well. (When would it be issued? Whenever bool(<time>) is called?

Yes! Yes! Yes!  Just joking -- that's got to be the wrong thing to do,
because it would warn on precisely the construct (None as sentinel for
"uninitialized") that's the only reason anybody wants this.

 > Or only when it's about to return False?)

That seems reasonable to me.  People who are testing for this are
presumably *constructing* instances of midnight, either with a crontab
of "* * * * *" or in a simulation -- they'll hit it frequently.  The
other case that seems plausible is using a zero time as a sentinel for
"uninitialized" or "unspecified", and that *must* get a warning.

 > Fearing I have said nothing new,

IIRC, you're the only person to have mentioned the pragmatic issues
about the DeprecationWarning.


From ethan at stoneleaf.us  Fri Mar  7 19:07:19 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 07 Mar 2014 10:07:19 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org>
References: <201403051216.12392.shai@platonix.com>
 <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com>
 <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com>
 <201403051715.29773.shai@platonix.com>
 <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com>
 <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com>
 <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com>
 <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com>
 <lf7q2a$vpl$1@ger.gmane.org>
Message-ID: <531A0AD7.6020504@stoneleaf.us>

On 03/05/2014 10:23 AM, Mark Lawrence wrote:
> On 05/03/2014 18:12, Amber Yust wrote:
>>
>> I think this is the wrong stage to evaluate cost. After all, one of the
>> tenets of open source is that the core devs don't have to be the source
>> of all change.
>>
>
> But the core devs have to review the change and ultimately decide whether
> or not to commit it.  In this case it looks as if a patch would not be
>  accepted, so why keep going on about it, especially when there's a known
>  work around?

The point of discussion is to find out what all the issues are, not to give up with "oh well, the core devs aren't going 
to accept it anyway..."

--
~Ethan~

From harrismh777 at gmail.com  Fri Mar  7 20:01:01 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 11:01:01 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307101235.GR28804@ando>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
Message-ID: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>



On Friday, March 7, 2014 4:12:35 AM UTC-6, Steven D'Aprano wrote:

It's statements like this [ AI language ] that strongly suggest that you 
> don't really 
> understand how numbers work in computer programming. Or the practical 
> limitations of what is *possible*. And as soon as you start talking 
> about using an AI to decide what value the user intended, you've entered 
> crackpot territory. 


hi Steven,   fair enough.  When I refer to AI all I'm really stating is 
that some form of
computational intelligence needs to be employed beyond simple copying, &c. 
An 
intelligent decision needs to be used; that's all.  I'll stop using it (AI) 
, its not helpful.

Also, thank you for continuing to discuss this, it helps me grow, and it 
may lead to
an understanding or change. That's a good thing.

Here is the main post for today, from me, having slept on this and putting 
some things
back into historical timeframe.  Sometime before Py2.7.x it was not 
possible to promote
a binary float to a decimal float;  the system threw an exception (convert 
to string).

First,  that was the correct way to handle this issue from the inception of 
 decimal.Decimal. Throw
an exception and let everyone know that there is a problem; then let the 
user figure out how to
solve it.

It is not possible to correctly promote a binary float to a decimal float. 
The decision (in Py2.7?)
was to NOT throw an exception and to copy the binary float "exactly" into 
the decimal float.

Second, that was a bad decision--really--IMHO.  It was the wrong decision 
for two reasons 1) it denies what
is inherently poor about binary floating point representation in the first 
place, and 2) further 
compounds the problem by denying what is the benefit of using Decimal 
floating point in the
first place.  Not to mention, it gives the wrong answer;   from a decimal 
floating point perspective
it gives a VERY wrong answer.  (I realize most science and engineering 
solutions only require 
3-4 digits of accuracy---heck, I used to do that work on a slide rule). 

The point here is that binary floats should not be promoted to decimal 
floats by using "exact" copy
from binary float representation to decimal float representation.  This 
reminds me of the guys at
three mile island who turned off the alarms because they were annoying.

Steven,  I am going to try to go a little further without AI speak...   
What needs to happen is that 
binary floats need to be "correctly" promoted to decimal floats as 
appropriate.  This must not happen
by simple copying (that does not work).  There needs to be some policy in 
place that will correctly
"set" the decimal float value based on intelligence, not just simple 
copying. ( It may not be possible, or
it may require some funky   copy b float-->reduce context round-->set 
decimal .  I'm still thinking about it.

To make decimal the default, and to handle all numbers correctly giving the 
SciPy community the
ability to use binary floats easily too (without breaking codes) will 
require a complete rewrite of
numbers.Number  and maybe some other things.  Unifying the numbers concept 
in python is desirable 
but not practical at this point... I understand that; I get it. I am not 
asking anyone for this kind of "sweeping"
reform at this point... just think forward to the possibility.  Please.

Stefan's suggestion to set a trap is an excellent one, and I will play 
around with that a bit.  Although I was
initially asking for a method to more easily enter a literal decimal, and 
that is still important, I think really
what is called for here considering the history is to rescind the decision 
to promote binary floats to decimal
floats through an "exact" copy without flagging an error, or even giving a 
warning.

I am remembering my Zen of python... special cases aren't special enough to 
break the rules, although practicality
beats purity, errors should never pass silently.

Anyway, I know you wonderful guys will be thinking hard about this 
(evidenced by the discussion so far) and
I leave it in your competent hands.  I too will be thinking about it, 
perhaps adding some coding in the near future
that will be helpful. Again, I wish to thank Stefan Krah for his fine work 
on decimal.Decimal, without which none
of this discussion would be worth having in the first place.

Kind regards,
marcus



 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/d5fac344/attachment-0001.html>

From guido at python.org  Fri Mar  7 20:20:00 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 11:20:00 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
Message-ID: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>

Mark,

I don't know if it has occurred to you, but several of the rhetorical forms
you have used are pretty upsetting to many Python devs. (I can give you a
list offline.) It would really help the discussion if you tried to keep
your rhetoric in check. (A few other participants might also want to focus
more on the topic and less on Mark.)

Now I'd like to restate the core issue in ways that should appeal to Python
devs. I am describing the status quo in Python 3.4, since that's the
starting point for any evolution of Python. Please bear with me as I
describe the status quo -- I find it necessary as a context for any future
proposals.

Python's parser recognizes a few different types of number literals:
integers (e.g. 42), "decimal  point" notation (e.g. 3.14), "exponential
notation" (e.g. 2.1e12), and any of these followed by 'j'. There are also
binary, octal and hex notations for integers (e.g. 0b1010, 0o12, 0xa). The
parser, by design, does not have any understanding of the context in which
these numbers are used -- just as it doesn't know about the types of
variables (even when it's obvious -- e.g. after seeing "x = 4" the parser
does not keep track of the fact that x is now an integer with the value 4,
although it does remember it has seen an assignment to x).

What the parser does with those number literals (as with string literals
and with the keywords None, True, False) is that it creates an object of an
appropriate type with an appropriate value. This object is then used as a
value in the expression in which it occurs.

The current algorithm for creating the object is to create an int (which is
implemented as an exactly represented arbitrary-precision integer) when the
literal is either binary/octal/hex or decimal without point or exponent, a
Python float (which is an object wrapping an IEEE 754 double) when a
decimal point or exponent is present, and a Python complex (which wraps
*two* IEEE doubles) when a trailing 'j' is seen.

Note that negative numbers are not literals -- e.g. -10 is an expression
which applies the unary minus operator to an integer object with the value
10. (Similarly, IEEE inf and NaN don't have literals but requires
evaluating an expression, e.g. float('inf').)

Not all numbers come from literals -- most come from expressions such as
x+y, some are read from files or other input media. (E.g. the struct module
can "read" IEEE floats and doubles from byte strings containing their
"native" binary encoding, and float() with a string argument can interpret
decimal numbers with optional decimal point and/or exponent.)

The clever thing about expressions (in Python as in many languages) is that
the overloading of operators can make the "right" think happen when numbers
of different types are combined. Consider x+y. If x and y are both ints,
the result is an int (and is computed exactly). If either is a float and
the other an int, the result is a float. If either is a complex and the
other is an int or float, the result is a float.

If you are defining your own number-ish type in Python or C code, you can
make it play this game too, and this is how the current Decimal and
Fraction types work. Python's operator overloading machinery is flexible
enough so that e.g. Fraction can say "if a Fraction and an int are added,
the result is a Fraction; but if a Fraction and a float are added, the
result is a float." (Because Python is dynamic, it would be *possible* to
define a Fraction type that returns a plain int for results whose
denominator is 1, but the stdlib's Fraction type doesn't do this.)

Most of the time, the built-in types are conservative in their return type
(by which I mean that adding two ints returns an int, not a float), but
there are a few exceptions: int/int returns a float (because returning an
int would require truncation or a compound (div, mod) result), and the **
(power) operator also has a few special cases (int**negative_int and
negative_float**non_integer).

Now let's move on to the problems, which begin with the current float type.
Given the representation it is obvious that various limitations exist;
results may have to be truncated to fit into 64 bits, and conversion to and
from decimal cannot always preserve the exact value. It turns out that the
truncation of results doesn't bother most users (everyone knows what a
typical hand calculator does for 1/3), but the issues around conversion to
and from decimal often trip over users on their initial forays into
learning the language. The improvement we made to the output conversion
(dropping digits that don't affect round-tripping) take some of the sting
out of this, but Python newbies still excitedly point out some of the
unavoidable anomalies like 1.1 + 2.2 ==> 3.3000000000000003 when they first
discover it.

So what to do about it?

If we aren't too worried about strict backwards compatibility, there's an
easy answer that should shut up the newbies: change the parser so that when
it sees a number with a decimal point or an exponent it returns a Decimal
instance, and make a bunch of other adjustments to match. (E.g repr() of
the Decimal 3.14 should return '3.14' instead of "Decimal('3.13')".) The
float() builtin should also return a Decimal (we should probably just
rename the whole type to 'float'). The math module will have to be
rewritten. We should probably preserve the existing binary float under some
other name, perhaps requiring an import. We'll have to decide what to do
about complex numbers. (One option would be to remove supporting them in
the parser.) The details can be worked out in a PEP. I don't expect this
process to be easy or without controversies, but I'm confident we could
come up with a good design.

But there are big problems with such a proposal.

Those newbies perhaps aren't Python's most important user population. (And
for many of them it is actually a moment of enlightenment on their way to
becoming experts, once they hear the explanation.) The problems with binary
floats don't affect most actual calculations (never mind 1.1+2.2, once
you've seen what 1/3 looks like you switch all your output to some kind of
rounded format). Many of the problems with floating point that actually
matter (such as truncation, or infinities, or NaN) don't go away just by
switching everything to Decimal.

For many Python programs the switch to decimal would not matter at all. For
example, in all my own recent coding (which focuses on network I/O), the
only uses I've had for float is for timekeeping (which is inherently
approximate) and to prints some percentages in a report. Such code wouldn't
be affected at all -- it wouldn't break, but it wouldn't be any simpler
either. Decimal/binary simply isn't an issue here.

However, for the growing contingent of scientists who use Python as a
replacement for Matlab (not Mathematica!), it could be a big nuisance. They
don't care about decimal issues (they actually like binary better) and they
write lots of C++ code that interfaces between CPython's internal API and
various C++ libraries for numeric data processing, all of which use IEEE
binary. (If anything, they'd probably want a 32-bit binary float literal
more than a decimal float literal.) Python already defines some macros for
converting between standard Python float objects and C++ doubles (e.g.
PyFloat_AS_DOUBLE), so technically we could support source-code
compatibility for these users, but performance would definitely suffer
(currently that macro just returns the C++ double value already present in
the object; it would have to be changed to call a costly decimal-to-binary
macro).

There is also the huge effort of actually implementing such a proposal.
It's not insurmountable, but I estimate it would be at least as much work
as the str->unicode conversion we did for Python 3.

All in all I just don't see the stars aligned for this proposal. (Nor for
anything even more sweeping like representing 1/3 as a fraction or some
other symbolic representation.)

So what to do instead? A few simper proposals have been made.

Maybe we can add a new literal notation meaning 'decimal'; that would be a
relatively straightforward change to the parser (once the new decimal
extension module is incorporated), but it would not do much to avoid
surprising newbies (certainly you can't go teaching them to always write
3.14d instead of 3.14). However, it would probably help out frequent users
of Decimal. (Then again, they might not be using literals that much -- I
imagine the source of most such programs is user or file input.)

I'm not excited about a notation for Fraction -- the use cases are too
esoteric.

Maybe we can fix the conversion between Decimal and float (if this is
really all that matters to Mark, as it appears to be from his last email --
I'd already written most of the above before it arrived). Could it be as
simple as converting the float to a string using repr()? Given the smarts
in the float repr() that should fix the examples Mark complained about. Are
there any roadblocks in the implementation or assumptions of the Decimal
type here? Perhaps the default Decimal context being set for a much higher
precision makes it philosophically unacceptable?

I think I've said as much as I can, for now.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/547dd34f/attachment.html>

From mertz at gnosis.cx  Fri Mar  7 21:28:41 2014
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 7 Mar 2014 12:28:41 -0800
Subject: [Python-ideas] Fwd: Python Numbers as Human Concept Decimal System
In-Reply-To: <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com>
 <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com>
Message-ID: <CAEbHw4aNgFuGV+JYz1Z_T4XNnMg_ftpkZmnVE3RGvfEwZSEpow@mail.gmail.com>

On Thu, Mar 6, 2014 at 6:51 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

> I hate this:
> >>> from decimal import Decimal
> >>> a=Decimal(1)
> >>> b=Decimal(.1)
> >>> a+b
> Decimal('1.100000000000000005551115123')
> >>>
> ... and I think the people who DO know intimately how python works, should
> do something to fix it.
>

The problem here--and the problem with the "default number type is decimal"
is that what you want fixed is "unfixable."  As a number of other people
have noted, what you are demanding is that numbers have finite sized and
exact representations, which they simply don't and can't in computers with
finite memory.

Your example, Mark, is merely one where you have misspelled your Python
code:

>>> from decimal import Decimal as D
>>> D('1') + D('.1')
Decimal('1.1')

Fix your spelling error, and in this case you get what you want.  However,
what about this:

>>> D(1/3) * 3
Decimal('0.9999999999999999444888487687')

There's no way to fix that spelling.  Yes, you might be able to play with a
decimal.Context in a way that gets this one example to happen to round the
way you want it to, but no context will do so generically for every similar
expected identity under reciprocal operators.

By coincidence, this works under binary FB *because* of the limited
precision of floats:

>>> 1/3 * 3
1.0

But that's just luck, basically, it's not because we've managed to
precisely represent 1/3 as a binary float.  We don't have to look all that
far for a problem case:

>>> 1/49 * 49
0.9999999999999999

Of course, we do have Fractions, which will always do the right thing under
the reciprocal division and multiplication operators:

>>> F('1/3') * 3
Fraction(1, 1)

Don't misspell this either, of course:

>>> F(1/3) * 3
Fraction(18014398509481983, 18014398509481984)

However, even if Fraction were to become the default numeric type for
Python, and preserve the reciprocity of div/mul, that wouldn't help us any
under the hoped reciprocals of sqrt() and square. Similarly, of course, for
trigonometric identities and various other numeric functions that produce
irrational answers. E.g.:

>>> from fractions import Fraction as F
>>> sqrt(F('1/3'))**2
0.3333333333333333
>>> F(sqrt(F('1/3'))**2)
Fraction(6004799503160661, 18014398509481984)

Well, that answer is a binary floating point; we leave Fraction land pretty
easily in Python.  The float we get, of course, is something that's not
exactly 1/3 (but is pretty close).

Even if we imagined some future version of Python that did sqrt() purely as
a Fraction without converting to IEEE 754, we STILL couldn't ever have an
exact identity.  There simply does not exist any Fraction that can
accurately represent sqrt(F('1/3')); there are only rational numbers that
are "pretty close."  Maybe this one, for example:

>>> F(sqrt(F('1/3')))
Fraction(1300077228592327, 2251799813685248)

Maybe this hypothetical Python 4000 that didn't do the conversion to
floating point would produce a different approximation, but it would always
be an approximation, not the Real (irrational) answer.



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/c4ea31e5/attachment-0001.html>

From random832 at fastmail.us  Fri Mar  7 21:39:03 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 07 Mar 2014 15:39:03 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com>
 <20140305120410.GY28804@ando>
 <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com>
Message-ID: <1394224743.8914.91895745.755B4C9A@webmail.messagingengine.com>

On Wed, Mar 5, 2014, at 7:30, Oscar Benjamin wrote:
> In C and some other languages the f suffix indicates a numeric literal
> that has type "float" e.g. "6.0f". You can use upper case there as
> well but the convention is lower case and to include the .0 when it's
> an integer. I just though that 3F looks sufficiently distinct from the
> way it's typically done in C.

How about 3r?

On the subject of suffixes, it might be worth considering that they
often use "d" for "double" as well. C# uses "m" (for money) for decimal.

From random832 at fastmail.us  Fri Mar  7 21:45:51 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 07 Mar 2014 15:45:51 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
Message-ID: <1394225151.11384.91897925.4D8F1ECA@webmail.messagingengine.com>

On Wed, Mar 5, 2014, at 21:21, Mark H. Harris wrote:
>  "Then, and only then, a function that requires PI or half_PI  
> will first check to see if  __gpi__  has been cached and if so pull it 
> back, or if not calculate it (once).  If the context
> does not change then the function can take half of some whopping value of 
> PI, for the duration of the context, and
> use that,  but if not needed for the context will never bind it !

What if you want a number type to represent a fraction of pi exactly, so
you can take its sine and get the square root of a rational number, then
square it to get the rational number?

Or to take the sine of an arc tangent of a rational number.

From random832 at fastmail.us  Fri Mar  7 21:48:04 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 07 Mar 2014 15:48:04 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com>

On Thu, Mar 6, 2014, at 7:46, Stephen J. Turnbull wrote:
> Steven D'Aprano writes:
> 
>  > The problem is that fractions can be unbounded in memory,
> 
> Seminumerical Algorithms has a few problems on "floating slash", ie a
> representation of numbers where the total memory allocated to a
> fraction is fixed, but the amounts allocated to numerator and
> denominator are variable.  I don't know if it has ever been tried in
> practice, though.  And of course any reasonable fixed size would have
> very limited ability to express very large or very small magnitudes.

Or, for that matter, you could limit both the numerator and the
denominator to some maximum value - when an intermediate result exceeds
it (won't be more than the square of the maximum for most operations),
you find the closest representable value.

From abarnert at yahoo.com  Fri Mar  7 22:24:06 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 7 Mar 2014 13:24:06 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
Message-ID: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>

On Mar 7, 2014, at 11:01, "Mark H. Harris" <harrismh777 at gmail.com> wrote:
> 
> Here is the main post for today, from me, having slept on this and putting some things
> back into historical timeframe.  Sometime before Py2.7.x it was not possible to promote
> a binary float to a decimal float;  the system threw an exception (convert to string).
> 
> First,  that was the correct way to handle this issue from the inception of  decimal.Decimal. Throw
> an exception and let everyone know that there is a problem; then let the user figure out how to
> solve it.

The decision was discussed at the time, and all the pros and cons were hashed out. If you're not willing to read that discussion, your opinion that the change was a mistake is worth exactly as much as that of any individual user who asked for the change.

> It is not possible to correctly promote a binary float to a decimal float. The decision (in Py2.7?)
> was to NOT throw an exception and to copy the binary float "exactly" into the decimal float.

There is no such thing as copying a binary float exactly into a decimal float. Decimal float values are not a superset of binary float values; in either direction, you have to round. Which is what Python 2.7+ does.

> Steven,  I am going to try to go a little further without AI speak...   What needs to happen is that 
> binary floats need to be "correctly" promoted to decimal floats as appropriate.  This must not happen
> by simple copying (that does not work).  There needs to be some policy in place that will correctly
> "set" the decimal float value based on intelligence, not just simple copying. ( It may not be possible, or
> it may require some funky   copy b float-->reduce context round-->set decimal .  I'm still thinking about it.

Python is _already_ doing exactly what you're asking for. Each binary float is rounded to the closest possible decimal float.

And it's hard to imagine what else it _could_ be doing. There is no closer decimal than the closest decimal, so any "smarter" algorithm would either do the exact same thing, or be worse.

It's true that this "doesn't work" to solve your problem, but that's only because you're trying to solve an impossible problem. Once you've rounded a number inexactly (whether to binary or decimal), you can't get the lost data back.

> To make decimal the default, and to handle all numbers correctly giving the SciPy community the
> ability to use binary floats easily too (without breaking codes) will require a complete rewrite of
> numbers.Number  and maybe some other things.  

Why do you keep saying that even after it's been repeatedly explained to you that it wouldn't? Number and its subclasses do not care about the representation of the number. Guido explained to you exactly what would and wouldn't need to be changed, and the abstract base classes do not enter into it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/14713e49/attachment.html>

From harrismh777 at gmail.com  Fri Mar  7 23:18:57 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 14:18:57 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
Message-ID: <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com>



On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote:
>
> It is not possible to correctly promote a binary float to a decimal float. 
> The decision (in Py2.7?)
> was to NOT throw an exception and to copy the binary float "exactly" into 
> the decimal float.
>
>
> There is no such thing as copying a binary float exactly into a decimal 
> float. Decimal float values are not a superset of binary float values; in 
> either direction, you have to round. Which is what Python 2.7+ does.
>
>
Andrew, you are deliberately side-stepping the point. You apparently 
disagree with Steven D, and others,
who have already stated repeatedly on this list that on 2.7+ binary float 
literals are copies "exactly" to
decimal float representation!  That is what is causing this entire problem, 
sir.  That is the reason for this:
>>> from decimal import Decimal
>>> Decimal(.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')   
 <====== this is not a rounding problem
>>> 
 
It is a copy problem,  and it should not be allowed.  It was a bad decision 
to approach the problem this
way, and the correct response to it is to not allow it, throw an exception, 
and let the user fix it.
Otherwise, as I've stated earlier, there needs to be a policy that promotes 
the binary floating point 
representation to decimal floating point representation without the mess 
from the 16th digit out....  and,
that may not be possible (needs to be discussed, and researched).

It is clear from Guido's response earlier this afternoon that he 
understands the problem, and has some
good ideas for how to proceed.  I am happy with that, and will keep 
monitoring the news to see how
things go. 

Thanks for your feedback.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/1e718cc8/attachment-0001.html>

From cf.natali at gmail.com  Fri Mar  7 23:33:27 2014
From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Fri, 7 Mar 2014 22:33:27 +0000
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
In-Reply-To: <5319A9FC.1010609@gmail.com>
References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol>
 <5319A9FC.1010609@gmail.com>
Message-ID: <CAH_1eM0UJTPXTueg9+kRRAxBgnZTsZd38pFEjmHE-K9qWQY9KQ@mail.gmail.com>

accept4() was added for atomicity (O_CLOEXEC), not performance.
For pipe2() vs pipe(), this could make a measurable difference because the
base syscall is so fast, but for accept(), which is a blocking syscall,
shaving one or two syscalls are unlikely to yield any performance gain.

IMO that's not a reason good enough to make accept() signature more complex.

cf



2014-03-07 11:14 GMT+00:00 Sa?l Ibarra Corretg? <saghul at gmail.com>:

> On 03/07/2014 11:40 AM, Antoine Pitrou wrote:
>
>> On Fri, 07 Mar 2014 10:37:58 +0100
>> Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
>> wrote:
>>
>>>
>>> accept4 allows us to do this with a single syscall, and it made it into
>>> FreeBSD 10, so that's another system that could benefit from this
>>> optimization.
>>>
>>
>> If it doesn't, then perhaps the configure script needs to be fixed.
>>
>>
> Probably not, I'll have a look.
>
>
>  The idea is to accept an incoming connection and make it non-blocking at
>>> the same time, asyncio and other frameworks would benefit from this by
>>> doing one function call and 2 syscalls less (on supported systems, that
>>> is).
>>>
>>
>> That's not likely to do a significant difference (benchmarks welcome).
>>
>>
> Actually, after http://hg.python.org/cpython/rev/5f0d1aad7322/ it's 2
> function calls (accept + set_blocking)+ 2 syscalls (accept + ioctl FIONBIO)
> vs 1 function call (accept) + 1 syscall (accept4).
>
>
> --
> Sa?l Ibarra Corretg?
> bettercallsaghul.com
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/9ee8b132/attachment.html>

From ckaynor at zindagigames.com  Fri Mar  7 23:33:18 2014
From: ckaynor at zindagigames.com (Chris Kaynor)
Date: Fri, 7 Mar 2014 14:33:18 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
Message-ID: <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com>

On Fri, Mar 7, 2014 at 1:24 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> There is no such thing as copying a binary float exactly into a decimal
> float. Decimal float values are not a superset of binary float values; in
> either direction, you have to round. Which is what Python 2.7+ does.


Presuming enough digits of precision, every binary float could be
represented precisely in decimal. Decimal is base 10, which factors to 2
and 5, meaning all powers of 1/2 and 1/5 can be precisely represented.
Binary floats can only be powers of 1/2.

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/92bf222e/attachment.html>

From mertz at gnosis.cx  Sat Mar  8 00:12:00 2014
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 7 Mar 2014 15:12:00 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com>
Message-ID: <CAEbHw4Yavjb1x71dpTAZrjRaYk6302pm=BV2wWtT96RYtyqvsQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 2:33 PM, Chris Kaynor <ckaynor at zindagigames.com>wrote:

> Presuming enough digits of precision, every binary float could be
> represented precisely in decimal. Decimal is base 10, which factors to 2
> and 5, meaning all powers of 1/2 and 1/5 can be precisely represented.
> Binary floats can only be powers of 1/2.
>

This is half true.  Yes, the factors of 10 are 2 and 5.  So every number
that can be exactly represented in base-2 or in base-5 can also be
represented exactly in base-10 (given enough digits.

The problem is that the converse is not the case.  So in the case Mark
presents several times:

>>> from decimal import Decimal
>>> a = 0.1   # store a binary float
>>> Decimal(a)
Decimal('0.1000000000000000055511151231257827021181583404541015625')

That decimal number *is* an exact representation of the value referenced by
'a'.  So if that's all you want, no problem exists.

But the problem that does exist is that there is *no* binary floating point
representation that is exactly the same numeric value as Decimal('0.1').
 This would be true, not only for float32 or float64 values that Python has
(depending on the machine architecture and compilation switches), it would
also be true if we used float128, or float256, or float8192 numbers.

And again, while that one example might lead you to think that decimal
floating point is simply better, we don't need to go far to find a value
like Fraction('1/3') that simply cannot have an exact representation in
decimal either, no matter how many digits of precision we decide to use.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/aefcef70/attachment.html>

From abarnert at yahoo.com  Sat Mar  8 00:14:59 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 7 Mar 2014 15:14:59 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com>
Message-ID: <5A794BFD-C3D8-4E09-9FAE-B693AD664B8F@yahoo.com>

On Mar 7, 2014, at 14:18, "Mark H. Harris" <harrismh777 at gmail.com> wrote:

> Andrew, you are deliberately side-stepping the point. You apparently disagree with Steven D, and others,
> who have already stated repeatedly on this list that on 2.7+ binary float literals are copies "exactly" to
> decimal float representation!  That is what is causing this entire problem, sir.  

What's causing the problem is that 0.1 and float(0.1) can both be represented exactly in Decimal--and that they are different. You've lost information in converting to float (I may have stated that the wrong way around last time; if so, apologies), and you cannot regain that information when converting back.

If you assume people will only ever try to work with numbers that are representable in a few decimal digits, you can use that additional information to retrieve the list data. But that assumption clearly doesn't make sense generally, and if you treat all float values that way, you'll be adding more error on average.

> That is the reason for this:
> >>> from decimal import Decimal
> >>> Decimal(.1)
> Decimal('0.1000000000000000055511151231257827021181583404541015625')    <====== this is not a rounding problem

Yes it is, it's a rounding problem converting the ".1" literal to a float. There is no exact decimal->float->decimal conversion, which is why the right way to solve this is to avoid the float conversion--e.g., with your decimal literal syntax, or just constructing decimals from stings.


From harrismh777 at gmail.com  Sat Mar  8 00:39:59 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 15:39:59 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
Message-ID: <1c6f1e58-f30d-4b12-b7bf-abee9a801215@googlegroups.com>



On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote:
 

> The decision was discussed at the time, and all the pros and cons were 
> hashed out. If you're not willing to read that discussion, your opinion 
> that the change was a mistake is worth exactly as much as that of any 
> individual user who asked for the change.
>
> hi Andrew,  I am satisfied that the problem is well understood and that we 
no longer 
need to discuss the root cause, nor the computer science of trying to 
represent 
numbers in memory that are not representable. I admire you guys too much to 
keep
going with the previous discussion.  I am completely satisfied with Guido's 
last 
answer (thank you, by the way), and I look forward to seeing how the 
community
will address the main issue.

In response to your statement quoted above, I would like to suggest seeing 
if there
might be any way you could think of wording that, in future dialogue?

The response quoted above is not only another *ad hominem* attack, it is 
also a
straw man attack.  Your presumption that I am unwilling to read the 
discussion 
archive is unfounded and faulty thinking. In fact, I began a thorough 
attempt to 
understand the previous discussion as soon as I discovered the need through 
Steven's revelation last evening. In fact, I am very interested in the 
discussion
which led to the decision in question. I not only find it interesting 
philosophically
I also am interested in who was involved and how they were thinking at the 
time.  Of course none of that changes my opinion at this time that the 
decision 
was not correct.   I don't fully know yet, but I think it was made the way 
many 
decisions are made by young men;  like lightning,  ---shortest path to 
ground.

It would be helpful going forward in future discussions with me that you 
presume
not to put words in my mouth, nor to presume to know how I think or feel, 
nor 
presume to attack my character or my person.  I would really appreciate it.

Kind regards,
marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/1768c691/attachment.html>

From oscar.j.benjamin at gmail.com  Sat Mar  8 00:42:14 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 7 Mar 2014 23:42:14 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
Message-ID: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>

On 7 March 2014 19:20, Guido van Rossum <guido at python.org> wrote:
>
> Maybe we can fix the conversion between Decimal and float (if this is really
> all that matters to Mark, as it appears to be from his last email -- I'd
> already written most of the above before it arrived). Could it be as simple
> as converting the float to a string using repr()? Given the smarts in the
> float repr() that should fix the examples Mark complained about. Are there
> any roadblocks in the implementation or assumptions of the Decimal type
> here?

The current Decimal constructor comes with a guarantee of exactness.
It accepts int, str and float (and tuple) and creates a Decimal object
having the exact value of the int or float or the exact value that
results from a decimal interpretation of the str. This exactness is
not mandated by any of the standards on which the decimal module is
based but I think it is a good thing. I strongly oppose a change that
would have it use heuristics for interpreting other numeric types.

>Perhaps the default Decimal context being set for a much higher
> precision makes it philosophically unacceptable?

The context is ignored by the Decimal constructor which will always
create an exact value. If you want the created value rounded to
context use unary +:

>>> from decimal import Decimal as D
>>> D(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
>>> +D(0.1)
Decimal('0.1000000000000000055511151231')


Oscar

From guido at python.org  Sat Mar  8 01:10:34 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 16:10:34 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
Message-ID: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>

On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> On 7 March 2014 19:20, Guido van Rossum <guido at python.org> wrote:
> >
> > Maybe we can fix the conversion between Decimal and float (if this is
> really
> > all that matters to Mark, as it appears to be from his last email -- I'd
> > already written most of the above before it arrived). Could it be as
> simple
> > as converting the float to a string using repr()? Given the smarts in the
> > float repr() that should fix the examples Mark complained about. Are
> there
> > any roadblocks in the implementation or assumptions of the Decimal type
> > here?
>
> The current Decimal constructor comes with a guarantee of exactness.
>

But Decimal(<float>) is relatively new (it was an error in 2.6). I know
it's annoying to change it again, but I also have the feeling that there's
not much use in producing a Decimal with 53 (or more?) *decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe we
can relent on this guarantee and make the constructor by default use fewer
digits (e.g. using the same algorithm used by repr(<float>)) and have an
optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?


> It accepts int, str and float (and tuple) and creates a Decimal object
> having the exact value of the int or float or the exact value that
> results from a decimal interpretation of the str. This exactness is
> not mandated by any of the standards on which the decimal module is
> based but I think it is a good thing. I strongly oppose a change that
> would have it use heuristics for interpreting other numeric types.
>

Well, given that most floating point values are the result of operations
that inherently rounded to 53 bits anyway, I'm not sure that this guarantee
is useful. Sure, I agree that there should be *a* way to construct the
exact Decimal for any IEEE 754 double (since it is always possible, 10
being a multiple of 2), but I'm not sure how important it is that this is
the default constructor.


> >Perhaps the default Decimal context being set for a much higher
> > precision makes it philosophically unacceptable?
>
> The context is ignored by the Decimal constructor which will always
> create an exact value. If you want the created value rounded to
> context use unary +:
>
> >>> from decimal import Decimal as D
> >>> D(0.1)
> Decimal('0.1000000000000000055511151231257827021181583404541015625')
> >>> +D(0.1)
> Decimal('0.1000000000000000055511151231')


I guess that's too late.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/fdc61c24/attachment-0001.html>

From oscar.j.benjamin at gmail.com  Sat Mar  8 01:27:42 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 8 Mar 2014 00:27:42 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
Message-ID: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>

On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote:
> On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>
> wrote:
>>
>> The current Decimal constructor comes with a guarantee of exactness.
>
> But Decimal(<float>) is relatively new (it was an error in 2.6). I know it's
> annoying to change it again, but I also have the feeling that there's not
> much use in producing a Decimal with 53 (or more?) *decimal digits* of
> precision from a float with 53 *bits* -- at least not by default. Maybe we
> can relent on this guarantee and make the constructor by default use fewer
> digits (e.g. using the same algorithm used by repr(<float>)) and have an
> optional argument to produce the full precision? Or reserve
> Decimal.from_float() for that?

I'd rather a TypeError than an inexact conversion. The current
behaviour allows to control how rounding occurs. (I don't use the
decimal module unless this kind of thing is important).

>> It accepts int, str and float (and tuple) and creates a Decimal object
>> having the exact value of the int or float or the exact value that
>> results from a decimal interpretation of the str. This exactness is
>> not mandated by any of the standards on which the decimal module is
>> based but I think it is a good thing. I strongly oppose a change that
>> would have it use heuristics for interpreting other numeric types.
>
> Well, given that most floating point values are the result of operations
> that inherently rounded to 53 bits anyway, I'm not sure that this guarantee
> is useful. Sure, I agree that there should be *a* way to construct the exact
> Decimal for any IEEE 754 double (since it is always possible, 10 being a
> multiple of 2), but I'm not sure how important it is that this is the
> default constructor.

It's hard to reason about the accuracy of calculations that use the
decimal module. It's harder than with a fixed-width type because of
the possible presence of higher-than-context precision Decimals. The
invariant that the constructor is exact (or a TypeError) is of
significant benefit when using Decimal for code that accepts inputs of
different numeric types.


Oscar

From donald at stufft.io  Sat Mar  8 01:48:46 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 7 Mar 2014 19:48:46 -0500
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com>
 <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz>
 <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com>
 <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
Message-ID: <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io>


On Mar 7, 2014, at 12:17 PM, Guido van Rossum <guido at python.org> wrote:

> If I had to do it over again I would *definitely* never make a time value "falsy". The datetime module was conceived long ago, when the dangers of falsy objects that weren't clearly trying to be "like numbers" or "like collections" weren't so clear. Today I would argue that times aren't enough "like numbers" to qualify.
> 
> The only question left in my mind is how safe it is to change the current behavior, given that it's been documented and implemented for over a decade. If this was for Python 3.0 (or even 3.1) I would definitely vote for fixing it, damn the torpedoes. But given that we're talking about 3.5, I'm more hesitant. It's also not something for which a deprecation warning would work well. (When would it be issued? Whenever bool(<time>) is called? Or only when it's about to return False?)
> 
> Still, my intuition tells me that it's pretty much a nuisance feature and few people have actually relied on it, so I'd be okay with fixing (and breaking!) this in 3.5, perhaps after a thorough search for how much the feature is actually relied upon and how legitimate or important those uses are. A search should *also* try to unearth code that is technically *broken* by the current behavior. I would expect there to be quite a lot of the latter kind, and very few of the former.

Hrm, I?m not actually sure how to even achieve such a search. Due to Python?s dynamic nature it?s pretty hard to determine what type is being used in a boolean expression, if that data is even available (It could be coming from a database or some other data source).

Do you have any idea how to do such a search?

> 
> Fearing I have said nothing new,
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/8b351013/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/8b351013/attachment.sig>

From guido at python.org  Sat Mar  8 02:02:15 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 17:02:15 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
Message-ID: <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote:
> > On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin <
> oscar.j.benjamin at gmail.com>
> > wrote:
> >>
> >> The current Decimal constructor comes with a guarantee of exactness.
> >
> > But Decimal(<float>) is relatively new (it was an error in 2.6). I know
> it's
> > annoying to change it again, but I also have the feeling that there's not
> > much use in producing a Decimal with 53 (or more?) *decimal digits* of
> > precision from a float with 53 *bits* -- at least not by default. Maybe
> we
> > can relent on this guarantee and make the constructor by default use
> fewer
> > digits (e.g. using the same algorithm used by repr(<float>)) and have an
> > optional argument to produce the full precision? Or reserve
> > Decimal.from_float() for that?
>
> I'd rather a TypeError than an inexact conversion. The current
> behaviour allows to control how rounding occurs. (I don't use the
> decimal module unless this kind of thing is important).
>

My proposal still gives you control (e.g. through a keyword argument, or by
using Decimal.from_float() instead of the constructor). It just changes the
default conversion.

I have to admit I don't recall the discussion that led to accepting float
arguments to Decimal(), so I don't know if the exactness guarantee was ever
challenged or even discussed. For all other argument types (str, tuple of
digits, Decimal) it makes intuitive sense to guarantee exactness, because
the input is already in decimal form, so why would the caller pass in
digits they don't care about. But for a float the caller has no such
control -- I can call round(math.pi, 2) but the resulting float (taken at
face value) is still equal to
3.140000000000000124344978758017532527446746826171875, even though just
'3.14' is enough as input to float() to produce that exact value.

And unfortunately the default precision for Decimal is much larger than for
IEEE double, so using unary + does not get rid of those extraneous digits
-- it produces Decimal('3.140000000000000124344978758'). So you really have
to work hard to recover produce the intended Decimal('3.14') (or
Decimal('3.140000000000000')).

>> It accepts int, str and float (and tuple) and creates a Decimal object
> >> having the exact value of the int or float or the exact value that
> >> results from a decimal interpretation of the str. This exactness is
> >> not mandated by any of the standards on which the decimal module is
> >> based but I think it is a good thing. I strongly oppose a change that
> >> would have it use heuristics for interpreting other numeric types.
> >
> > Well, given that most floating point values are the result of operations
> > that inherently rounded to 53 bits anyway, I'm not sure that this
> guarantee
> > is useful. Sure, I agree that there should be *a* way to construct the
> exact
> > Decimal for any IEEE 754 double (since it is always possible, 10 being a
> > multiple of 2), but I'm not sure how important it is that this is the
> > default constructor.
>
> It's hard to reason about the accuracy of calculations that use the
> decimal module. It's harder than with a fixed-width type because of
> the possible presence of higher-than-context precision Decimals. The
> invariant that the constructor is exact (or a TypeError) is of
> significant benefit when using Decimal for code that accepts inputs of
> different numeric types.


Can you provide an example of this benefit? I can't seem to guess whether
you are talking about passing Decimal instances into code that then just
uses +, * etc., or whether you are talking about code that uses Decimal
internally and takes arguments of different types (e.g. int, float, string).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/4f1a5760/attachment-0001.html>

From guido at python.org  Sat Mar  8 02:03:39 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 17:03:39 -0800
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io>
References: <201403051216.12392.shai@platonix.com>
 <201403051805.15363.shai@platonix.com>
 <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com>
 <201403051838.25342.shai@platonix.com>
 <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com>
 <5317BF90.9030901@btinternet.com>
 <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com>
 <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io>
 <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com>
 <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org>
 <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com>
 <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io>
Message-ID: <CAP7+vJJgQ0+0_kHNKMvu5ucs95XUEvUra5CFG4zRpziLqR7meQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 4:48 PM, Donald Stufft <donald at stufft.io> wrote:

>
> On Mar 7, 2014, at 12:17 PM, Guido van Rossum <guido at python.org> wrote:
>
> If I had to do it over again I would *definitely* never make a time value
> "falsy". The datetime module was conceived long ago, when the dangers of
> falsy objects that weren't clearly trying to be "like numbers" or "like
> collections" weren't so clear. Today I would argue that times aren't enough
> "like numbers" to qualify.
>
> The only question left in my mind is how safe it is to change the current
> behavior, given that it's been documented and implemented for over a
> decade. If this was for Python 3.0 (or even 3.1) I would definitely vote
> for fixing it, damn the torpedoes. But given that we're talking about 3.5,
> I'm more hesitant. It's also not something for which a deprecation warning
> would work well. (When would it be issued? Whenever bool(<time>) is called?
> Or only when it's about to return False?)
>
> Still, my intuition tells me that it's pretty much a nuisance feature and
> few people have actually relied on it, so I'd be okay with fixing (and
> breaking!) this in 3.5, perhaps after a thorough search for how much the
> feature is actually relied upon and how legitimate or important those uses
> are. A search should *also* try to unearth code that is technically
> *broken* by the current behavior. I would expect there to be quite a lot of
> the latter kind, and very few of the former.
>
> Hrm, I'm not actually sure how to even achieve such a search. Due to
> Python's dynamic nature it's pretty hard to determine what type is being
> used in a boolean expression, if that data is even available (It could be
> coming from a database or some other data source).
>
> Do you have any idea how to do such a search?
>

I know someone at Sourcegraph who thinks he knows how to modify Sourcegraph
to do it.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/67d53f0a/attachment.html>

From steve at pearwood.info  Sat Mar  8 02:05:26 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 12:05:26 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
Message-ID: <20140308010526.GV28804@ando>

On Fri, Mar 07, 2014 at 11:01:01AM -0800, Mark H. Harris wrote:

> Here is the main post for today, from me, having slept on this and 
> putting some things back into historical timeframe.  Sometime before 
> Py2.7.x it was not possible to promote a binary float to a decimal 
> float; the system threw an exception (convert to string).
> 
> First, that was the correct way to handle this issue from the 
> inception of decimal.Decimal. Throw an exception and let everyone know 
> that there is a problem; then let the user figure out how to solve it.

No, it was the *conservative* way to handle the issue. By disallowing 
the feature, it was possible to add it later without breaking backwards 
compatibility. If it had been allowed from the beginning, it would have 
been impossible to remove the feature if it had turned out to be a 
mistake.


> It is not possible to correctly promote a binary float to a decimal 
> float. The decision (in Py2.7?) was to NOT throw an exception and to 
> copy the binary float "exactly" into the decimal float.

That's incorrect. The way Python converts between the two is the right 
way to do the conversion. Given a decimal d and a float f constructed 
from d, f is the closest possible float to d. And the same applies for 
conversions the other way around.

(This may only apply on platforms with IEEE-754 semantics, but I think 
that's nearly all platforms CPython runs on these days.)


> Second, that was a bad decision--really--IMHO.  It was the wrong 
> decision for two reasons 1) it denies what is inherently poor about 
> binary floating point representation in the first place,

What is inherently poor about binary floats is also poor for Decimal 
floats: limited precision, rounding errors, and violation of fundamental 
laws of the real number system. These problems, save one, apply equally 
to Decimal, just in different places with different numbers.

The *sole* difference is that with Decimal there is no rounding between 
what the user writes as a decimal string "1.23456" and what the value 
generated is. (Decimal also includes a bunch of extra features, like 
control of the precision, which floats don't support, but they *could* 
be supported if there was enough interest to justify the work.)

This benefit is mostly of use if you are dealing with (1) numerically 
naive users or (2) using Python as an interactive calculator.

In the case of (1), numerically naive users are not the most important 
subset of users for Python. In fact, I would say that numerically 
sophisticated users (scientists, mostly) are far more important to the 
Python ecosystem, and they *do not want your suggested change*. If we 
forced Decimals on them, they would be around with flaming torches and 
pitchforks. Seriously, the numeric community would likely abandon Python 
(with much wailing and gnashing of teeth) if we forced this on them.

In the case of (2), anyone who has used a common pocket calculator is 
used to seeing values calculated with a string of 9s or 0s at the end:

2.01000000000001
2.00999999999999

instead of the expected 2.01. In fact, even on advanced scientific 
calculators which use decimal floats internally, such as the TI-89, we 
still *frequently* witness this problem. It seems to me that the average 
numerically naive user has simply learned that "calculators and 
computers do approximate calculations" (which is actually correct, when 
compared to the mathematically exact result), and don't particularly 
care too much about it.

But if *you* care, that's okay. Python is a programming language. You 
can *program* it to make an interactive calculator with whatever 
properties you desire, including using Decimal floats exclusively. 
Python's done 97% of the work for you: the Decimal module, and the cmd 
module which makes building interactive command-line oriented 
applications easy. You can even simulate the Python interactive 
interpreter.

In other words, if you don't want to convert from floats to Decimal, 
there is *absolutely no reason* why you should.


> and 2) further compounds the problem by denying what is the benefit of 
> using Decimal floating point in the first place.

I must admit I'm not clear as to what you think is the benefit of 
Decimal which is being denied.


> Not to mention, it gives the wrong answer; from a decimal floating 
> point perspective it gives a VERY wrong answer.

Now that's simply not true. The relative error between the "right" 
answer:

py> from decimal import Decimal as D
py> a = D("2.01").sqrt()**2

and the "wrong" answer:

py> b = D(2.01).sqrt()**2

is *miniscule*, just one part in ten thousand million million:

py> (a-b)/a
Decimal('1.060511545905472636815920399E-16')

[...]
> What needs to happen is that binary floats need to be "correctly" 
> promoted to decimal floats as appropriate.  This must not happen by 
> simple copying (that does not work).

If I have given you the impression that float to Decimal conversion 
occurs by copying the bits from one into the other, I apologise, that 
was not my intention. That is not what happens.

What actually happens is that given any float, not just floats that are 
typed in by hand by the user, are converted to the closest possible 
Decimal. What you appear to want is for Python to inspect the value of a 
float like 2.0099999999999997868 (the actual float you get from typing 
2.01) and intelligently decide that what you *actually* wanted is the 
decimal 2.01.

Even if this intelligence was possible, it has one serious flaw that 
cripples the whole exercise. It cannot apply only to numbers typed in by 
the user, but would have to apply across the board. Python cannot 
distinguish between these two cases:

    Decimal(1.8703152)  # Give me this precise decimal

and 

    x = 1519.63802016624    # result of some intermediate calculation
    y = 812.5037  # result of another intermediate calculation
    z = x/y  # yet another intermediate calculation
    Decimal(z)  # convert from float to nearest decimal

But z is the binary float 1.8703152, and Decimal cannot do one thing in 
the first case and a different thing in the second, because it cannot 
see where the number came from, only what it is. It does not and can 
not see the literal string, only the float.

In the first case, you want 1.8703152 because that's what the user 
typed, but in the second case, we *must* return the result:

    Decimal('1.8703152000000000665380639475188218057155609130859375')

because that's the closest conversion, the one with the least error. 
Rounding it to 7 decimal places may be sufficient for some applications, 
but that's not Python's decision to make, any more than Python can 
decide that if you type 1.99999998 that you must have meant exactly 2.

> There needs to be some policy in place that will correctly "set" the 
> decimal float value based on intelligence

http://www.catb.org/jargon/html/D/DWIM.html



-- 
Steven

From steve at pearwood.info  Sat Mar  8 02:36:05 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 12:36:05 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
Message-ID: <20140308013605.GW28804@ando>

On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote:
> On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin
> <oscar.j.benjamin at gmail.com>wrote:
> 
> > On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote:
[...]

> > > I also have the feeling that there's not
> > > much use in producing a Decimal with 53 (or more?) *decimal digits* of
> > > precision from a float with 53 *bits* -- at least not by default. Maybe
> > > we can relent on this guarantee and make the constructor by default use
> > > fewer digits (e.g. using the same algorithm used by repr(<float>)) and 
> > > have an optional argument to produce the full precision? Or reserve
> > > Decimal.from_float() for that?
> >
> > I'd rather a TypeError than an inexact conversion. The current
> > behaviour allows to control how rounding occurs. (I don't use the
> > decimal module unless this kind of thing is important).

Like Oscar, I would not like to see this change.


> My proposal still gives you control (e.g. through a keyword argument, or by
> using Decimal.from_float() instead of the constructor). It just changes the
> default conversion.

Who is this default conversion meant to benefit?

I think that anyone passing floats to Decimal ought to know what they 
are doing, and be quite capable of controlling the rounding via the 
decimal context, or via an intermediate string:

value = Decimal("%.3f" % some_float)

I believe that Mark wants to use Python as an interactive calculator, 
and wants decimal-by-default semantics without having to type quotes 
around his decimal numbers. I suspect that for the amount of time and 
effort put into this discussion, he probably could have written an 
interactive calculator using Decimals with the cmd module in half the 
time :-)

It seems to me that Decimal should not try to guess what precision the 
user wants for any particular number. Mark is fond of the example of 
2.01, and wants to see precisely 2.01, but that assumes that some human 
being typed in those three digits. If it's the result of some 
intermediate calculation, there is no reason to believe that a decimal 
with the value 2.01 exactly is better than one with the value 
2.00999999999999978...

Note that the issue is more than just Decimal:

py> from fractions import Fraction
py> Fraction(2.01)
Fraction(1131529406376837, 562949953421312)

Even more so than for Decimal, I will go to the barricades to defend 
this exact conversion. Although perhaps Fraction could grow an extra 
argument to limit the denominator as a short-cut for this:

py> Fraction(2.01).limit_denominator(10000)
Fraction(201, 100)

I think it is important that Decimal and Fraction perform conversions as 
exactly as possible by default. You can always throw precision away 
afterwards, but you can't recover what was never saved in the first 
place.


[...]
> And unfortunately the default precision for Decimal is much larger than for
> IEEE double, so using unary + does not get rid of those extraneous digits
> -- it produces Decimal('3.140000000000000124344978758'). So you really have
> to work hard to recover produce the intended Decimal('3.14') (or
> Decimal('3.140000000000000')).

I wouldn't object to Decimal gaining a keyword argument for precision, 
although it is easy to use an intermediate string. In other words,

    Decimal(math.pi, prec=2)

could be the same as

    Decimal("%.2f" % math.pi)

But what would prec do for non-float arguments? Perhaps we should leave 
Decimal() alone and put any extra bells and whistles that apply only to 
floats into the from_float() method.



-- 
Steven

From guido at python.org  Sat Mar  8 03:02:02 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 18:02:02 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308010526.GV28804@ando>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
Message-ID: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>

On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> [...] The way Python converts between the two is the right
> way to do the conversion.


It's *exact*. I don't know we all agree it is the *right* way.


> Given a decimal d and a float f constructed
> from d, f is the closest possible float to d. And the same applies for
> conversions the other way around.
>

It's actually stronger the other way around: when d is constricted from f,
d is *equal* to the mathematical value of f.

The issue (as I see it) is that there are many different decimals d that
all convert to the same float f (because of rounding). The d that is
constructed by taking the exact value of f is gross overkill. If d was
constructed from f by always rounding to (I think) 17 digits it would still
back convert exactly to f; the new float repr() gives us an even "better"
(i.e. shorter) string of digits that are still guaranteed to to be
converted into exactly f (because there's simply no other float that is
closer to d than f).

(This may only apply on platforms with IEEE-754 semantics, but I think
> that's nearly all platforms CPython runs on these days.)
>

Sure.


> > Second, that was a bad decision--really--IMHO.  It was the wrong
> > decision for two reasons 1) it denies what is inherently poor about
> > binary floating point representation in the first place,
>
> What is inherently poor about binary floats is also poor for Decimal
> floats: limited precision, rounding errors, and violation of fundamental
> laws of the real number system. These problems, save one, apply equally
> to Decimal, just in different places with different numbers.
>
> The *sole* difference is that with Decimal there is no rounding between
> what the user writes as a decimal string "1.23456" and what the value
> generated is. (Decimal also includes a bunch of extra features, like
> control of the precision, which floats don't support, but they *could*
> be supported if there was enough interest to justify the work.)
>
> This benefit is mostly of use if you are dealing with (1) numerically
> naive users or (2) using Python as an interactive calculator.
>
> In the case of (1), numerically naive users are not the most important
> subset of users for Python. In fact, I would say that numerically
> sophisticated users (scientists, mostly) are far more important to the
> Python ecosystem, and they *do not want your suggested change*.


What is the "suggested change" here? If it's "default float literals to
Decimal" I agree. But if the suggestion is my "Decimal(<float>) should be
implemented as Decimal(repr(<float>))" I don't think most scientists care
(few of them use Decimal, they stay entirely in the realm of binary
floating point).


> If we
> forced Decimals on them, they would be around with flaming torches and
> pitchforks. Seriously, the numeric community would likely abandon Python
> (with much wailing and gnashing of teeth) if we forced this on them.
>

Right. In Mark's "post of the day" he already accepts that "decimal by
default" is untenable -- and my claim is that even if it was desirable, it
would be too big an undertaking. So it's dead.


> In the case of (2), anyone who has used a common pocket calculator is
> used to seeing values calculated with a string of 9s or 0s at the end:
>
> 2.01000000000001
> 2.00999999999999
>
> instead of the expected 2.01.


But the cause is calculations, like 1/3 ==> 0.333333333 and then multiply
by 3 ==> 0.999999999. If you enter 2.1, a pocket calculater displays 2.1.


> In fact, even on advanced scientific
> calculators which use decimal floats internally, such as the TI-89, we
> still *frequently* witness this problem.


I suspect that if Python exhibited the problem in exactly those cases where
a TI-89 exhibits it, few people would complain. The complaints would most
likely come from numerically *sophisticated* users who object against using
decimal (since binary has much better rounding behavior for arithmetic
operations).


> It seems to me that the average
> numerically naive user has simply learned that "calculators and
> computers do approximate calculations" (which is actually correct, when
> compared to the mathematically exact result), and don't particularly
> care too much about it.
>

And yet I have seen plenty of Tweets, emails and other public discussions
where people gleefully pointed out that "Python has a bug." I think I've
also seen tracker items created for such issues.


> But if *you* care, that's okay. Python is a programming language. You
> can *program* it to make an interactive calculator with whatever
> properties you desire, including using Decimal floats exclusively.
>

That sounds a bit condescending. :-(

Python's done 97% of the work for you: the Decimal module, and the cmd
> module which makes building interactive command-line oriented
> applications easy. You can even simulate the Python interactive
> interpreter.
>
> In other words, if you don't want to convert from floats to Decimal,
> there is *absolutely no reason* why you should.
>
>
> > and 2) further compounds the problem by denying what is the benefit of
> > using Decimal floating point in the first place.
>
> I must admit I'm not clear as to what you think is the benefit of
> Decimal which is being denied.
>

If I may venture a guess, I think the assumed benefit of Decimal here is
that it doesn't shatter naive users' expectations.


> > Not to mention, it gives the wrong answer; from a decimal floating
> > point perspective it gives a VERY wrong answer.
>
> Now that's simply not true. The relative error between the "right"
> answer:
>
> py> from decimal import Decimal as D
> py> a = D("2.01").sqrt()**2
>
> and the "wrong" answer:
>
> py> b = D(2.01).sqrt()**2
>
> is *miniscule*, just one part in ten thousand million million:
>
> py> (a-b)/a
> Decimal('1.060511545905472636815920399E-16')
>

This feels like unnecessary pedantry. (Not the first time in this thread.
:-( )


> [...]
> > What needs to happen is that binary floats need to be "correctly"
> > promoted to decimal floats as appropriate.  This must not happen by
> > simple copying (that does not work).
>
> If I have given you the impression that float to Decimal conversion
> occurs by copying the bits from one into the other, I apologise, that
> was not my intention. That is not what happens.
>
> What actually happens is that given any float, not just floats that are
> typed in by hand by the user, are converted to the closest possible
> Decimal.


Well, actually, a Decimal that is mathematically *equal*. (I understand
that this is technically the "closest possible" but the use of the latter
phrase suggests that it's not always *equal*, which the actual algorithm
used always produces a value that is mathematically equal to the input
float.)


> What you appear to want is for Python to inspect the value of a
> float like 2.0099999999999997868 (the actual float you get from typing
> 2.01) and intelligently decide that what you *actually* wanted is the
> decimal 2.01.
>
> Even if this intelligence was possible, it has one serious flaw that
> cripples the whole exercise. It cannot apply only to numbers typed in by
> the user, but would have to apply across the board. Python cannot
> distinguish between these two cases:
>
>     Decimal(1.8703152)  # Give me this precise decimal
>
> and
>
>     x = 1519.63802016624    # result of some intermediate calculation
>     y = 812.5037  # result of another intermediate calculation
>     z = x/y  # yet another intermediate calculation
>     Decimal(z)  # convert from float to nearest decimal
>
> But z is the binary float 1.8703152, and Decimal cannot do one thing in
> the first case and a different thing in the second, because it cannot
> see where the number came from, only what it is. It does not and can
> not see the literal string, only the float.
>
> In the first case, you want 1.8703152 because that's what the user
> typed, but in the second case, we *must* return the result:
>
>     Decimal('1.8703152000000000665380639475188218057155609130859375')
>
> because that's the closest conversion, the one with the least error.
>

Steven, that's actually not true. print(z) produces 1.8703152, and if I
enter float(Decimal('1.8703152')) I get the exact same value of z. Try it.
(In Python 2.7 or 3, please.) So I claim that if Decimal(z) produced the
same value as Decimal('1.8703152') nothing would be lost.


> Rounding it to 7 decimal places may be sufficient for some applications,
> but that's not Python's decision to make, any more than Python can
> decide that if you type 1.99999998 that you must have meant exactly 2.
>

The repr() function does not round to a fixed number of decimals. It
produces (in theory, dynamically, although I suspect that the current
algorithm is better) the shortest decimal string that, when converted back
to binary, equals *exactly* the input.


>
> > There needs to be some policy in place that will correctly "set" the
> > decimal float value based on intelligence
>
> http://www.catb.org/jargon/html/D/DWIM.html


Also condescending. :-(

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/b7bfde77/attachment.html>

From rosuav at gmail.com  Sat Mar  8 03:10:11 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Mar 2014 13:10:11 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
Message-ID: <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote:
> The repr() function does not round to a fixed number of decimals. It
> produces (in theory, dynamically, although I suspect that the current
> algorithm is better) the shortest decimal string that, when converted back
> to binary, equals *exactly* the input.

If that's the definition of a float's repr, then I'd support using
that by default for the construction of a Decimal from a float. You
can always use .as_integer_ratio() to get the exact stored
representation.

ChrisA

From guido at python.org  Sat Mar  8 03:15:11 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 18:15:11 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308013605.GW28804@ando>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
Message-ID: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>

On Fri, Mar 7, 2014 at 5:36 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote:
> > On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin
> > <oscar.j.benjamin at gmail.com>wrote:
> >
> > > On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote:
> [...]
>
> > > > I also have the feeling that there's not
> > > > much use in producing a Decimal with 53 (or more?) *decimal digits*
> of
> > > > precision from a float with 53 *bits* -- at least not by default.
> Maybe
> > > > we can relent on this guarantee and make the constructor by default
> use
> > > > fewer digits (e.g. using the same algorithm used by repr(<float>))
> and
> > > > have an optional argument to produce the full precision? Or reserve
> > > > Decimal.from_float() for that?
> > >
> > > I'd rather a TypeError than an inexact conversion. The current
> > > behaviour allows to control how rounding occurs. (I don't use the
> > > decimal module unless this kind of thing is important).
>
> Like Oscar, I would not like to see this change.
>

Can you argue your position?


>  > My proposal still gives you control (e.g. through a keyword argument,
> or by
> > using Decimal.from_float() instead of the constructor). It just changes
> the
> > default conversion.
>
> Who is this default conversion meant to benefit?
>
> I think that anyone passing floats to Decimal ought to know what they
> are doing,


I think that is provably not the case. (That they don't know -- we can
still argue about whether they ought to know.)


> and be quite capable of controlling the rounding via the
> decimal context, or via an intermediate string:
>
> value = Decimal("%.3f" % some_float)
>

Or, Decimal(repr(some_float)), which DWIM.


> I believe that Mark wants to use Python as an interactive calculator,
> and wants decimal-by-default semantics without having to type quotes
> around his decimal numbers. I suspect that for the amount of time and
> effort put into this discussion, he probably could have written an
> interactive calculator using Decimals with the cmd module in half the
> time :-)
>

Mark doesn't like it when we try to guess what he means or believes, but my
guess is that he's actually trying to argue on behalf of Python users who
would benefit from using Decimal but don't want to be bothered with a lot
of rules. Telling such people "use the Decimal class" is much easier than
telling them "use the Decimal class, and always put quotes around your
numbers" (plus the latter advice is still very approximate, so the real
version would be even longer and more confusing).


> It seems to me that Decimal should not try to guess what precision the
> user wants for any particular number. Mark is fond of the example of
> 2.01, and wants to see precisely 2.01, but that assumes that some human
> being typed in those three digits. If it's the result of some
> intermediate calculation, there is no reason to believe that a decimal
> with the value 2.01 exactly is better than one with the value
> 2.00999999999999978...
>

Given the effort that went into making repr(<float>) do the right thing I
think that you're dismissing a strawman here. The real proposal does not
require guessing intent -- it just uses an existing, superior conversion
from float to decimal that is already an integral part of the language. And
it uses some static information that Decimal currently completely ignores,
i.e. all Python floats (being IEEE doubles) are created with 53 bits of
precision.


> Note that the issue is more than just Decimal:
>
> py> from fractions import Fraction
> py> Fraction(2.01)
> Fraction(1131529406376837, 562949953421312)
>


> Even more so than for Decimal, I will go to the barricades to defend
> this exact conversion. Although perhaps Fraction could grow an extra
> argument to limit the denominator as a short-cut for this:
>
> py> Fraction(2.01).limit_denominator(10000)
> Fraction(201, 100)
>

An inferior solution. Anyway, nobody cares about Fraction.


> I think it is important that Decimal and Fraction perform conversions as
> exactly as possible by default.


Again, why produce 56 *digits* of precision by default for something that
only intrinsically has 53 *bits*?


> You can always throw precision away
> afterwards, but you can't recover what was never saved in the first
> place.
>

There should definitely be *some* API to convert a float to the exact
mathematical Decimal. But I don't think it necessarily needs to be the
default constructor. (It would be different if Decimal was a fixed-point
type. But it isn't.)


> [...]
> > And unfortunately the default precision for Decimal is much larger than
> for
> > IEEE double, so using unary + does not get rid of those extraneous digits
> > -- it produces Decimal('3.140000000000000124344978758'). So you really
> have
> > to work hard to recover produce the intended Decimal('3.14') (or
> > Decimal('3.140000000000000')).
>
> I wouldn't object to Decimal gaining a keyword argument for precision,
> although it is easy to use an intermediate string. In other words,
>
>     Decimal(math.pi, prec=2)
>
> could be the same as
>
>     Decimal("%.2f" % math.pi)
>

This is actually another strawman. Nobody in this thread has asked for a
way to truncate significant digits like that. It's the *insignificant*
digits that bother some. The question is, what should Decimal(math.pi) do.
It currently returns
Decimal('3.141592653589793115997963468544185161590576171875'), which helps
noone except people interested in how floats are represented internally.


> But what would prec do for non-float arguments? Perhaps we should leave
> Decimal() alone and put any extra bells and whistles that apply only to
> floats into the from_float() method.
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/9b127260/attachment.html>

From guido at python.org  Sat Mar  8 03:19:14 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 18:19:14 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>
Message-ID: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>

On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote:
> > The repr() function does not round to a fixed number of decimals. It
> > produces (in theory, dynamically, although I suspect that the current
> > algorithm is better) the shortest decimal string that, when converted
> back
> > to binary, equals *exactly* the input.
>
> If that's the definition of a float's repr, then I'd support using
> that by default for the construction of a Decimal from a float. You
> can always use .as_integer_ratio() to get the exact stored
> representation.
>

Sadly I can't point to exactly where this is documented, but since Python
2.7 and some early version of 3.x that is indeed how repr(<float>) works.

(And str(<float>) too -- before 2.7, str() would give only 12 digits while
repr() gave 17, but the new repr() makes such shenanigans unnecessary. So I
could have saved myself a few characters by using str() instead of repr()
everywhere in this thread. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/c2f7661c/attachment.html>

From rosuav at gmail.com  Sat Mar  8 03:46:14 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Mar 2014 13:46:14 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>
 <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>
Message-ID: <CAPTjJmobvscTGCyiuK=+eSFB-9VRDCMo03vy3y+evSAEPKLCgQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 1:19 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote:
>> > The repr() function does not round to a fixed number of decimals. It
>> > produces (in theory, dynamically, although I suspect that the current
>> > algorithm is better) the shortest decimal string that, when converted
>> > back
>> > to binary, equals *exactly* the input.
>>
>> If that's the definition of a float's repr, then I'd support using
>> that by default for the construction of a Decimal from a float. You
>> can always use .as_integer_ratio() to get the exact stored
>> representation.
>
> Sadly I can't point to exactly where this is documented, but since Python
> 2.7 and some early version of 3.x that is indeed how repr(<float>) works.

I couldn't find it anywhere - help(float.__repr__) just points to it
being the implementation of repr(some_float), and poking around on the
docs didn't show up any explanation. It'd be something worth
mentioning somewhere, as that's a great feature, but I don't really
know where, heh.

ChrisA

From eric at trueblade.com  Sat Mar  8 03:50:22 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 07 Mar 2014 21:50:22 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>
 <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>
Message-ID: <531A856E.1020207@trueblade.com>

On 3/7/2014 9:19 PM, Guido van Rossum wrote:
> On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com
> <mailto:rosuav at gmail.com>> wrote:
>     If that's the definition of a float's repr, then I'd support using
>     that by default for the construction of a Decimal from a float. You
>     can always use .as_integer_ratio() to get the exact stored
>     representation.
> 
> 
> Sadly I can't point to exactly where this is documented, but since
> Python 2.7 and some early version of 3.x that is indeed how
> repr(<float>) works.

It's mentioned at the bottom of this page:
http://docs.python.org/2/tutorial/floatingpoint.html

But oddly the wording in
http://docs.python.org/3/tutorial/floatingpoint.html is not as precise.
There are some weasel words in there because we do have a fallback if we
can't run Gay's code on some systems (sys.float_repr_style='legacy').
But I think for all practical purposes it's always true.

> (And str(<float>) too -- before 2.7, str() would give only 12 digits
> while repr() gave 17, but the new repr() makes such shenanigans
> unnecessary. So I could have saved myself a few characters by using
> str() instead of repr() everywhere in this thread. :-)

Indeed!

Eric.

From nad at acm.org  Sat Mar  8 03:51:35 2014
From: nad at acm.org (Ned Deily)
Date: Fri, 07 Mar 2014 18:51:35 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com>
 <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com>
Message-ID: <nad-0F1CB1.18513507032014@news.gmane.org>

In article 
<CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw at mail.gmail.com>,
 Guido van Rossum <guido at python.org> 
 wrote:
> On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico 
> <rosuav at gmail.com> wrote:
> > If that's the definition of a float's repr, then I'd support using
> > that by default for the construction of a Decimal from a float. You
> > can always use .as_integer_ratio() to get the exact stored
> > representation.
> Sadly I can't point to exactly where this is documented, but since Python
> 2.7 and some early version of 3.x that is indeed how repr(<float>) works.

If nowhere else:

http://docs.python.org/3/whatsnew/3.1.html#other-language-changes
http://bugs.python.org/issue1580

http://docs.python.org/2/whatsnew/2.7.html#other-language-changes
http://bugs.python.org/issue7117

-- 
 Ned Deily,
 nad at acm.org


From random832 at fastmail.us  Sat Mar  8 04:24:04 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 07 Mar 2014 22:24:04 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
Message-ID: <1394249044.23276.91986741.2D83D0A2@webmail.messagingengine.com>



On Fri, Mar 7, 2014, at 16:24, Andrew Barnert wrote:
> There is no such thing as copying a binary float exactly into a decimal
> float. Decimal float values are not a superset of binary float values; in
> either direction, you have to round. Which is what Python 2.7+ does.

Actually, arbitrary-precision decimal floats are a superset of binary
floats, because 2 is a factor of 10. As the binary floats get smaller
you need more and more digits to represent the decimal though.

>>> decimal.Decimal(1/2**1074)
Decimal('4.940656458412465441765687928682213723650598026143247644255856825006755072702087518652998363616359923797965646954457177309266567103559397963987747960107818781263007131903114045278458171678489821036887186360569987307230500063874091535649843873124733972731696151400317153853980741262385655911710266585566867681870395603106249319452715914924553293054565444011274801297099995419319894090804165633245247571478690147267801593552386115501348035264934720193790268107107491703332226844753335720832431936092382893458368060106011506169809753078342277318329247904982524730776375927247874656084778203734469699533647017972677717585125660551199131504891101451037862738167250955837389733598993664809941164205702637090279242767544565229087538682506419718265533447265625E-324')

From stephen at xemacs.org  Sat Mar  8 04:46:50 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 08 Mar 2014 12:46:50 +0900
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com>
Message-ID: <871tydpd1x.fsf@uwakimon.sk.tsukuba.ac.jp>

random832 at fastmail.us writes:

 > Or, for that matter, you could limit both the numerator and the
 > denominator to some maximum value - when an intermediate result exceeds
 > it (won't be more than the square of the maximum for most operations),
 > you find the closest representable value.

Knuth calls that "fixed slash" (actually somewhat more general than I
think you mean -- numerator and denominator can have different maximum
values).


From paul.dubois at gmail.com  Sat Mar  8 04:55:50 2014
From: paul.dubois at gmail.com (Paul Du Bois)
Date: Fri, 7 Mar 2014 19:55:50 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
Message-ID: <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>

On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org> wrote:

> Or, Decimal(repr(some_float)), which DWIM.
>

Because I haven't seen anyone else bring this up, using a non-exact
conversion breaks the invariant "Decimal(f)==f".

There are so many pairs of numeric types that break that invariant that it
might not be a big deal; but in all other cases the invariant is broken
because it is theoretically impossible.

from decimal import Decimal
f = 2.2
print(Decimal(f)==f)
print(Decimal(repr(f))==f)


p
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/4f52a9e9/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 04:59:11 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 19:59:11 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
Message-ID: <6cf3dba9-ec77-431e-ade6-383b2d09ff80@googlegroups.com>



On Friday, March 7, 2014 8:15:11 PM UTC-6, Guido van Rossum wrote:
>
>
> . . . but my guess is that he's actually trying to argue on behalf of 
> Python users who would benefit from using Decimal but don't want to be 
> bothered with a lot of rules. Telling such people "use the Decimal class" 
> is much easier than telling them "use the Decimal class, and always put 
> quotes around your numbers" (plus the latter advice is still very 
> approximate, so the real version would be even longer and more confusing).
>
>
Yes, that is correct.   Although, I would modify "with certain rules".  I 
mean, its ok to expect some 
rules (I think that's what we mean by semantic).  But rules that seem to go 
against the grain of 
the normal human experience ...  like quoting a numerical input. I mean if 
you tell me as a programmer
that's what I have to do, well that's what I have to do. On the other hand 
(the hand I'm really concerned 
about) if you tell a naive user they have to quote their numerical inputs-- 
that seems like the wrong 
approach.  Yes, its much better to say, "Please use the Decimal module".

marcus

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/20a8d222/attachment-0001.html>

From rosuav at gmail.com  Sat Mar  8 05:00:05 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Mar 2014 15:00:05 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
Message-ID: <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>

On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois <paul.dubois at gmail.com> wrote:
> On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> Or, Decimal(repr(some_float)), which DWIM.
>
>
> Because I haven't seen anyone else bring this up, using a non-exact
> conversion breaks the invariant "Decimal(f)==f".
>
> There are so many pairs of numeric types that break that invariant that it
> might not be a big deal; but in all other cases the invariant is broken
> because it is theoretically impossible.
>
> from decimal import Decimal
> f = 2.2
> print(Decimal(f)==f)
> print(Decimal(repr(f))==f)

How is Decimal==float implemented? Is it by calling Decimal(rhs) and
then comparing? If so, changing how Decimal(float) works won't break
the invariant, as it'll make the same conversion each time.

ChrisA

From guido at python.org  Sat Mar  8 05:11:26 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 20:11:26 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
Message-ID: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>

On Fri, Mar 7, 2014 at 8:00 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois <paul.dubois at gmail.com>
> wrote:
> > On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org>
> wrote:
> >>
> >> Or, Decimal(repr(some_float)), which DWIM.
>
> > Because I haven't seen anyone else bring this up, using a non-exact
> > conversion breaks the invariant "Decimal(f)==f".
> >
> > There are so many pairs of numeric types that break that invariant that
> it
> > might not be a big deal; but in all other cases the invariant is broken
> > because it is theoretically impossible.
> >
> > from decimal import Decimal
> > f = 2.2
> > print(Decimal(f)==f)
> > print(Decimal(repr(f))==f)
>
> How is Decimal==float implemented? Is it by calling Decimal(rhs) and
> then comparing? If so, changing how Decimal(float) works won't break
> the invariant, as it'll make the same conversion each time.
>

There's a special function to convert the other operand for the purpose of
comparisons, and it currently uses Decimal.from_float(). I am not (yet :-)
proposing to change the behavior of that function -- it is and has always
been the API function that does exact float-to-Decimal conversion.
Decimal(repr(f)) == f returns False today (for the majority of float values
anyway) and under my proposal Decimal(f) == f would also return False. I
don't (yet :-) think that's a deal breaker.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/2fb78557/attachment.html>

From rosuav at gmail.com  Sat Mar  8 05:16:06 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Mar 2014 15:16:06 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
Message-ID: <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com>

On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum <guido at python.org> wrote:
>> How is Decimal==float implemented? Is it by calling Decimal(rhs) and
>> then comparing? If so, changing how Decimal(float) works won't break
>> the invariant, as it'll make the same conversion each time.
>
>
> There's a special function to convert the other operand for the purpose of
> comparisons, and it currently uses Decimal.from_float(). I am not (yet :-)
> proposing to change the behavior of that function -- it is and has always
> been the API function that does exact float-to-Decimal conversion.
> Decimal(repr(f)) == f returns False today (for the majority of float values
> anyway) and under my proposal Decimal(f) == f would also return False. I
> don't (yet :-) think that's a deal breaker.

Maybe not a deal breaker, but certainly another panel on the bikeshed.
It would make good sense to retain that invariant by making the
comparison use repr just like the constructor.

ChrisA

From guido at python.org  Sat Mar  8 05:38:54 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 20:38:54 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com>
Message-ID: <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com>

On Fri, Mar 7, 2014 at 8:16 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum <guido at python.org> wrote:
> >> How is Decimal==float implemented? Is it by calling Decimal(rhs) and
> >> then comparing? If so, changing how Decimal(float) works won't break
> >> the invariant, as it'll make the same conversion each time.
>
> > There's a special function to convert the other operand for the purpose
> of
> > comparisons, and it currently uses Decimal.from_float(). I am not (yet
> :-)
> > proposing to change the behavior of that function -- it is and has always
> > been the API function that does exact float-to-Decimal conversion.
> > Decimal(repr(f)) == f returns False today (for the majority of float
> values
> > anyway) and under my proposal Decimal(f) == f would also return False. I
> > don't (yet :-) think that's a deal breaker.
>
> Maybe not a deal breaker, but certainly another panel on the bikeshed.
> It would make good sense to retain that invariant by making the
> comparison use repr just like the constructor.
>

Not sure I agree. I find the invariant Decimal(x) == x rather
uninteresting. I find the idea that Decimal.__eq__ compares the exact
mathematical values very appealing.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/901ba17b/attachment.html>

From tim.peters at gmail.com  Sat Mar  8 05:39:59 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 7 Mar 2014 22:39:59 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
Message-ID: <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>

[Guido]
> Decimal(repr(f)) == f returns False today (for the majority of float values
> anyway) and under my proposal Decimal(f) == f would also return False. I
> don't (yet :-) think that's a deal breaker.

To the contrary, that's "a feature".  Current Pythons take equality
testing seriously, striving to return True when & only when values are
mathematically ("as if using infinite precision") identical, even when
that's inconvenient to do:

>>> 2**500 == 2.0**500
True
>>> 2**500 + 1 == 2.0**500
False

If information is lost when converting, it's doing nobody a favor for
"==" to pretend it hasn't been lost.  So long as Decimal.from_float(f)
== f is still True, then equality would be working as expected in all
cases.

From guido at python.org  Sat Mar  8 05:40:49 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 20:40:49 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com>
 <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com>
Message-ID: <CAP7+vJ+a7xq3-+ZPWRfqQ8TNRR_r8AhaOud3PfMpZYtooLVCrw@mail.gmail.com>

Sometime today I think I wrote that the C implementation of Decimal isn't
yet in the stdlib. I stand corrected. It is. Yay!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/3cf9c040/attachment.html>

From guido at python.org  Sat Mar  8 05:44:18 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 20:44:18 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
Message-ID: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>

On Fri, Mar 7, 2014 at 8:39 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [Guido]
> > Decimal(repr(f)) == f returns False today (for the majority of float
> values
> > anyway) and under my proposal Decimal(f) == f would also return False. I
> > don't (yet :-) think that's a deal breaker.
>
> To the contrary, that's "a feature".  Current Pythons take equality
> testing seriously, striving to return True when & only when values are
> mathematically ("as if using infinite precision") identical, even when
> that's inconvenient to do:
>
> >>> 2**500 == 2.0**500
> True
> >>> 2**500 + 1 == 2.0**500
> False
>
> If information is lost when converting, it's doing nobody a favor for
> "==" to pretend it hasn't been lost.  So long as Decimal.from_float(f)
> == f is still True, then equality would be working as expected in all
> cases.
>

I agree. But what do you think of my main proposal? I would retract it if
you advised me so.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/58ffd2d3/attachment.html>

From tim.peters at gmail.com  Sat Mar  8 06:01:07 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 7 Mar 2014 23:01:07 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
 <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
Message-ID: <CAExdVNkAdkeNdD4JPne12z=DPGmptT6Bqyr5hDeQwo0pqnTR-Q@mail.gmail.com>

[Guido]
> I agree. But what do you think of my main proposal? I would retract it if
> you advised me so.

Oh, I stick to tiny detail these days, to avoid getting sucked into
endless pointless arguments ;-)

That said, I agree that the default Decimal constructor acting like
Decimal(x) == Decimal(repr(x)) would be far less surprising to the
vast majority of users, and no problem at all for "expert" users
(provided Decimal.from_float(x) were still available).

Against it, the change would invalidate dozens of stackoverflow
answers advising users to try printing Decimal(x) to see where their
naive binary floating-point expectations are going wrong.  And I have
no feel for how much production code may be relying on current
Decimal(x) behavior (I have none).

But, if we're willing to risk changing the meaning of
bool(datetime,time(arguments creating an object in some timezones -
but not all - that converts to midnight UTC )), then I suppose the
floodgates are waaaaaay open now ;-)

From abarnert at yahoo.com  Sat Mar  8 06:33:38 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 7 Mar 2014 21:33:38 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
 <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
Message-ID: <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com>

On Mar 7, 2014, at 20:44, Guido van Rossum <guido at python.org> wrote:

> I agree. But what do you think of my main proposal? I would retract it if you advised me so.

I think this may be a good time to raise my points from off-list, in hopes that someone can say "those potential problems are not real problems anyone would ever care about".

Today, given any real, Decimal(float('real')) always gives you the value of the closest binary float to that real. With this change, it will sometimes give you the value of the second-closest repr-of-a-binary-float to that real. This means the error across any range of reals increases. It's still below the rule-of-thumb cutoff that everyone uses for converting through floats, but it is higher by a nonzero amount that doesn't cancel out.

Similarly, today, the distribution of float values across the real number line is... not uniform (because of exponents), and I don't know the right technical term, you know what it looks like. The distribution of repr-of-float variables is not.

Do users of Decimal (or, rather, users of Decimal(float) conversion) care about either of those issues? I don't know. I've never written any code that converted floats to Decimals and then used them as anything other than low-precision fixed-point values (like dollars and cents).

By the way, to dismiss any potential performance worries: with both 3.3 and trunk, Decimal(repr(f)) is actually faster than Decimal.from_float(f) by a factor of 1.3 to 3.2 in a variety of quick tests. If that weren't true, you might have to define Decimal(f) as if it were building and then parsing a string but them implement more complicated code that feeds digits from one algorithm to the other, but fortunately, it looks like this is a three-line change.

From harrismh777 at gmail.com  Sat Mar  8 06:40:47 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 21:40:47 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
Message-ID: <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>



On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote:
 

> The decision was discussed at the time, and all the pros and cons were 
> hashed out. If you're not willing to read that discussion, your opinion 
> that the change was a mistake is worth exactly as much as that of any 
> individual user who asked for the change.
>

hi Andrew,  I have been studying the python-ideas archive & the python-dev 
archive all
night. I have read hundreds of posts. I am finding something very 
interesting. My proposal
has been coming up (time and again) in different flavors for many years; 
 with all the same
people participating (with all of the very same discussion almost verbatim).

Just for history sake, I thought you might be interested in a blast from 
the past from Raymond
Hettinger in response to Lennart Benschop who made the decimal literal 
proposal in Oct, 2007:

Just for historical context only:

On Oct 26, 1:54 am, Lennart Benschop <[hidden email]<http://python.6.x6.nabble.com/user/SendEmail.jtp?type=node&node=1580823&i=0>> 
wrote: 
> My proposal: 
> - Any decimal constant suffixed with the letter "D" or "d" will be 
>   interpreted as a literal of the Decimal type. This also goes for 
>   decimal constants with exponential notation. 

There's nothing new here that hasn't already been proposed and 
discussed on python-dev.  There were no major objections to the idea; 
however, it will need to wait until there is a good C implementation 
of the decimal module (which is in the works but coming along very, 
very slowly). 

{from the history department} 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/92054f2d/attachment-0001.html>

From steve at pearwood.info  Sat Mar  8 07:08:30 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 17:08:30 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
Message-ID: <20140308060830.GX28804@ando>

On Fri, Mar 07, 2014 at 09:40:47PM -0800, Mark H. Harris wrote:

> hi Andrew, I have been studying the python-ideas archive & the 
> python-dev archive all night. I have read hundreds of posts. I am 
> finding something very interesting. My proposal has been coming up 
> (time and again) in different flavors for many years; with all the 
> same people participating (with all of the very same discussion almost 
> verbatim).

Which proposal? You have made a few, and I cannot keep track of which 
ones you still stand by and which ones you have abandoned.

- that Python unify all numeric types to one "Python 
Number" (I think this is abandoned)

- that Decimal(float) use AI to determine what number the programmer 
wanted (you've agreed to stop using the term "AI", but I'm not entirely 
sure whether you've moved away from this position entirely -- you 
still describe Python as needing to "intelligently" convert floats)

- that allowing Decimal(float) at all is a mistake and should be 
prohibited

- that the default numeric type when the programmer enters a numeral 
with a decimal point like 2.01 be Decimal


and possibly others. Which proposal are you referring to here?


> Just for history sake, I thought you might be interested in a blast from 
> the past from Raymond
> Hettinger in response to Lennart Benschop who made the decimal literal 
> proposal in Oct, 2007:
[snip]

Would you mind providing a URL for that message so we can see the 
context please?


-- 
Steven

From tjreedy at udel.edu  Sat Mar  8 07:14:46 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Mar 2014 01:14:46 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
Message-ID: <lfecgo$609$1@ger.gmane.org>

On 3/7/2014 2:20 PM, Guido van Rossum wrote:


> However, for the growing contingent of scientists who use Python as a
> replacement for Matlab (not Mathematica!), it could be a big nuisance.
> They don't care about decimal issues (they actually like binary better)

One reason knowledgeable users of floating point numbers as 
approximations like binary better is that the binary floating point 
system is much 'smoother' than the decimal floating point system. For 
example, with 3 decimal digits, consider .999, 1.00, 1.01. The first 
difference is .001, the second is .01, 10 x larger.  This is one of the 
problems of working with base 10 slide rules. For binary, the largest 
ratio between differences is 2 rather than 10.

> and they write lots of C++ code that interfaces between CPython's
> internal API and various C++ libraries for numeric data processing, all
> of which use IEEE binary.

This too ;-).

-- 
Terry Jan Reedy


From abarnert at yahoo.com  Sat Mar  8 07:18:53 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 7 Mar 2014 22:18:53 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308060830.GX28804@ando>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
Message-ID: <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>

From: Steven D'Aprano <steve at pearwood.info>

Sent: Friday, March 7, 2014 10:08 PM


> On Fri, Mar 07, 2014 at 09:40:47PM -0800, Mark H. Harris wrote:
> 
>>  hi Andrew, I have been studying the python-ideas archive & the 
>>  python-dev archive all night. I have read hundreds of posts. I am 
>>  finding something very interesting. My proposal has been coming up 
>>  (time and again) in different flavors for many years; with all the 
>>  same people participating (with all of the very same discussion almost 
>>  verbatim).
> 
> Which proposal? You have made a few, and I cannot keep track of which 
> ones you still stand by and which ones you have abandoned.

I agree that some clarity would be nice here, but I think from the quote he quoted it's pretty clear that?he's talking about the decimal-literal proposal, not any of the ones that you mentioned.

> - that Python unify all numeric types to one "Python?

> Number" (I think this is abandoned)
> 
> - that Decimal(float) use AI to determine what number the programmer 
> wanted (you've agreed to stop using the term "AI", but I'm not 
> entirely 
> sure whether you've moved away from this position entirely -- you 
> still describe Python as needing to "intelligently" convert floats)


I'm still not sure exactly what Mark's proposal is here, but Guido has developed this one into something concrete and feasible: change the default conversion from float to Decimal (that is, Decimal(float)) to use the repr. I don't want to put works in Mark's mouth and say that's what he's interested in here, but I can say that it looks like it would solve the problem cases Mark has brought up.

> - that allowing Decimal(float) at all is a mistake and should be 
> prohibited
> 
> - that the default numeric type when the programmer enters a numeral 
> with a decimal point like 2.01 be Decimal

From steve at pearwood.info  Sat Mar  8 07:36:04 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 17:36:04 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com>
References: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
 <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
 <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com>
Message-ID: <20140308063604.GY28804@ando>

On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote:

> Today, given any real, Decimal(float('real')) always gives you the 
> value of the closest binary float to that real. With this change, it 
> will sometimes give you the value of the second-closest 
> repr-of-a-binary-float to that real.

Please don't talk about "reals". "Real" has a technical meaning in 
mathematics, and no computer number system is able to represent the 
reals. In Python, we have floats, which are C doubles, and Decimals.

Putting that aside, I'm afraid I don't understand how Guido's suggestion 
to use repr() in the Decimal constructor will sometimes give the 
second-closest value. Can you explain in more detail? If you can show an 
example, that would be great too.


> This means the error across any 
> range of reals increases. It's still below the rule-of-thumb cutoff 
> that everyone uses for converting through floats, but it is higher by 
> a nonzero amount that doesn't cancel out.
>
> Similarly, today, the distribution of float values across the real 
> number line is... not uniform (because of exponents), and I don't know 
> the right technical term, you know what it looks like. The 
> distribution of repr-of-float variables is not.

Nope, sorry, I don't get you.

I think that what you are trying to say is that for each binary 
exponent, the floats with that same exponent are distributed uniformly:

|...|...|...|...|...|...|...|...|

but their reprs are not:

|...|..|..|....|...|....|..|....|

Is this what you mean?



-- 
Steven

From guido at python.org  Sat Mar  8 07:42:20 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Mar 2014 22:42:20 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <CAP7+vJLX8EJn7o+ivMgF1kwTQhwXw4=dBEji6Aq8fNsXqvC2YQ@mail.gmail.com>
 <1E332207-7BA6-4739-8B62-A01D1008885B@yahoo.com>
 <CAP7+vJJ7Qd0LgLmC8RJLdijt=XSpk068eEGd=QkJYwAYa50ZYw@mail.gmail.com>
 <CAP7+vJKayzZfKBTN1=PQ2+92xTREuErpaMKU7FVUMh47EY=nYA@mail.gmail.com>
 <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com>
Message-ID: <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com>

[CC back to the list because you posted the same argument there but without
the numerical example, and my working through that might help others
understand your point]

On Fri, Mar 7, 2014 at 9:18 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> The main point I'm getting at is that by rounding 0.100000000000000012 to
> 0.1 instead of 0.10000000000000000555..., You're no longer rounding it to
> the nearest binary float, but instead to the second nearest
> Decimal(repr(binary float)) (since 0.10000000000000002 is closer than
> 0.1).
>

OK, let me walk through that carefully. Let's name the exact mathematical
values and assign them to strings:

>>> a = '0.100000000000000012'
>>> b = '0.1000000000000000055511151231257827021181583404541015625'
>>> c = '0.10000000000000002'

Today, Decimal(float(a)) == Decimal(b). Under my proposal,
Decimal(float(a)) == Decimal('0.1'). The difference between float('0.1')
and float(c) is 1 ulp (2**-56), and a is between those, but closer to c;
but it is even closer to b (in the other direction). IOW for the
mathematical values, 0.1 < b < a < c, where a is closer to b than to c, So
if the choices for rounding a would be b or c, b is preferred. So far so
good. (And still good if we replace c with the slightly smaller exact value
of float(c).)

And your point is that if we change the allowable choices to '0.1' or c, we
find that float(b) == float('0.1'), but a is closer to c than to 0.1. This
is less than 1 ulp, but more than 0.5 ulp.

I find the argument intriguing, but I blame it more on what happens in
float(a) than in what Decimal() does to the resulting value. If you
actually had the string a, and wanted to convert it to Decimal, you would
obviously write Decimal(a), not Decimal(float(a)), so this is really only a
problem when someone uses a as a literal in a program that is passed to
Decimal, i.e. Decimal(0.100000000000000012).

That's slightly unfortunate, but easy to fix by adding quotes. The only
place where I think something like this might occur in real life is when
someone copies a numerical recipe involving some very precise constants,
and mindlessly applies Decimal() without string quotes to the constants.
But that's a "recipe" for failure anyway, since if the recipe really uses
more precision than IEEE double can handle, *with* the quotes the recipe
would be calculated more exactly anyway. Perhaps another scenario would be
if the constant was calculated (by the recipe-maker) within 0.5 ulp using
IEEE double and rendered with exactly the right number of digits.

But these scenarios sound like either they should use the quotes anyway, or
the calculation would be better off done in double rather than Decimal. So
I think it's still pretty much a phantom problem.


> Of course that's not true for all reals (0.1 being the obvious
> counterexample), but it's true for some with your proposal, while today
> it's true for none. So the mean absolute error in Decimal(repr(f)) across
> any range of reals is inherently higher than Decimal.from_float(f). Put
> another way, you're adding additional rounding error. That additional
> rounding error is still less than the rule-of-thumb cutoff that people use
> when talking about going through float, but it's nonzero and not guaranteed
> to cancel out.
>
> On top of that, the distribution of binary floats is uniform (well, more
> complicated than uniform because they have an exponent as well as a
> mantissa, but you know what I mean); the distribution of closest-repr
> values to binary floats is not.
>
> I have no idea whether either of these are properties that users of
> Decimal (or, rather, Decimal and float together) care about. But they are
> properties that Decimal(float) has today that would be lost.
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/993daf51/attachment-0001.html>

From harrismh777 at gmail.com  Sat Mar  8 07:47:59 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 22:47:59 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com>
Message-ID: <3eb20953-682b-412b-8ea5-bb56a024ade5@googlegroups.com>



On Saturday, March 8, 2014 12:40:36 AM UTC-6, Mark H. Harris wrote:

Steve, Andrew,   here:   
http://osdir.com/ml/python.ideas/2008-01/msg00079.html

Mr Toronto was crazy like me... 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/7e5c37ad/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 07:40:36 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 22:40:36 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308060830.GX28804@ando>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
Message-ID: <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com>



On Saturday, March 8, 2014 12:08:30 AM UTC-6, Steven D'Aprano wrote:
 

> Which proposal are you referring to here? 
>

hi Steven,  well,  all of them. The archive has discussions pertaining to 
every
one of the discussions we have had on this list.  From "python has a bug," 
to 
python needs a decimal literal, to "how come this Decimal(.01) doesn't 
work," to
(believe this or not) one person on the ideas archive actually proposed 
that 
"all python numbers be unified," I kid you not. Guido put that one to bed 
easily
with one post and there was no further discussion of the topic, but the 
idea is
that I'm not the first one to think about these things.  The discussions
are pretty much the same as the ones we've had here on this list with 
people 
frustrated for pretty much the same reasons. ( some of it ignorance, some 
naivet?,
some over-zealous, one or two crazy like me). 

Go here: http://python.6.x6.nabble.com/Python-python-dev-f1801473.html

and search for   D'Aprano decimal    or     D'Aprano Decimal

or search on      unify python numbers


The discussions are very interesting, very educational, and very 
entertaining.

marcus



 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/b15a141c/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 07:52:32 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 22:52:32 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>
Message-ID: <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com>



On Saturday, March 8, 2014 12:18:53 AM UTC-6, Andrew Barnert wrote:
 

> but Guido has developed this one into something concrete and feasible: 
> change the default conversion from float to Decimal (that is, 
> Decimal(float)) to use the repr. 
>

What you and Guido are discussing is perfect.  Thank you.

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/53478a2a/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 08:06:47 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 23:06:47 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308060830.GX28804@ando>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
Message-ID: <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com>



On Saturday, March 8, 2014 12:08:30 AM UTC-6, Steven D'Aprano wrote:
 

> Which proposal? 
>

hi Steve,  the reading I would like to find, and can't, is the discussion 
that led to 
the change in behavior in Py2.7;  where convert from float stopped giving 
the 
(convert to string message).  

I am very interested in reading that discussion.  Do you have a link to 
that?

Thanks

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/eb429934/attachment.html>

From steve at pearwood.info  Sat Mar  8 08:08:40 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 18:08:40 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
Message-ID: <20140308070840.GZ28804@ando>

On Fri, Mar 07, 2014 at 06:02:02PM -0800, Guido van Rossum wrote:
> On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> > [...] The way Python converts between the two is the right
> > way to do the conversion.
> 
> 
> It's *exact*. I don't know we all agree it is the *right* way.

Fair point. I can't argue with that :-)


> > Given a decimal d and a float f constructed
> > from d, f is the closest possible float to d. And the same applies for
> > conversions the other way around.
> >
> 
> It's actually stronger the other way around: when d is constricted from f,
> d is *equal* to the mathematical value of f.

Ah, so it is.


> The issue (as I see it) is that there are many different decimals d that
> all convert to the same float f (because of rounding). The d that is
> constructed by taking the exact value of f is gross overkill. 

Decimal -> float is many-to-one: more than one Decimal will round to a 
single float. But float -> Decimal is always one-to-one, I think, 
regardless of whether you use the current exact conversion or repr 
first. The crux of the matter is whether or not it is overkill for 
Decimal to use the exact value.

Correct me if I'm wrong, but I think we agree that there ought to be a 
way to convert floats exactly to Decimal, we just disagree on whether 
that ought to be spelled Decimal(x) or Decimal.from_float(x).

Likewise I think we agree that there ought to be some way to convert 
floats to the nearest "simple" Decimal, with the question being whether 
that ought to be spelled Decimal(repr(x)) or Decimal(x).

If we're all in agreement that this is the only serious area of 
disagreement for a change which has any hope of appearing in 3.5, then 
we can put aside the more speculative proposals



-- 
Steven

From steve at pearwood.info  Sat Mar  8 08:35:11 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Mar 2014 18:35:11 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com>
References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com>
Message-ID: <20140308073511.GA28804@ando>

On Fri, Mar 07, 2014 at 11:06:47PM -0800, Mark H. Harris wrote:

> hi Steve, the reading I would like to find, and can't, is the 
> discussion that led to the change in behavior in Py2.7; where convert 
> from float stopped giving the (convert to string message).
> 
> I am very interested in reading that discussion.  Do you have a link to 
> that?

http://bugs.python.org/issue8257
https://mail.python.org/pipermail/python-dev/2010-March/098699.html

and in particular:

https://mail.python.org/pipermail/python-dev/2010-March/098717.html


-- 
Steven


From harrismh777 at gmail.com  Sat Mar  8 08:50:38 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Fri, 7 Mar 2014 23:50:38 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com>
Message-ID: <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com>



On Saturday, March 8, 2014 12:52:32 AM UTC-6, Mark H. Harris wrote:
 

> but Guido has developed this one into something concrete and feasible: 
>> change the default conversion from float to Decimal (that is, 
>> Decimal(float)) to use the repr. 
>>
>
> What you and Guido are discussing is perfect.  Thank you.
>


Guido, Andrew,   I have a question about repr() in my current codes for 
 3.3.4 and below:

Would it be reasonable to add a function to pdeclib to be used internally 
like this:

def Dec(numform):
        return Decimal(repr(numform))


Does this work correctly (for pdeclib internally) ?  I think this works 
better than my first
stab at this (just using str() )  because repr() acts like an eval() ??

Do I understand this correctly? 

It looks like what you are proposing will fix the problem I've been whining 
about.  Thank you.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/804b56a7/attachment.html>

From abarnert at yahoo.com  Sat Mar  8 08:53:40 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 7 Mar 2014 23:53:40 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308063604.GY28804@ando>
References: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
 <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
 <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com>
 <20140308063604.GY28804@ando>
Message-ID: <1394265220.1728.YahooMailNeo@web181005.mail.ne1.yahoo.com>

From: Steven D'Aprano <steve at pearwood.info>

Sent: Friday, March 7, 2014 10:36 PM


> On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote:
> 
>>  Today, given any real, Decimal(float('real')) always gives you the 
>>  value of the closest binary float to that real. With this change, it 
>>  will sometimes give you the value of the second-closest 
>>  repr-of-a-binary-float to that real.
> 
> Please don't talk about "reals". "Real" has a technical 
> meaning in 
> mathematics, and no computer number system is able to represent the 
> reals. In Python, we have floats, which are C doubles, and Decimals.


I am using "real" in the technical sense. That's the whole crux of the problem: you want to use the real[1] number 0.1 in Python, and you can't, because there is no IEEE double that matches that value. If we ignore reals and only talk about floats, then 0.1 and?0.100000000000000012 are the same number, so there is no problem.

? ? [1]: I suppose we could talk about rationals instead of reals here, because (a) you can't enter irrationals as literals, and (b) (b) Guido's proposal obviously doesn't attempt to help with them. I don't think that makes a difference here, but I could be wrong; if I am, please point out below where this leads me astray.

Put another way: This cannot be just about converting IEEE double floats to Decimal floats, because Python already does that perfectly. It's about recovering information lost in representing real numbers (or, if you prefer, rational numbers, or Python literals) as IEEE double floats, when converting them to Decimal floats, by taking advantage of the fact that (a)?we know that humans tend to use numbers like 0.1 more often than numbers like?0.100000000000000005551, and (b)?we know the precision of IEEE double and can guarantee that we're not losing any real information.

> Putting that aside, I'm afraid I don't understand how Guido's 
> suggestion 
> to use repr() in the Decimal constructor will sometimes give the 
> second-closest value. Can you explain in more detail? If you can show an 
> example, that would be great too.


OK, take the number?0.100000000000000012.

The closest IEEE double to this number is?0.1000000000000000055511151231257827021181583404541015625. The next closest is 0.10000000000000001942890293094023945741355419158935546875. Today's design gives you the former when you write Decimal(0.100000000000000012). You get the closest one in this case, and in every case.

The closest repr of an IEEE double to this number is?0.10000000000000002. But with Guido's proposal, the one you get is 0.1, which is farther from the number you wanted.

So, if you treat evaluate-literal-as-float-then-Decimal.from_float as a (mathematical) function, it has the property that it always gives you the closest value from its range to your input. If you treat evaluate-literal-as-float-then-repr-then-Decimal as a function, it does not have that property.

>>  This means the error across any?

>>  range of reals increases. It's still below the rule-of-thumb cutoff 
>>  that everyone uses for converting through floats, but it is higher by 
>>  a nonzero amount that doesn't cancel out.

Was this second point clear? In case it wasn't, let me expand on it:

In the example above,?Guido's proposal gives you nearly double the error (1.2e-17 vs. 6.4+e-18) of the current design. Of course there are also cases where it gives a lower error, 0.1 being an obvious example. I'm pretty sure (but I don't know how to prove?) that if you integrate the absolute error over any sufficiently large[1] set of reals, it's higher in Guido's proposal than in the current design.

? ? [1]: The "sufficiently large" there is just so you don't choose a set so small that all of its elements map to a single IEEE double, like { 0.1 <= x <?0.100000000000000001 }.

>>  Similarly, today, the distribution of float values across the real?

>>  number line is... not uniform (because of exponents), and I don't know 
>>  the right technical term, you know what it looks like. The 
>>  distribution of repr-of-float variables is not.
> 
> Nope, sorry, I don't get you.
> 
> I think that what you are trying to say is that for each binary 
> exponent, the floats with that same exponent are distributed uniformly:
> 
> |...|...|...|...|...|...|...|...|
> 
> but their reprs are not:
> 
> |...|..|..|....|...|....|..|....|
> 
> Is this what you mean?


That's part of it.?But on top of that, I think (but again can't prove) that the reprs are on average skewed further to the left than the right, which means that, e.g., the average of a collection of reprs-of-floats will also be skewed in that direction.

So, those are my three points. Remember that I don't know if any of them actually matter to anyone, and in fact am hoping they do not.

From abarnert at yahoo.com  Sat Mar  8 09:16:26 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 00:16:26 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com>
 <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com>
Message-ID: <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com>

From: Mark H. Harris <harrismh777 at gmail.com>
Sent: Friday, March 7, 2014 11:50 PM


>On Saturday, March 8, 2014 12:52:32 AM UTC-6, Mark H. Harris wrote:


>Guido, Andrew, ? I have a question about repr() in my current codes for ?3.3.4 and below:
>
>Would it be reasonable to add a function to pdeclib to be used internally like this:
>
>
>def Dec(numform):
>? ? ? ? return Decimal(repr(numform))


Yes.?If you know that repr(numform) is the string decimal representation that you want for numform, then Decimal(repr(numform)) is guaranteed to be the?decimal?floating-point value that you want. So if numform is a float, in Python 3.3, given your examples, then this is exactly what you want.

>Does this work correctly (for pdeclib internally) ? ?I think this works better than my first

>stab at this (just using str() ) ?because repr() acts like an eval() ??


I think you're a little confused here.

First, in 3.3, for float, repr and str return the same thing. You may be working with other types for which that isn't true?but in that case, you need to figure out which one gives you the decimal representation you want, and use that one, whether it's repr or str (or something else).

Second, repr() doesn't act like an eval(). It's sort of an _inverse_ of eval, for some types.[1] For example,?the repr of Decimal('0.1') is the string "Decimal('0.1')", and eval("Decimal('0.1')") will give you Decimal('0.1').
?
? ? [1] Please don't rely on this. First, it's not true for all types. Second, even if you restrict yourself to types where it is true, repr is not the best way to exchange or persist values, and eval leads to dangerous and fragile code. But this is very useful for things like playing with Python at the interactive interpreter.

>It looks like what you are proposing will fix the problem I've been whining about. ?Thank you.


I think you may be better off using an explicit repr, even if numform is guaranteed to be a float. It makes your intentions clear: you want the one and only Decimal value that matches the decimal string representation of numform, not just arbitrarily any of the many Decimal values that can round-trip back to the float numform, right?

Of course if at all possible, the best thing to do is to avoid creating floats in the first place. Use Decimal('0.1') for constants in your code, use Decimal(user_input) rather than converting user_input with float or eval and then trying to recover the lost information later, etc. But obviously there are cases where you can't do this, because you've, e.g., received the number as a 64-bit IEEE double over the wire from some other program you don't control. That's where Guido's proposal would help you.


From abarnert at yahoo.com  Sat Mar  8 09:32:33 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 00:32:33 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <CAP7+vJLX8EJn7o+ivMgF1kwTQhwXw4=dBEji6Aq8fNsXqvC2YQ@mail.gmail.com>
 <1E332207-7BA6-4739-8B62-A01D1008885B@yahoo.com>
 <CAP7+vJJ7Qd0LgLmC8RJLdijt=XSpk068eEGd=QkJYwAYa50ZYw@mail.gmail.com>
 <CAP7+vJKayzZfKBTN1=PQ2+92xTREuErpaMKU7FVUMh47EY=nYA@mail.gmail.com>
 <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com>
 <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com>
Message-ID: <1394267553.65888.YahooMailNeo@web181006.mail.ne1.yahoo.com>

From: Guido van Rossum <guido at python.org>
Sent: Friday, March 7, 2014 10:42 PM


>[CC back to the list because you posted the same argument there but without the numerical example, and my working through that might help others understand your point]

Thank you for presenting my point better than I could; it's a lot clearer this way.

>On Fri, Mar 7, 2014 at 9:18 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>
>The main point I'm getting at is that by rounding?0.100000000000000012 to 0.1 instead of?0.10000000000000000555..., You're no longer rounding it to the nearest binary float, but instead to the second nearest Decimal(repr(binary float)) (since?0.10000000000000002 is closer than 0.1).
>
>OK, let me walk through that carefully. Let's name the exact mathematical values and assign them to strings:
>
>>>> a = '0.100000000000000012'
>>>> b = '0.1000000000000000055511151231257827021181583404541015625'
>>>> c = '0.10000000000000002'
>
>Today, Decimal(float(a)) == Decimal(b). Under my proposal, Decimal(float(a)) == Decimal('0.1'). The difference between float('0.1') and float(c) is 1 ulp (2**-56), and a is between those, but closer to c; but it is even closer to b (in the other direction). IOW for the mathematical values, 0.1 < b < a < c, where a is closer to b than to c, So if the choices for rounding a would be b or c, b is preferred. So far so good. (And still good if we replace c with the slightly smaller exact value of float(c).)
>
>And your point is that if we change the allowable choices to '0.1' or c, we find that float(b) == float('0.1'), but a is closer to c than to 0.1. This is less than 1 ulp, but more than 0.5 ulp.

Yes. It's the same two problems that inspired Clinger's correct rounding papers[1][2]: it does not have the closest-match property, and it can lose?almost twice as much accuracy. But the context is very different, so I'm not sure Clinger's arguments are relevant here.

? ? [1]:?http://citeseer.ist.psu.edu/william90how.html
? ? [2]:?ftp://ftp.ccs.neu.edu/pub/people/will/retrospective.pdf

>I find the argument intriguing, but I blame it more on what happens in float(a) than in what Decimal() does to the resulting value. If you actually had the string a, and wanted to convert it to Decimal, you would obviously write Decimal(a), not Decimal(float(a)), so this is really only a problem when someone uses a as a literal in a program that is passed to Decimal, i.e. Decimal(0.100000000000000012).

Agreed on both counts. However, the entire problem you're trying to solve here is caused by what happens in float(a). You're effectively attempting to recover the information lost in float() in Decimal(), in a way that often does what people want, and otherwise never does anything too bad.

So long as giving up the correct-rounding property,?doubling the error (but still staying under 1 ulp), and skewing the distribution of Decimals created this way (by less than 0.5 ulp, but possibly in a way that can accumulate) are not "too bad", I believe your proposal succeeds completely.

>That's slightly unfortunate, but easy to fix by adding quotes.

Yes, but the motivating example to this whole thread, Decimal(1.1), is just as easy to fix by using quotes.

I think I can see the distinction: Novices don't know to use quotes; people trying to implement numerical recipes in Python do (or at least really, really should); therefore a change that helps the former but hurts the latter, when they both leave off the quotes, is a net gain. Yes?

From dickinsm at gmail.com  Sat Mar  8 10:43:58 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 8 Mar 2014 09:43:58 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
Message-ID: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 12:10 AM, Guido van Rossum <guido at python.org> wrote:

>
> But Decimal(<float>) is relatively new (it was an error in 2.6). I know
> it's annoying to change it again, but I also have the feeling that there's
> not much use in producing a Decimal with 53 (or more?) *decimal digits* of
> precision from a float with 53 *bits* -- at least not by default. Maybe we
> can relent on this guarantee and make the constructor by default use fewer
> digits (e.g. using the same algorithm used by repr(<float>)) and have an
> optional argument to produce the full precision? Or reserve
> Decimal.from_float() for that?
>

Using the same algorithm as used by repr(float) feels like a bad idea to
me:  my gut feeling is that a conversion from float to Decimal is a
fundamental operation that should be easily describable in simple terms,
and should not involve the level of complication and 'magic' involved in
the float repr.  (That level of magic seems fine for the repr, because in
most cases you don't actually care too much what string you get so long as
it evaluates back to the correct float;  you're not doing arithmetic with
the result.)

IEEE 754-2008 requires simply that conversions between numbers in different
formats round correctly.  It specifies a "convertFormat" operation,
described as follows:

"""
If the conversion is to a format in a different radix or to a narrower
precision in the same radix, the result shall be rounded as specified in
Clause 4. Conversion to a format with the same radix but wider precision
and range is always exact. ...
"""

... where clause 4 is the one that outlines the general rounding rules that
apply to almost all IEEE 754 operations.

I see three sane options for float to Decimal conversion:

1. Raise an exception.
2. Round to the nearest Decimal value using the current context for that
round operation.
3. Do what we're currently doing, and do an exact conversion.

Option 2 would seem closest to what IEEE 754 specifies.  To augment option
2, we could also make it easier to create Decimal contexts corresponding to
the IEEE 754 recommended interchange formats (decimal32, decimal64,
decimal128).  This was already proposed in http://bugs.python.org/issue8786.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/0f2441de/attachment.html>

From dickinsm at gmail.com  Sat Mar  8 11:01:47 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 8 Mar 2014 10:01:47 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
Message-ID: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

>
> I see three sane options for float to Decimal conversion:
>
> 1. Raise an exception.
> 2. Round to the nearest Decimal value using the current context for that
> round operation.
> 3. Do what we're currently doing, and do an exact conversion.
>

Proposals for change should also take into consideration that Decimal
already does *exact* conversions for integers (and I believe has done since
it first existed).  It would be quite surprising for `Decimal(2**1000)` and
`Decimal(2.0**1000)` to be different numbers.  If we change the float
behaviour, we might also want to change conversion from int to round to the
nearest Decimal using the current context.  Again, that's closer to what
IEEE 754 specifies for the "convertFromInt" operation.

On the other hand, we probably shouldn't lend *too* much weight to IEEE
754, especially when talking about choice of precision.  IEEE 754 isn't a
perfect fit for Decimal:  the IEEE standard is mostly concerned with fixed
width decimal formats, which is subtly different from Mike Cowlishaw's
approach of "extensible precision" where the precision is not so closely
tied to the format.  Python's decimal module is based on Cowlishaw's
standard, not on IEEE 754.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/c3bff20b/attachment.html>

From eric at trueblade.com  Sat Mar  8 12:20:22 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Sat, 08 Mar 2014 06:20:22 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com>
 <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com>
 <20140308013605.GW28804@ando>
 <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com>
 <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com>
 <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com>
 <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com>
 <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com>
 <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com>
Message-ID: <531AFCF6.5070901@trueblade.com>

On 3/7/2014 11:44 PM, Guido van Rossum wrote:
> I agree. But what do you think of my main proposal? I would retract it
> if you advised me so.

I'd like to hear what Mark Dickinson has to say. I assume like most
people he lost interest in this thread. I've sent him a message asking
him to take a look.

Eric.


From solipsis at pitrou.net  Sat Mar  8 13:01:47 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 8 Mar 2014 13:01:47 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <lfecgo$609$1@ger.gmane.org>
Message-ID: <20140308130147.1dcfbc74@fsol>

On Sat, 08 Mar 2014 01:14:46 -0500
Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/7/2014 2:20 PM, Guido van Rossum wrote:
> 
> 
> > However, for the growing contingent of scientists who use Python as a
> > replacement for Matlab (not Mathematica!), it could be a big nuisance.
> > They don't care about decimal issues (they actually like binary better)
> 
> One reason knowledgeable users of floating point numbers as 
> approximations like binary better is that the binary floating point 
> system is much 'smoother' than the decimal floating point system. For 
> example, with 3 decimal digits, consider .999, 1.00, 1.01. The first 
> difference is .001, the second is .01, 10 x larger.  This is one of the 
> problems of working with base 10 slide rules. For binary, the largest 
> ratio between differences is 2 rather than 10.

Well, can you explain what difference it does in practice?

Regards

Antoine.



From ron3200 at gmail.com  Sat Mar  8 15:05:48 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 08 Mar 2014 08:05:48 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
Message-ID: <lff83f$9ls$1@ger.gmane.org>



On 03/07/2014 01:01 PM, Mark H. Harris wrote:
> The point here is that binary floats should not be promoted to decimal
> floats by using "exact" copy from binary float representation to decimal
> float representation.

In the case of writing out a number.  Decimal(2.1),  A decimal literal 
solves that case since no floats are involved.  It would be the same as the 
exact copy, but that is perfectly expected as well.  (and already works 
when a string is used in the constructor.


Regarding not raising an error when a float is used directly in the decimal 
constructor:  As wrong as it may seem at first, also consider python does 
not raise an error for this.

    >>> int(.1)
    0

The users is expected to know that the result will not always equal to what 
is given.  It's just really obvious in this case because, this is what this 
function has always done, and what we expect it to do.

The decimal case isn't that different except that it isn't so obvious. 
What that means is the docs needs be adjusted to make it more easily 
findable...  But currently...



Help on class Decimal in module decimal:

class Decimal(builtins.object)
  |  Decimal(value="0", context=None): Construct a new Decimal object.
  |  value can be an integer, string, tuple, or another Decimal object.
  |  If no value is given, return Decimal('0'). The context does not affect
  |  the conversion and is only passed to determine if the InvalidOperation
  |  trap is active.
  |


Which says nothing about using floats, and should if it is allowed.  It 
does talk about floats in the from_float method including comments on exact 
representation.


Help on built-in function from_float:

from_float(...) method of builtins.type instance
     from_float(f) - Class method that converts a float to a decimal 
number, exactly.
     Since 0.1 is not exactly representable in binary floating point,
     Decimal.from_float(0.1) is not the same as Decimal('0.1').

         >>> Decimal.from_float(0.1)
         Decimal('0.1000000000000000055511151231257827021181583404541015625')
         >>> Decimal.from_float(float('nan'))
         Decimal('NaN')
         >>> Decimal.from_float(float('inf'))
         Decimal('Infinity')
         >>> Decimal.from_float(float('-inf'))
         Decimal('-Infinity')



So I think the first case should say, it uses from_float() when a floating 
point is used, and explicitly also say to look at the from_float docs.  (or 
better yet include them directly in the class doc string.


As far as Ulps go.  Unless the average error can be reduced I don't see any 
reason to change it.  If the docs say it's uses from_float and from_float 
gives the same result.  I think it's perfectly reasonable and likely to 
answer most users questions where they expect to find the answer.

Cheers,
     Ron




























From dickinsm at gmail.com  Sat Mar  8 15:15:59 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 8 Mar 2014 14:15:59 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308130147.1dcfbc74@fsol>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <lfecgo$609$1@ger.gmane.org> <20140308130147.1dcfbc74@fsol>
Message-ID: <CAAu3qLUpoGB2SPOqZApvchqyr3jYFETOEtJ9SD+0QZ+qrvFLdw@mail.gmail.com>

On Sat, Mar 8, 2014 at 12:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> > problems of working with base 10 slide rules. For binary, the largest
> > ratio between differences is 2 rather than 10.
>
> Well, can you explain what difference it does in practice?
>

Probably not much that the average user would care about, but there are a
whole host of 'nice' properties that work in binary floating-point but not
in decimal.  For example, in binary, assuming IEEE 754, round-to-nearest,
etc., you're guaranteed that for any two representable floats x and y, the
"average" (x+y)/2 lies in the interval [x, y] (even strictly between x and
y provided that x and y are at least 2 ulps apart), so a naively written
floating-point bisection search will converge.  In decimal that's not true:
you can lose a whole digit of precision when adding x and y and end up with
a result that's outside [x, y].

>>> from decimal import *

>>> getcontext().prec = 3

>>> x = Decimal('0.516')

>>> y = Decimal('0.518')

>>> (x + y) / 2

Decimal('0.515')  # ouch!

Then if you're doing numerical analysis and error computations, the
"wobble" (the variation in scale of the ratio between the mathematical
relative error and the error expressed in ulps) is 2 for binary, 10 for
decimal.  That makes for weaker error bounds and faster-growing errors for
operations done in decimal floating-point rather than binary.  (And it's
even worse for hexadecimal floating-point, which is why IEEE 754 is a big
improvement over IBM's hex float format.)

Binary is just better for serious numerical work. :-)

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/1b03b50c/attachment.html>

From stefan at bytereef.org  Sat Mar  8 15:18:20 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 8 Mar 2014 15:18:20 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
Message-ID: <20140308141820.GA20171@sleipnir.bytereef.org>

Mark Dickinson <dickinsm at gmail.com> wrote:
> I see three sane options for float to Decimal conversion:
> 
> 1. Raise an exception.
> 2. Round to the nearest Decimal value using the current context for that round
> operation.
> 3. Do what we're currently doing, and do an exact conversion.

I agree that these are the main reasonable options.  All functions
in Cowlishaw's specification use all digits from overlong input
operands, so there should be a convenient way of creating such
operands exactly.

I think we should not make an exeception for floats, unless we move into
the direction of IEEE 754-2008 (but that surely is a different topic).


Some general comments:

The current model works well, since we already have a fallback
constructor that rounds:

>>> from decimal import *
>>> context = getcontext()
>>> Decimal(1.1)
Decimal('1.100000000000000088817841970012523233890533447265625')
>>> context.create_decimal(1.1)
Decimal('1.10')
>>>


If you want safety against accidental conversions, set FloatOperation:

>>> context.traps[FloatOperation] = True
>>> Decimal(1.1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
decimal.FloatOperation: [<class 'decimal.FloatOperation'>]


If you want extra safety against *any* accidental float comparisons,
check the flags manually. This is necessary, since equality operations
cannot raise due to the fact that they are also used for membership
testing:

>>> context.clear_flags()
>>> Decimal(9) in [9.0]
True
>>> if context.flags[FloatOperation]:
...     raise FloatOperation
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
decimal.FloatOperation


You see that the float operation is still recorded, even though it
isn't raised automatically.  Such a check could be done periodically
or at the end of a program.


To summarize, I think we should leave things as they are or turn on
FloatOperation by default.


Stefan Krah




From harrismh777 at gmail.com  Sat Mar  8 17:13:39 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 08:13:39 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com>
 <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com>
 <20140308060830.GX28804@ando>
 <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com>
 <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com>
 <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com>
 <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com>
Message-ID: <8541b8b9-95a4-4897-ac24-2ea0e6fa4184@googlegroups.com>



On Saturday, March 8, 2014 2:16:26 AM UTC-6, Andrew Barnert wrote:
>
> I think you're a little confused here.
>
>  repr() doesn't act like an eval(). It's sort of an _inverse_ of eval, for 
> some types.[1] For example, the repr of Decimal('0.1') is the string 
> "Decimal('0.1')", and eval("Decimal('0.1')") will give you Decimal('0.1').
>
You are correct, I was confused about the inverse relationship, for some 
types, because the http doc says:

                 For many types, this function makes an attempt to return a 
string that would 

                  yield an object with the same value when passed to eval(),

But,  the system doc help() makes things more clear:

repr(...)
    repr(object) -> string
    
    Return the canonical string representation of the object.
    For most object types, eval(repr(object)) == object.



Thank you,  your explanation of the caveats for repr() were helpful to me.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/5a735ce8/attachment.html>

From guido at python.org  Sat Mar  8 17:54:24 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Mar 2014 08:54:24 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
Message-ID: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>

I'll try to respond to Mark Dickinson's second message (and nothing else
that happens in the thread since last night), because (a) it concisely
summarizes his position and (b) brings up a new strawman.

On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>
>>
>> I see three sane options for float to Decimal conversion:
>>
>> 1. Raise an exception.
>> 2. Round to the nearest Decimal value using the current context for that
>> round operation.
>> 3. Do what we're currently doing, and do an exact conversion.
>>
>
I think you're writing this entirely from the POV of an expert in floating
point. And I'm glad we have experts like you around! I don't consider
myself an expert at all, but I do think I have something to bring to the
table -- insight the experience of non-expert users.

When a non-expert writes Decimal(1.1), each of the three above outcomes
surprises. We know that (1) was unpopular, that's why we changed it. We now
know that (3) is unpopular at least in some circles (Mark Harrison can't be
the only one who doesn't like it). Changing to (2) wouldn't do much to
address this, because the default context has way more precision than
float, so it still shows a lot of extraneous digits.

Yes, it's trivial to get rid of those extra digits, just use quotes. But if
*my* proposal were adopted, it would be trivial for numerical experts to
get the extra digits -- just use from_float(). At this point, my claim is
that we're talking essentially about what is the better experience for most
users, and while I am not much of a user of Decimal myself, I believe that
my proposal has benefits more people and situations than it has downsides.


> Proposals for change should also take into consideration that Decimal
> already does *exact* conversions for integers (and I believe has done since
> it first existed).  It would be quite surprising for `Decimal(2**1000)` and
> `Decimal(2.0**1000)` to be different numbers.
>

This feels like a strawman, since Decimal(2**1000 + 1) and
Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot
of digits but is one too small), and the similar examples with a large
exponent (10000) differ dramatically (the float version raises an
exception).

For most purposes it's a fluke that 2.0**1000 can be represented exactly as
a float, and the argument doesn't convince me at all. There are just too
many odd examples like that, and it always will remain a minefield.


> If we change the float behaviour, we might also want to change conversion
> from int to round to the nearest Decimal using the current context.  Again,
> that's closer to what IEEE 754 specifies for the "convertFromInt" operation.
>

I don't think that's no the table. Python's int doesn't lose precision, and
users know this and depend on it. (In fact, I find ignoring the current
context in the constructor a totally reasonable approach -- it does this
uniformly, regardless of the argument type. My proposal doesn't change
this, the context would *still* not be used to decide how a float is
converted. Only the fact that it's a float would.)


>
> On the other hand, we probably shouldn't lend *too* much weight to IEEE
> 754, especially when talking about choice of precision.  IEEE 754 isn't a
> perfect fit for Decimal:  the IEEE standard is mostly concerned with fixed
> width decimal formats, which is subtly different from Mike Cowlishaw's
> approach of "extensible precision" where the precision is not so closely
> tied to the format.  Python's decimal module is based on Cowlishaw's
> standard, not on IEEE 754.
>

I wonder what Cowlishaw would say about our current discussion. He is also
the father of Rexx...

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/a05e7e03/attachment.html>

From skreft at gmail.com  Sat Mar  8 17:57:45 2014
From: skreft at gmail.com (Sebastian Kreft)
Date: Sat, 8 Mar 2014 17:57:45 +0100
Subject: [Python-ideas] Enhance exceptions by attaching some more
 information to them
In-Reply-To: <530210C5.8000406@btinternet.com>
References: <CAFuVyPkZ+0hPkTKsh4M7r6veCj5Dq0J7i1LL=V5DdESm1iQx9Q@mail.gmail.com>
 <530210C5.8000406@btinternet.com>
Message-ID: <CAFuVyPkJxUti_HDH4EQsPRFkC_dYL7Q6kFF2ogPeoxCJLucyZg@mail.gmail.com>

Thanks for the comments, I will start working on a PEP for Step 1.


On Mon, Feb 17, 2014 at 2:38 PM, Rob Cliffe <rob.cliffe at btinternet.com>wrote:

>  +1 on the idea (I can't comment on the implementation).  One of Python's
> great strengths is its excellent error messages.  But if they can be made
> still better, that would be fantastic.
> Rob Cliffe
>
>
> On 15/02/2014 15:40, Sebastian Kreft wrote:
>
> More than once I've been in a situation where I wish that some of the
> stdlib exceptions had a better message or some more information to help me
> diagnose the problem.
>
>  For example:
> a = [1, 2, 3, 4, 5]
>  a[5]
>  IndexError: list index out of range
>
>  In this case there's no reference to neither the length of the array nor
> to the offending index.
>
>  I'm of the idea that we could extend the exceptions by adding some more
> information to them, so 3rd party libraries could use them for
> debugging/logging.
>
>  For example, one such use case would be to have a modified test runner,
> that in case of exceptions automatically prints some debug information.
> Another would be a logger system that in case of an exception also logs
> some debug info that could be relevant to understand and solve the issue.
>
>  I propose extending (at least) the following exceptions with the
> following attributes:
> KeyError: key, object
> IndexError: index, object
> AttributeError: attribute, object
> NameError: name
>
>  Of course that populating these attributes could be controlled by a flag.
>
>  I know that some of this information is already present in some
> exceptions, depending on the context. However, I propose adding these
> attributes, as in this way a tool could easily and reliably extract the
> information and work with it, as opposed to have to parse the huge variety
> of different messages there are.
>
>  For the first use case mentioned above I have a working prototype,
> although it does not use this proposal, but a series of hacks (I'm
> modifying the bytecode to save a reference to the key and object :() and
> parsing of the exception messages. But I want to share what the output of
> such a tool could be.
>
>  ======================================================================
> ERROR: test_attribute (example.ExampleTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/skreft/test/debug_exception/example.py.py", line 18, in
> test_attribute
> AttributeError: 'str' object has no attribute 'Lower'. Did you mean
> 'islower', 'lower'?
> Debug info:
>     Object: ''
>     Type: <type 'str'>
>
>  ======================================================================
> ERROR: test_index (example.ExampleTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/skreft/test/debug_exception/example.py.py", line 6, in
> test_index
> IndexError: list index out of range
> Debug info:
>     Object: [1, 2]
>     Object len: 2
>     Index: 2
>
>  ======================================================================
> ERROR: test_key (example.ExampleTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/skreft/test/debug_exception/example.py.py", line 10, in
> test_key
> KeyError_: 'fooo', did you mean 'foo'?
> Debug info:
>     Object: {'foo': 1}
>     Key: 'fooo'
>
>  ======================================================================
> ERROR: test_name (example.ExampleTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/skreft/test/debug_exception/example.py.py", line 14, in
> test_name
> NameError: global name 'fooo' is not defined. Did you mean 'foo'?
>
>  ----------------------------------------------------------------------
> Ran 4 tests in 0.005s
>
>  --
> Sebastian Kreft
>
>
> _______________________________________________
> Python-ideas mailing listPython-ideas at python.orghttps://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Sebastian Kreft
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ab1431dd/attachment-0001.html>

From ron3200 at gmail.com  Sat Mar  8 18:35:34 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 08 Mar 2014 11:35:34 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308070840.GZ28804@ando>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <20140308070840.GZ28804@ando>
Message-ID: <lffkco$7rl$1@ger.gmane.org>



On 03/08/2014 01:08 AM, Steven D'Aprano wrote:
> On Fri, Mar 07, 2014 at 06:02:02PM -0800, Guido van Rossum wrote:
>> >On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano<steve at pearwood.info>  wrote:


>> >The issue (as I see it) is that there are many different decimals d that
>> >all convert to the same float f (because of rounding). The d that is
>> >constructed by taking the exact value of f is gross overkill.

> Decimal -> float is many-to-one: more than one Decimal will round to a
> single float. But float -> Decimal is always one-to-one, I think,
> regardless of whether you use the current exact conversion or repr
> first. The crux of the matter is whether or not it is overkill for
> Decimal to use the exact value.

> Correct me if I'm wrong, but I think we agree that there ought to be a
> way to convert floats exactly to Decimal, we just disagree on whether
> that ought to be spelled Decimal(x) or Decimal.from_float(x).


Decimal(x),  just like you do... float(x) when x is an int.


I think the from_float on the decimal type was an attempt to solve a 
problem that should have been solved by more explicit docs on the decimal 
class.  A constructor should call the type to_type method when it's higher 
precision type.

       float(decimal('2.1'))    # float call's decimals to_float() method.

       int(decima('2.1'))       # int call's decimals to_int() mehtod. *1


    1.[* or __int__]


Going in the other direction is different.  They aren't symmetric 
operations and to think they are is a mistake.


       float(int('2.1'))       # float knows how to convert ints.

       decimal(float('2.1))    # decimal knows how to convert floats.



What this says if an object can supply it's own converter for less accurate 
types, but should ask the other how to covert for more accurate types.  (or 
ask if it doesn't know how to covert.)

So the decimal.from_float method is redundant, as it was combined into the 
constructor.  (which is the more consistent to python interface. I'm not 
sure if that was the main reason, but it makes sense to me.)


> Likewise I think we agree that there ought to be some way to convert
> floats to the nearest "simple" Decimal, with the question being whether
> that ought to be spelled Decimal(repr(x)) or Decimal(x).

It should be this...  I'm absolutely certain! :-)

      Decimal(str(n))

No problems with that as long as it's an explicit choice.


And by adding a decimal literal, we avoid the float to decimal conversion 
completely for setting decimal value constants and entering in raw data by 
hand.

     2.1d

The use of str(n) above is more about getting a human readable form.


When converting the internal value of a float to a decimal, it should equal 
the floats exact value.  A repr should give the exact value its object if 
it's suppose to be a machine readable version of it.  (As numbers __repr__ 
should do.)

The __str__ method should be the human readable version.  Possibly the 
console can have a setting to use str() in place of repr().  So when you 
just use the console as a calculator, it will work more like one.

In other words, don't fix decimal, because floats repr isn't showing it's 
exact value.  That's really an issue with float, not decimal.  And don't 
use repr() if you want a nice human readable value,  use str().



> If we're all in agreement that this is the only serious area of
> disagreement for a change which has any hope of appearing in 3.5, then
> we can put aside the more speculative proposals

The following seem like important relationships to me.


For all numbers (n), with exact representation in int.

       '1' -> int -> float -> decimal    (exact)
       decimal' -> float -> int -> '1'   True

       >>> n = D(float(int(1)))
       >>> int(float(n))
       1


For all numbers (n), with exact representations in float.

       n -> float -> decimal      (exact)
       decimal -> float -> n      True

       >>> n = D(float('2.5'))
       >>> float(n)
       2.5

Is this valid in python 3.4?


When going from higher precision to lower precision values, (as stored 
internally), the best answer is to use the best value with the lowest 
possible error.  (Not necessarily the one that looks the nicest.)

I say potential, because if you don't know what the error may be, you 
shouldn't guess.  It's possible someone somewhere is studying exact float 
representations as a set.  ;-)

Data input inaccuracies are for the user to determine and handle,  but 
python should supply the tools to help them do that.  It just shouldn't do 
it prematurely.

Cheers,
    Ron










From rosuav at gmail.com  Sat Mar  8 18:43:56 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 04:43:56 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lffkco$7rl$1@ger.gmane.org>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org>
Message-ID: <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com>

On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam <ron3200 at gmail.com> wrote:
> A repr should give the exact value its object if it's suppose to be a
> machine readable version of it.  (As numbers __repr__ should do.)

As I understand it, float.__repr__ does indeed give the exact value,
in terms of reconstructing the float. There are infinitely many float
literals that will result in the exact same bit pattern, so any of
them is valid for repr(n) to return.

>>> 1.233999999999999985 == 1.234
True
>>> repr(1.233999999999999985)
'1.234'

ChrisA

From ron3200 at gmail.com  Sat Mar  8 19:10:45 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 08 Mar 2014 12:10:45 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org>
 <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com>
Message-ID: <lffmen$to0$1@ger.gmane.org>



On 03/08/2014 11:43 AM, Chris Angelico wrote:
> On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com>  wrote:

>> >A repr should give the exact value its object if it's suppose to be a
>> >machine readable version of it.  (As numbers __repr__ should do.)


> As I understand it, float.__repr__ does indeed give the exact value,
> in terms of reconstructing the float.

What I'm thinking about is...

If floats repr is changed to disregard exta digits, will this still be 
true?  How is float to know what exta digits should be disregarded?


> There are infinitely many float literals that will result in the exact
> same bit pattern, so any of them is valid for repr(n) to return.

When are float literals actually converted with floats.  It seems to me, 
that the decimal functions can be used to get the closest one.  Then they 
will be consistent with each other. (if that isn't already being done.)

Cheers,
    Ron




From oscar.j.benjamin at gmail.com  Sat Mar  8 19:11:54 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 8 Mar 2014 18:11:54 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
Message-ID: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>

On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote:
> I'll try to respond to Mark Dickinson's second message (and nothing else
> that happens in the thread since last night), because (a) it concisely
> summarizes his position and (b) brings up a new strawman.
>
> On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>
>> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>>
>>>
>>> I see three sane options for float to Decimal conversion:
>>>
>>> 1. Raise an exception.
>>> 2. Round to the nearest Decimal value using the current context for that
>>> round operation.
>>> 3. Do what we're currently doing, and do an exact conversion.
>
> I think you're writing this entirely from the POV of an expert in floating
> point. And I'm glad we have experts like you around! I don't consider myself
> an expert at all, but I do think I have something to bring to the table --
> insight the experience of non-expert users.

Standards compliance is important though. Mark is essentially
restating something that is repeated hundreds of times throughout
various floating point standards documents: a result may be exact or
it must be correctly rounded according to the current context (with
appropriate flags set and traps fired).

It is not mandated that results requiring more precision than the
current context be exact. It is mandated that if a result is to be
inexact then the implementation must use a very specific type of
inexactness. I don't believe that a standards-compliant decimal module
has any wiggle room to invent a new kind of rounding (which I think
would be required to achieve what you suggest).

> When a non-expert writes Decimal(1.1), each of the three above outcomes
> surprises. We know that (1) was unpopular, that's why we changed it. We now
> know that (3) is unpopular at least in some circles (Mark Harrison can't be
> the only one who doesn't like it). Changing to (2) wouldn't do much to
> address this, because the default context has way more precision than float,
> so it still shows a lot of extraneous digits.
>
> Yes, it's trivial to get rid of those extra digits, just use quotes. But if
> *my* proposal were adopted, it would be trivial for numerical experts to get
> the extra digits -- just use from_float(). At this point, my claim is that
> we're talking essentially about what is the better experience for most
> users, and while I am not much of a user of Decimal myself, I believe that
> my proposal has benefits more people and situations than it has downsides.

If you write Decimal(1.1) and are surprised by the result then you
have misunderstood something. It may be that you have little
understanding of the difference between binary and decimal floating
point (but then why are you using Decimal?). Perhaps you don't fully
understand the literal->float->Decimal pathway that occurs when the
expression is evaluated because you're new to Python or just haven't
really thought about it before.

In any case if the result of Decimal(1.1) surprises you then it's
because you're expecting it do something that should be done in a
different way. Hiding the extra digits does not help a user to
understand how to use Decimal.

I actually use this in teaching to demonstrate how binary floating
point works. I think it's important when teaching my students for them
to understand that the following is a lie:

    >>> a = 1.1
    >>> a
    1.1

The helpful digit-hiding repr is lying to you. There is no float with
the value 1.1. I can sort-of demonstrate this with some arithmetic:

    >>> 0.1 + 0.11 - 0.11 - 0.1
    1.3877787807814457e-17

But that gives the misleading impression that inexact arithmetic is
the source of the error. It's important to understand that the error
occurs straight away in the literal "1.1". I demonstrate this by
showing that

    >>> from decimal import Decimal
    >>> a = 1.1
    >>> Decimal(a)
    Decimal('1.100000000000000088817841970012523233890533447265625')

which also shows the true value stored in the float.

The fact that I use this in teaching is not supposed to serve as an
argument for keeping it (I could just as easily use from_float). My
point is that showing the digits helps someone to understand what is
going on. If a user is converting float to Decimal without knowing
what they're doing then the extra digits are a clue that they don't
fully understand what's happening and haven't necessarily used the
best approach.

There is a good solution to the problem of non-experts wanting to
write 1.1 and get the exact value 1.1: decimal literals. With that
they can just write 1.1d and not need to learn any more about it.
Earlier in this thread you reject that idea saying that you can't
teach it to "newbies":
'''
Maybe we can add a new literal notation meaning 'decimal'; that would
be a relatively straightforward change to the parser (once the new
decimal extension module is incorporated), but it would not do much to
avoid surprising newbies (certainly you can't go teaching them to
always write 3.14d instead of 3.14). However, it would probably help
out frequent users of Decimal. (Then again, they might not be using
literals that much -- I imagine the source of most such programs is
user or file input.)
'''
I disagree. If you're at the point of wanting to use the Decimal
module then you're at the point where it's reasonable to learn about
Decimal literals.

Also I've written code using the decimal module for high precision
calculation and it has lots of decimal literals. That is I do (and I
see other's doing)

    >>> from decimal import Decimal as D
    >>> d = D('1e-20')

even though this is never used in the decimal documentation. This is
the closest you can get to decimal literals right now.


Oscar

From abarnert at yahoo.com  Sat Mar  8 19:43:23 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 10:43:23 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lffmen$to0$1@ger.gmane.org>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org>
 <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com>
 <lffmen$to0$1@ger.gmane.org>
Message-ID: <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com>

From: Ron Adam <ron3200 at gmail.com>

Sent: Saturday, March 8, 2014 10:10 AM


> On 03/08/2014 11:43 AM, Chris Angelico wrote:
>>  On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com>? wrote:
> 
>>>  >A repr should give the exact value its object if it's suppose 
> to be a
>>>  >machine readable version of it.? (As numbers __repr__ should do.)
> 
> 
>>  As I understand it, float.__repr__ does indeed give the exact value,
>>  in terms of reconstructing the float.
> 
> What I'm thinking about is...
> 
> If floats repr is changed to disregard exta digits, will this still be 
> true?? How is float to know what exta digits should be disregarded?


No one is suggesting such a change, and it would be shot down if anyone did.

The old repr and str for float used to discard (different numbers of) digits. The current version does not. Instead, if picks the shortest string that would evaluate back to the same value if passed to the float constructor (or to eval).

So, repr(0.100000000000000006) == '0.1', not because repr is discarding digits, but because?0.100000000000000006 ==?0.1 (because the closest binary IEEE double value to both is?0.1000000000000000055511151231257827021181583404541015625).

>>  There are infinitely many float literals that will result in the exact

>>  same bit pattern, so any of them is valid for repr(n) to return.
> 
> When are float literals actually converted with floats.? It seems to me, 
> that the decimal functions can be used to get the closest one.? Then they 
> will be consistent with each other. (if that isn't already being done.)


Float literals are converted to floats at compile time. When the compiler sees 0.1, or?0.100000000000000006, it works out the nearest IEEE double to that literal and stores that double.

So, by the time any Decimal function sees the float, there's no way to tell whether it was constructed from the literal 0.1, the literal?0.100000000000000006, or some long chain of transcendental functions whose result happened to have a result within 1 ulp of 0.1.

The current behavior guarantees that, for any float, float(Decimal(f)) == float(repr(Decimal(f))) == f. Guido's proposal would preserve that guarantee. If that's all you care about, nothing would change. Guido is just suggesting that, instead of using the middle Decimal from the infinite set of Decimals that would make that true, we use the shortest one.

From dickinsm at gmail.com  Sat Mar  8 19:49:02 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 8 Mar 2014 18:49:02 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
Message-ID: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>

On Sat, Mar 8, 2014 at 4:54 PM, Guido van Rossum <guido at python.org> wrote:

> When a non-expert writes Decimal(1.1), each of the three above outcomes
> surprises. We know that (1) was unpopular, that's why we changed it. We now
> know that (3) is unpopular at least in some circles (Mark Harrison can't be
> the only one who doesn't like it). Changing to (2) wouldn't do much to
> address this, because the default context has way more precision than
> float, so it still shows a lot of extraneous digits.
>

That at least could be addressed by making the default context one that
corresponds to IEEE's decimal64, which has a precision of 16 decimal
digits; there doesn't seem to be any particularly good rationale for the
current choice of default context.  And that would address your point
(which I agree with) that `Decimal(x)` currently shows way more digits than
are useful.  It wouldn't address Mark Harris's needs, though, and would in
some sense make things worse for non-expert users:  for *most* floats x,
Decimal(x) would then give what the user expects purely because 16 digits
is a good match for the binary float precision, but for a small fraction of
inputs it would give a surprising result.

>>> from decimal import Decimal, getcontext

>>> getcontext().prec = 16

>>> +Decimal(1.1)  # + to simulate rounding to 16 digits

Decimal('1.100000000000000')

>>> +Decimal(1.2)

Decimal('1.200000000000000')

>>> +Decimal(9.7)

Decimal('9.699999999999999')

This feels like a strawman, since Decimal(2**1000 + 1) and
> Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot
> of digits but is one too small), and the similar examples with a large
> exponent (10000) differ dramatically (the float version raises an
> exception).
>

> For most purposes it's a fluke that 2.0**1000 can be represented exactly
> as a float, and the argument doesn't convince me at all. There are just too
> many odd examples like that, and it always will remain a minefield.
>

Accepted, but I believe the proposal would break a number of other
expectations too with respect to ints and floats.  For one, we currently
have the unsurprising, and I believe desirable, property that conversion to
Decimal is monotonic: for any finite numbers (int, float or Decimal) x and
y,  if x <= y then Decimal(x) <= Decimal(y).  The proposal would break that
property, too: you could find examples of an integer x and float y such
that `x < y` and `Decimal(x) > Decimal(y)` would be simultaneously True.

>>> x = 49534541648432951

>>> y = x + 2.0

>>> x < y

True

>>> Decimal(x) > Decimal(repr(y))  # simulating proposed meaning of
Decimal(y)

True

I'm still struggling a bit to express exactly what it is that bothers me so
much about the proposal.  It's a combination of the above with:

- there's an obvious *correct* way to convert any value to Decimal, and
that's to round to the context precision using the context rounding mode -
that's what's done almost universally for Decimal arithmetic operations;
 it feels wrong that something as direct as `Decimal(x)` would do anything
else (the current exact conversions actually *do* rankle with me a bit, but
this is what I'd replace them with)

- this proposal would not play well with other binary number formats if
they were ever introduced:  if we introduced a float128 type (or a float32
type), it would be awkward to reconcile the proposed conversion with a
conversion from float128 to Decimal.  That's mostly because the semantics
of the proposed conversion use information about the *format* of the input
type, which is unusual for floating-point and floating-point standards:
operations tend to be based purely on the *values* of the inputs,
disregarding the formats.

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough:
Decimal(1.1 + 2.2) will still surprise, as will
Decimal(0.12345123451234512345)

- contrarily, the fix goes too far: it feels wrong to be changing the
semantics of float to Decimal conversion when the real problem is that the
user wants or expects floating-point literals to represent decimal numbers.

If the proposal goes forward, I'll live with it, and will simply avoid
using the `Decimal` type to convert from floats or ints.  But I'd really
prefer to keep the short Decimal(x) spelling as something simple and
non-magic, and find more complicated ways (quotes!) of spelling the magic
stuff.

I wonder what Cowlishaw would say about our current discussion. He is also
> the father of Rexx...
>

I wonder that, too.

I've said too much already;  beyond registering my strong -1 on this
proposal, I'm going to keep out of further discussion.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/60e1a2b2/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 19:51:59 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 10:51:59 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
Message-ID: <eaeb9a71-318e-4cd2-80bd-5e04b23e165c@googlegroups.com>



On Saturday, March 8, 2014 4:01:47 AM UTC-6, Mark Dickinson wrote

On the other hand, we probably shouldn't lend *too* much weight to IEEE 
> 754, especially when talking about choice of precision.  IEEE 754 isn't a 
> perfect fit for Decimal:  the IEEE standard is mostly concerned with fixed 
> width decimal formats, which is subtly different from Mike Cowlishaw's 
> approach of "extensible precision" where the precision is not so closely 
> tied to the format.  Python's decimal module is based on Cowlishaw's 
> standard, not on IEEE 754.
>

hi Mark,   Mike's notes include the following:

 -- the package meets the requirements of IEEE 854-1987 (with minor
     restrictions discussed below), including support for subnormal
     numbers, -0, NaNs, infinities, etc.  It also conforms to the
     floating-point arithmetic definition in ANSI X3.274-1996.

  

     here: http://grouper.ieee.org/groups/754/email/msg00429.html

The extensible precision is what makes the standard interesting.


marcus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/1b7c99ee/attachment-0001.html>

From solipsis at pitrou.net  Sat Mar  8 19:55:37 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 8 Mar 2014 19:55:37 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
Message-ID: <20140308195537.5928d3aa@fsol>

On Sat, 8 Mar 2014 18:49:02 +0000
Mark Dickinson <dickinsm at gmail.com> wrote:
> 
> If the proposal goes forward, I'll live with it, and will simply avoid
> using the `Decimal` type to convert from floats or ints.  But I'd really
> prefer to keep the short Decimal(x) spelling as something simple and
> non-magic, and find more complicated ways (quotes!) of spelling the magic
> stuff.

For the record, as a non-float expert, Mark's arguments convince me.

Just my 2 cents ;-)

Regards

Antoine.



From guido at python.org  Sat Mar  8 19:59:11 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Mar 2014 10:59:11 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
Message-ID: <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>

Thanks Oscar, that's a very well-reasoned post.

On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote:
> > I'll try to respond to Mark Dickinson's second message (and nothing else
> > that happens in the thread since last night), because (a) it concisely
> > summarizes his position and (b) brings up a new strawman.
> >
> > On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com>
> wrote:
> >> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com>
> wrote:
> >>> I see three sane options for float to Decimal conversion:
> >>>
> >>> 1. Raise an exception.
> >>> 2. Round to the nearest Decimal value using the current context for
> that
> >>> round operation.
> >>> 3. Do what we're currently doing, and do an exact conversion.
> >
> > I think you're writing this entirely from the POV of an expert in
> floating
> > point. And I'm glad we have experts like you around! I don't consider
> myself
> > an expert at all, but I do think I have something to bring to the table
> --
> > insight the experience of non-expert users.
>
> Standards compliance is important though. Mark is essentially
> restating something that is repeated hundreds of times throughout
> various floating point standards documents: a result may be exact or
> it must be correctly rounded according to the current context (with
> appropriate flags set and traps fired).
>

I have mixed feelings about such standards. I can see its importance. But
like the Unicode standard, it seems to want to grab authority over areas
that I think belong to the language design. Also at this point claiming
"compliance" with some standard is usually a morass of weasel-words rather
than clearly implementing a spec.


> It is not mandated that results requiring more precision than the
> current context be exact. It is mandated that if a result is to be
> inexact then the implementation must use a very specific type of
> inexactness. I don't believe that a standards-compliant decimal module
> has any wiggle room to invent a new kind of rounding (which I think
> would be required to achieve what you suggest).
>

That would be unfortunate.


>  > When a non-expert writes Decimal(1.1), each of the three above outcomes
> > surprises. We know that (1) was unpopular, that's why we changed it. We
> now
> > know that (3) is unpopular at least in some circles (Mark Harrison can't
> be
> > the only one who doesn't like it). Changing to (2) wouldn't do much to
> > address this, because the default context has way more precision than
> float,
> > so it still shows a lot of extraneous digits.
> >
> > Yes, it's trivial to get rid of those extra digits, just use quotes. But
> if
> > *my* proposal were adopted, it would be trivial for numerical experts to
> get
> > the extra digits -- just use from_float(). At this point, my claim is
> that
> > we're talking essentially about what is the better experience for most
> > users, and while I am not much of a user of Decimal myself, I believe
> that
> > my proposal has benefits more people and situations than it has
> downsides.
>
> If you write Decimal(1.1) and are surprised by the result then you
> have misunderstood something. It may be that you have little
> understanding of the difference between binary and decimal floating
> point (but then why are you using Decimal?). Perhaps you don't fully
> understand the literal->float->Decimal pathway that occurs when the
> expression is evaluated because you're new to Python or just haven't
> really thought about it before.
>

Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the
result, but it's still not what I want, and it's a pain to train myself to
do the extra thing that gives me what I want.


> In any case if the result of Decimal(1.1) surprises you then it's
> because you're expecting it do something that should be done in a
> different way. Hiding the extra digits does not help a user to
> understand how to use Decimal.
>

But does showing the extra digits do anything to help? It's just as likely
to teach them a trick (add quotes or a str() call) without any new
understanding.


> I actually use this in teaching to demonstrate how binary floating
> point works. I think it's important when teaching my students for them
> to understand that the following is a lie:
>
>     >>> a = 1.1
>     >>> a
>     1.1
>
> The helpful digit-hiding repr is lying to you. There is no float with
> the value 1.1. I can sort-of demonstrate this with some arithmetic:
>
>     >>> 0.1 + 0.11 - 0.11 - 0.1
>     1.3877787807814457e-17
>
> But that gives the misleading impression that inexact arithmetic is
> the source of the error. It's important to understand that the error
> occurs straight away in the literal "1.1". I demonstrate this by
> showing that
>
>     >>> from decimal import Decimal
>     >>> a = 1.1
>     >>> Decimal(a)
>     Decimal('1.100000000000000088817841970012523233890533447265625')
>
> which also shows the true value stored in the float.
>
> The fact that I use this in teaching is not supposed to serve as an
> argument for keeping it (I could just as easily use from_float). My
> point is that showing the digits helps someone to understand what is
> going on. If a user is converting float to Decimal without knowing
> what they're doing then the extra digits are a clue that they don't
> fully understand what's happening and haven't necessarily used the
> best approach.
>

I don't think every user of Decimal necessarily needs to be an expert
capable of explaining what's going on. Certainly that's not needed to be an
effective user of float -- the anomalies are explained to most people's
satisfaction by some hand-waving about imprecise results.


> There is a good solution to the problem of non-experts wanting to
> write 1.1 and get the exact value 1.1: decimal literals. With that
> they can just write 1.1d and not need to learn any more about it.
> Earlier in this thread you reject that idea saying that you can't
> teach it to "newbies":
> '''
> Maybe we can add a new literal notation meaning 'decimal'; that would
> be a relatively straightforward change to the parser (once the new
> decimal extension module is incorporated), but it would not do much to
> avoid surprising newbies (certainly you can't go teaching them to
> always write 3.14d instead of 3.14). However, it would probably help
> out frequent users of Decimal. (Then again, they might not be using
> literals that much -- I imagine the source of most such programs is
> user or file input.)
> '''
> I disagree. If you're at the point of wanting to use the Decimal
> module then you're at the point where it's reasonable to learn about
> Decimal literals.
>

You're right that I dismissed it too quickly. 3.14d is clearly even better
than Decimal(3.14) doing the right thing. It is also still a lot more work
(touches many parts of the code rather than just the Decimal class).

Also I didn't realize that the C-implemented decimal module was already
used in CPython (so I thought it would be even more work).


> Also I've written code using the decimal module for high precision
> calculation and it has lots of decimal literals. That is I do (and I
> see other's doing)
>
>     >>> from decimal import Decimal as D
>     >>> d = D('1e-20')
>
> even though this is never used in the decimal documentation. This is
> the closest you can get to decimal literals right now.


Right.

But I still have this nagging feeling that the precision Decimal(<float>)
currently gives you is, in a sense, *fake*, given that the input has much
less precision.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/e16f9768/attachment.html>

From ron3200 at gmail.com  Sat Mar  8 20:47:43 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 08 Mar 2014 13:47:43 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com>
References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org>
 <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com>
 <lffmen$to0$1@ger.gmane.org>
 <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com>
Message-ID: <lffs4h$opt$1@ger.gmane.org>



On 03/08/2014 12:43 PM, Andrew Barnert wrote:
> From: Ron Adam <ron3200 at gmail.com>
>
> Sent: Saturday, March 8, 2014 10:10 AM
>
>> On 03/08/2014 11:43 AM, Chris Angelico wrote:
>>>   On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com>  wrote:
>>
>>>>   >A repr should give the exact value its object if it's suppose
>> to be a
>>>>   >machine readable version of it.  (As numbers __repr__ should do.)

>>>   As I understand it, float.__repr__ does indeed give the exact value,
>>>   in terms of reconstructing the float.

>> What I'm thinking about is...
 >>
>> If floats repr is changed to disregard exta digits, will this still be
>> true?  How is float to know what exta digits should be disregarded?

> No one is suggesting such a change, and it would be shot down if anyone did.

Glad to hear it! :-)

> The old repr and str for float used to discard (different numbers of)
> digits. The current version does not. Instead, if picks the shortest
> string that would evaluate back to the same value if passed to the float
> constructor (or to eval).
>
> So, repr(0.100000000000000006) == '0.1', not because repr is discarding
> digits, but because 0.100000000000000006 == 0.1 (because the closest
> binary IEEE double value to both is
> 0.1000000000000000055511151231257827021181583404541015625).


>>> There are infinitely many float literals that will result in the
>>> exact
>>>
>>> same bit pattern, so any of them is valid for repr(n) to return.

>> When are float literals actually converted with floats.  It seems to
>> me, that the decimal functions can be used to get the closest one.
>> Then they will be consistent with each other. (if that isn't already
>> being done.)

> Float literals are converted to floats at compile time. When the
> compiler sees 0.1, or 0.100000000000000006, it works out the nearest
> IEEE double to that literal and stores that double.

> So, by the time any Decimal function sees the float, there's no way to
> tell whether it was constructed from the literal 0.1, the literal
> 0.100000000000000006, or some long chain of transcendental functions
> whose result happened to have a result within 1 ulp of 0.1.
>
> The current behavior guarantees that, for any float, float(Decimal(f))
> == float(repr(Decimal(f))) == f. Guido's proposal would preserve that
> guarantee. If that's all you care about, nothing would change. Guido is
> just suggesting that, instead of using the middle Decimal from the
> infinite set of Decimals that would make that true, we use the shortest
> one.

Sounds good to me..

Can that effect rounding where a value may round down instead of up? or 
vice versa.  If so, it should be the shortest string that does not cross 
the mid point of the two closest floats.  (I think I got that right.)

Not sure where I read it last night, but there was a mention that only a 
few languages do the this conversion with less than .5 Ulps error.  But it 
seems to me it might be more important to not to error in the wrong 
direction if there is a choice.


Well I'll leave it up you guys,  But it's a very interesting topic for sure.

Cheers,
    Ron











From harrismh777 at gmail.com  Sat Mar  8 20:13:24 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 11:13:24 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
Message-ID: <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>



On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: 
> Decimal(1.1 + 2.2) will still surprise, as will {snip}
>

Correct.   That is why a decimal literal notation is also needed.  I've 
emulated it here:
 
>>> from pdeclib import *
>>> sqrt(1.1+2.2)**2
Decimal('3.30000000000000026645352591003756970167159')    <==== this is a 
surprise, to a naive user
>>> sqrt(d(1.1)+d(2.2))**2
Decimal('3.29999999999999999999999999999999999999999')    <=== this if 
decimal literal,  
>>> 
        sqrt(1.1d + 2.2d)    

     But, you are correct that what is "really" wanted --someday-- is to 
have 
the literal be decimal (rather than float) to begin with.

Neither here nor there, step at a time over time is better than simple 
status quo.

Please let me be clear, I think Guido's proposal is a very good first step. 
It makes
sense for the most users (esp naive ones) and does not interfere with 
advanced users.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/b248c6e4/attachment.html>

From rosuav at gmail.com  Sat Mar  8 21:12:04 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 07:12:04 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>
Message-ID: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>

On Sun, Mar 9, 2014 at 6:13 AM, Mark H. Harris <harrismh777 at gmail.com> wrote:
>      But, you are correct that what is "really" wanted --someday-- is to
> have
> the literal be decimal (rather than float) to begin with.
>
> Neither here nor there, step at a time over time is better than simple
> status quo.
>
> Please let me be clear, I think Guido's proposal is a very good first step.
> It makes
> sense for the most users (esp naive ones) and does not interfere with
> advanced users.

It's probably time someone [1] wrote up a PEP about all this. The most
important points, as I see it, are:

1) Create a new Decimal literal notation - 1.234d seems to be the most
popular syntax. This is reasonably uncontroversial, but it has
consequences.
2) Create a new float literal notation - 1.234f or 1.234r or any of
the other proposals.
3) Possibly change repr(float) to include the tag.
4) Introduce a "from __future__ import decimal_literals" (named to
parallel unicode_literals - you can get a u"literal" without that
directive, but the default literal type becomes unicode)
5) What about int/int? Should that now be Decimal? Should it be per-module???
6) Further cans of worms like #5

Introducing 1/2/4 would let you stick the future directive into
PYTHONSTARTUP and then run Python as an interactive decimal
calculator.

ChrisA

[1] Not it!

From harrismh777 at gmail.com  Sat Mar  8 21:16:00 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 12:16:00 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
Message-ID: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>



On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:
 

> - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: 
> Decimal(1.1 + 2.2) will still surprise, as will {snip}
>

On the other hand, how is this now possible?

>>> ====== RESTART ==========================
>>> from pdeclib import *
>>> d(1.1+2.2)
Decimal('3.3')
>>> sqrt(1.1+2.2)
Decimal('1.81659021245849499925351968583091621951684')
>>> sqrt(1.1+2.2)**2
Decimal('3.29999999999999999999999999999999999999999')
>>> 

def sqrt(x):
    """ sqrt(x)     square root function

             (x may be string, int, float, or decimal)
             (returns decimal rounded to context precision)
    """
    y=x.__round__(15)
    with localcontext(ctx=None) as cmngr:
        cmngr.prec+=14
        sqr=Decimal(repr(y)).sqrt()
    return +sqr

What say you?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/221d8c53/attachment.html>

From oscar.j.benjamin at gmail.com  Sat Mar  8 21:32:58 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 8 Mar 2014 20:32:58 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
Message-ID: <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>

On 8 March 2014 18:59, Guido van Rossum <guido at python.org> wrote:
> On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>
>> On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote:
>>
>> If you write Decimal(1.1) and are surprised by the result then you
>> have misunderstood something.
>
> Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the
> result, but it's still not what I want, and it's a pain to train myself to
> do the extra thing that gives me what I want.

That "you" wasn't directed at *you* personally (and neither are the ones below).

If you're using the Decimal module then it is precisely because you do
need to care about rounding/accuracy etc. If you want to use it but
aren't prepared to take the effort to be careful about how to pass
exact input to the constructor then you're basically taking an
impossible position that no one else can help you with.

>> In any case if the result of Decimal(1.1) surprises you then it's
>> because you're expecting it do something that should be done in a
>> different way. Hiding the extra digits does not help a user to
>> understand how to use Decimal.
>
> But does showing the extra digits do anything to help? It's just as likely
> to teach them a trick (add quotes or a str() call) without any new
> understanding.

It's enough to make you think "Why did that happen?". It's clear when
you see those digits that you have not created an object with the
exact value that you wanted. The obvious next question is "How do I
make it do what I want?". The docs can lead you very quickly to the
correct way of doing it:
http://docs.python.org/3.4/library/decimal.html#quick-start-tutorial

>> There is a good solution to the problem of non-experts wanting to
>> write 1.1 and get the exact value 1.1: decimal literals. With that
>> they can just write 1.1d and not need to learn any more about it.
>
> You're right that I dismissed it too quickly. 3.14d is clearly even better
> than Decimal(3.14) doing the right thing. It is also still a lot more work
> (touches many parts of the code rather than just the Decimal class).
>
> Also I didn't realize that the C-implemented decimal module was already used
> in CPython (so I thought it would be even more work).

As Stefan mentioned earlier there are other issues to resolve around
how a decimal literal should work. It's not obvious that there should
be a straight-forward equivalence between 3.14d and D('3.14'). Perhaps
there should be new thread to consider how to potentially do that (and
whether or not it's worth it).

> But I still have this nagging feeling that the precision Decimal(<float>)
> currently gives you is, in a sense, *fake*, given that the input has much
> less precision.

That depends what you use it for. My most common reason for converting
a float to a Decimal is to test the accuracy of a float-based
calculation by comparing it against the corresponding much
higher-precision decimal calculation e.g.:

    with localcontext() as ctx:
        ctx.prec = 100
        error = f(D(x)) - D(f(x))

For this I want the constructor to give me the exact value of the float x.


Oscar

From harrismh777 at gmail.com  Sat Mar  8 21:25:58 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 12:25:58 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com>
Message-ID: <6f02d04b-5d9e-44d2-9e0d-4649f033018a@googlegroups.com>



On Saturday, March 8, 2014 2:23:23 PM UTC-6, Mark H. Harris wrote:
>
>
>  
>
>>     y=x.__round__(15)     <======  how to set the round within context so 
>> that works for float literals & decimals
>>     with localcontext(ctx=None) as cmngr:
>> {snip}
>>
>
>
> Is this possible..? 
>


   Only do this for float literals.  Yes?  Set a trap, and choose.

I gotta play with this a bit... 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/77ab3924/attachment.html>

From harrismh777 at gmail.com  Sat Mar  8 22:01:32 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 13:01:32 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
Message-ID: <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>



On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote:
>
>
>
> On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:
>  
>
>> - if we're aiming to eliminate surprises, the 'fix' doesn't go far 
>> enough: Decimal(1.1 + 2.2) will still surprise, as will {snip}
>>
>
> How about this then:   I think I've got it...

>>> ===== RESTART =========================
>>> from pdeclib import *
>>> d(1.1+2.2)
Decimal('3.3')
>>> sqrt(1.1+2.2)**2
Decimal('3.29999999999999999999999999999999999999999')
>>> x=d(1.1)
>>> y=d(2.2)
>>> sqrt(x+y)**2
Decimal('3.29999999999999999999999999999999999999999')
>>> 

Code Below--------------------------
 
#*****************************************************************/
# sqrt(x)    return decimal sqrt(x) rounded to context precision
#*****************************************************************/
def sqrt(x):
    """ sqrt(x)     square root function

             (x may be string, int, float, or decimal)
             (returns decimal rounded to context precision)
    """
    with localcontext(ctx=None) as cmngr:
        cmngr.prec+=14
        if (isinstance(x, float)==True):
            y=x.__round__(15)
            sqr=Decimal(repr(y)).sqrt()
        else:
            sqr=Decimal(x).sqrt()
    return +sqr

-------------------- what say you? ---------------------
marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/611d447d/attachment.html>

From p.f.moore at gmail.com  Sat Mar  8 22:32:34 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 8 Mar 2014 21:32:34 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140308195537.5928d3aa@fsol>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <20140308195537.5928d3aa@fsol>
Message-ID: <CACac1F8SjQJ7iRRdG6j=B5nVJXbEA320ywdJYuH9vV1rsm-TyA@mail.gmail.com>

On 8 March 2014 18:55, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 8 Mar 2014 18:49:02 +0000
> Mark Dickinson <dickinsm at gmail.com> wrote:
>>
>> If the proposal goes forward, I'll live with it, and will simply avoid
>> using the `Decimal` type to convert from floats or ints.  But I'd really
>> prefer to keep the short Decimal(x) spelling as something simple and
>> non-magic, and find more complicated ways (quotes!) of spelling the magic
>> stuff.
>
> For the record, as a non-float expert, Mark's arguments convince me.
>
> Just my 2 cents ;-)

I am in the same position. I "instinctively" wanted Decimal(1.1) to
return the same value as Decimal('1.1'), but as I read Mark's
argument, it became clearer to me that doing so involves a deliberate
loss of precision that is not present anywhere else in the chain of
conversions.

literal 1.1 -> float 1.1 is a precise conversion, in the sense that no
other float better represents the (decimal) literal 1.1.
The current float 1.1 -> decimal conversion is exact.

Converting float 1.1 to Decimal('1.1') deliberately drops precision,
even though an exact conversion is possible. I don't think that this
is something that should happen implicitly.

On the other hand, the *reason* people do Decimal(1.1) is because they
want a decimal value of 1.1. (Doh.) The fact that this isn't what they
are actually calculating is a mistake they make because Python doesn't
make it easy to *get* the decimal value 1.1 (I know the correct
approach is Decimal('1.1'), but I don't think that is obvious or
intuitive[1]). Decimal literals (1.1d) would resolve this issue
without introducing implicit rounding operations.

Paul

[1] The reason Decimal('1.1') feels non-obvious to me is that it feels
like it's getting to a decimal "via" a string. Of course, Decimal(1.1)
is also a going "via" a float. The *value* of the current behaviour is
that it leads people to an understanding that this is what it is.

From mertz at gnosis.cx  Sat Mar  8 22:33:09 2014
From: mertz at gnosis.cx (David Mertz)
Date: Sat, 8 Mar 2014 13:33:09 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
Message-ID: <CAEbHw4Yj5OA1Z68OwEvsENY_KKyW_Dd+aY0=2eV6cH6Sqq5qbQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 10:49 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

> Accepted, but I believe the proposal would break a number of other
> expectations too with respect to ints and floats.  For one, we currently
> have the unsurprising, and I believe desirable, property that conversion to
> Decimal is monotonic: for any finite numbers (int, float or Decimal) x and
> y,  if x <= y then Decimal(x) <= Decimal(y).  The proposal would break that
> property, too: you could find examples of an integer x and float y such
> that `x < y` and `Decimal(x) > Decimal(y)` would be simultaneously True.
>

I think that Mark Dickinson's point about breaking monotonicity is a STRONG
argument against the proposal to change the meaning of Decimal(float_num)
to produce Decimal(repr(float_num)).  That latter spelling is already
available if anyone wants it.

The focus on float value that have been entered as literals feels like a
distraction to me.  Floats come from other places as well, and making
Decimal(float_num) produce the *exact* value feels far more desirable than
producing "something which rounds to the same float.

As I think Guido has acknowledged in a recent post, a far better and more
intuitive approach is just to make decimal literals easier to write.
 Teaching users to write '3.14d' doesn't feel that hard to me, and it looks
notably prettier than 'decimal.Decimal("3.14")'.  A *would* also like an
optional explicit binary-float literal though, e.g. '3.14f'.  This wouldn't
actually mean anything different from '3.14', but then "PI" doesn't mean
anything different from u"PI" either.  The optional bin-float specifier, in
principle, allows for some Python 4000 in which decimal is the default
literal (although I don't think I'd ever support that, but no harm in
letting that decision be made years from now).

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough:
> Decimal(1.1 + 2.2) will still surprise, as will
> Decimal(0.12345123451234512345)
>

This is a simple illustration of why the "do what I mean" goal of the
proposed change will fall apart rather quickly for anything but the
simplest examples.  The status quo of Decimal(float_num) producing the
*exact* value continues to feel the best to me.

I've said too much already;  beyond registering my strong -1 on this
> proposal, I'm going to keep out of further discussion.
>

So yeah, me too: -1 on proposal.



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/6246cef1/attachment-0001.html>

From abarnert at yahoo.com  Sat Mar  8 22:35:14 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 13:35:14 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
Message-ID: <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>

On Mar 8, 2014, at 13:01, "Mark H. Harris" <harrismh777 at gmail.com> wrote:

> 
> 
> On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote:
>> 
>> 
>> 
>> On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:
>>  
>>> - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will {snip}
> How about this then:   I think I've got it...
> 
> >>> ===== RESTART =========================
> >>> from pdeclib import *
> >>> d(1.1+2.2)
> Decimal('3.3')
> >>> sqrt(1.1+2.2)**2
> Decimal('3.29999999999999999999999999999999999999999')
> >>> x=d(1.1)
> >>> y=d(2.2)
> >>> sqrt(x+y)**2
> Decimal('3.29999999999999999999999999999999999999999')
> >>> 
> 
> Code Below--------------------------
>  
> #*****************************************************************/
> # sqrt(x)    return decimal sqrt(x) rounded to context precision
> #*****************************************************************/
> def sqrt(x):
>     """ sqrt(x)     square root function
> 
>              (x may be string, int, float, or decimal)
>              (returns decimal rounded to context precision)
>     """
>     with localcontext(ctx=None) as cmngr:
>         cmngr.prec+=14
>         if (isinstance(x, float)==True):
>             y=x.__round__(15)
>             sqr=Decimal(repr(y)).sqrt()
>         else:
>             sqr=Decimal(x).sqrt()
>     return +sqr
> 
> -------------------- what say you? ---------------------

It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ad55f1ce/attachment.html>

From stefan at bytereef.org  Sat Mar  8 22:43:50 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 8 Mar 2014 22:43:50 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
References: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
Message-ID: <20140308214349.GA22909@sleipnir.bytereef.org>

Guido van Rossum <guido at python.org> wrote:
> I wonder what Cowlishaw would say about our current discussion. He is also the
> father of Rexx...

It's in the second paragraph here:

http://speleotrove.com/decimal/daconvs.html


My reading is that he prefers exact conversions (if possible).  This does not
surprise me:  Cowlishaw is a proponent of using all information from overlong
inputs, which all functions of the specification do (he recently mentioned in
a private mail that earlier versions of the specification did not have this
feature and it was initially added for Java's BigDecimal -- he called it
"an improvement").


Stefan Krah




From abarnert at yahoo.com  Sat Mar  8 23:15:43 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 14:15:43 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>
 <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>
Message-ID: <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com>

On Mar 8, 2014, at 12:12, Chris Angelico <rosuav at gmail.com> wrote:

> It's probably time someone [1] wrote up a PEP about all this.

> The most
> important points, as I see it, are:
> 
> 1) Create a new Decimal literal notation - 1.234d seems to be the most
> popular syntax. This is reasonably uncontroversial, but it has
> consequences.

1a) Move the Decimal type's implementation out of the module and into Objects, or make it a frozen module that's always loaded at startup, or invent some scheme to load it as needed.

1b) Possibly move the type to builtins? (Certainly not necessary--function is a built-in type, and you can create them without importing anything, but the type lives in types, not builtins.)

1c) Possibly add the implementation to the public C API? (With which methods? FromString, but what about From/AsDouble? Is there any need to From/AsTuple from C? Inspect the object in any other way? Get/set the current context?)

> 2) Create a new float literal notation - 1.234f or 1.234r or any of
> the other proposals.
> 3) Possibly change repr(float) to include the tag.

3a) Change repr(Decimal) to use the new literal.

> 4) Introduce a "from __future__ import decimal_literals" (named to
> parallel unicode_literals - you can get a u"literal" without that
> directive, but the default literal type becomes unicode)
> 5) What about int/int? Should that now be Decimal? Should it be per-module???
> 6) Further cans of worms like #5
> 
> Introducing 1/2/4 would let you stick the future directive into
> PYTHONSTARTUP and then run Python as an interactive decimal
> calculator.



From rosuav at gmail.com  Sat Mar  8 23:25:16 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 09:25:16 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>
 <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>
 <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com>
Message-ID: <CAPTjJmohRodSCbLE0evtA9ChC+OdwzYtFu+V98giN93fXXr_RQ@mail.gmail.com>

On Sun, Mar 9, 2014 at 9:15 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> 3) Possibly change repr(float) to include the tag.
>
> 3a) Change repr(Decimal) to use the new literal.

I'd put that in the consequences of part 1 - it's pretty obvious that,
with a decimal literal, the repr of a Decimal should be that. It'd
affect people's tests but shouldn't break much else. But changing
repr(float) means tagging everything, rather than just having a new
convenient notation for Decimal. It's the same physical/technical
change but it has a lot more implication :)

And yeah. You listed several significant quinseconses, and I've no
doubt there'll be others.

ChrisA

From ethan at stoneleaf.us  Sat Mar  8 22:59:32 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 08 Mar 2014 13:59:32 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
Message-ID: <531B92C4.3040502@stoneleaf.us>

On 03/08/2014 12:32 PM, Oscar Benjamin wrote:
> On 8 March 2014 18:59, Guido van Rossum wrote:
>>
>> But I still have this nagging feeling that the precision Decimal(<float>)
>> currently gives you is, in a sense, *fake*, given that the input has much
>> less precision.
>
> That depends what you use it for. My most common reason for converting
> a float to a Decimal is to test the accuracy of a float-based
> calculation by comparing it against the corresponding much
> higher-precision decimal calculation e.g.:
>
>      with localcontext() as ctx:
>          ctx.prec = 100
>          error = f(D(x)) - D(f(x))
>
> For this I want the constructor to give me the exact value of the float x.

I am not a mathematician, and it's been a long time since I took physics, but I seem to remember that a lot of 
importance was placed on significant digits.

So, how is this justified?

Python 3.4.0b3+ (default:aab7258a31d3, Feb  7 2014, 10:48:46)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
--> from decimal import Decimal as D
--> 9017.0109812864350067128347806
9017.010981286436
--> D(9017.0109812864350067128347806)
Decimal('9017.01098128643570817075669765472412109375')

In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits.  Why is 
the Decimal more "accurate" that the float it came from?

--
~Ethan~

From abarnert at yahoo.com  Sat Mar  8 23:27:25 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 8 Mar 2014 14:27:25 -0800
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
Message-ID: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>

On Mar 8, 2014, at 13:54, "Mark H. Harris" <harrismh777 at gmail.com> wrote:

> On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote:
> 
>> It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal?
> 
> That's fair... if it were me, or you, I wouldn't.   In fact, if it were just me, or just you,
> I don't think any of us would be talking about this--- I wouldn't even have brought it up.
> 
> I am advocating for people--- I am going to say, " if you want just a tad more than double
> just to check things out, please use the decimal module and import pdeclib.  These folks
> are what Guido called newbies, I'll call them naive users, and these folks are using 
> a calculator (like the TI89) and they will be prone to keying in  sqrt(2.345) /  

So your goal is that if someone is a TI89 expert but a Python novice, they can easily port a function from their TI89 to Python (or maybe even write a TI89-like desktop calculator program) with your library? That doesn't seem too unreasonable of a goal. 

The problem is that you're not quite achieving it. If you want a 16-digit-display calculator, using a mix of float64 values and decimals that expand as needed for intermediates is not equivalent to using 20-digit fixed decimals. So they will not get the same results for many functions that they port. And if they really care about having more than a few digits of precision, they either care about these differences, or really really should care. Hiding them in many but not all cases doesn't seem to be doing them a service.

A simpler solution might be to just only support a fixed precision of, say, 8 digits. Then you can handle float inputs without running into that one-too-few 3's problem you mentioned. But obviously this isn't great either--you'd just end up with users who say "your library is perfect for me, except that I need 10 digits, not 8"...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/718428b6/attachment.html>

From oscar.j.benjamin at gmail.com  Sat Mar  8 23:36:49 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 8 Mar 2014 22:36:49 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com>
 <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com>
Message-ID: <CAHVvXxTU211VAv5Nf9j2d9wWfPDnkioXs8Ftt1dZ+iDTq0zu6Q@mail.gmail.com>

On 8 March 2014 20:12, Chris Angelico <rosuav at gmail.com> wrote:
>
> It's probably time someone [1] wrote up a PEP about all this. The most
> important points, as I see it, are:
>
> 1) Create a new Decimal literal notation - 1.234d seems to be the most
> popular syntax. This is reasonably uncontroversial, but it has
> consequences.

I feel like I've boxed myself into the corner by arguing that
*someone* should do this. :)

Unless someone else especially wants the mantle I'll try to write a
PEP about decimal literals (I won't have any real time until Monday
though).


Oscar

From tjreedy at udel.edu  Sun Mar  9 00:16:52 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Mar 2014 18:16:52 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140307101235.GR28804@ando>
References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
Message-ID: <lfg8d6$jal$1@ger.gmane.org>

On 3/7/2014 5:12 AM, Steven D'Aprano wrote:

> For example, between 1 and 100000, about 12% of integer-valued floats
> fail the "1/(1/n) == n" test, but over 51% of the integer-valued
> Decimals:
>
> py> from decimal import Decimal as D
> py> sum(1/(1/n) != n for n in range(1, 100001))
> 11846
> py> sum(1/(1/D(n)) != n for n in range(1, 100001))
> 51665
>
>
> Likewise we can test how many fail the "(1/n)*n == 1" test:
>
> py> sum((1/n)*n != 1 for n in range(1, 100001))
> 13117
> py> sum((1/D(n))*n != 1 for n in range(1, 100001))
> 36806
>
> 13% for floats versus 36% for Decimals.
>
>
> One more to prove it isn't a fluke: the "sqrt(n)**2 == n" test:
>
> py> sum((n**0.5)**2 != n for n in range(1, 100001))
> 49544
> py> sum((n**D("0.5"))**2 != n for n in range(1, 100001))
> 71303
>
>
> That's three for three in favour of binary floats.

Thank you for these concrete examples. Yesterday I posted that binary 
floats are better because they are smoother, so that the relative 
approximation is more constant. I would not be surprised if this is 
behind the numbers above. In any case, the above convince me more 
strongly that we should stay with binary floats as default.

-- 
Terry Jan Reedy


From tim.peters at gmail.com  Sun Mar  9 00:20:39 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 8 Mar 2014 17:20:39 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
Message-ID: <CAExdVNmLTi9SurzzHAJZ50x7FTw0guRcskCb3VFpb+uv5Y2XTw@mail.gmail.com>

[Mark Dickinson]
>> Python's decimal module is based on Cowlishaw's
>> standard, not on IEEE 754.

[Guido]
> I wonder what Cowlishaw would say about our current discussion. He is also
> the father of Rexx...

I _expect_ Mike likes the status quo.  Explaining why by analogy:

I'm certain the 754 designers would not like the status quo.  To them
string<->number conversions were "operations", no different in
principle than the operations of, say, addition or root extraction.
And _all_ operations in 754 treat the input(s) as infinitely precise,
and produce an output correctly rounded according to the current
context.  Now that's technically not so for float->string operations
in 754, but that's an irrelevant distraction (754 allowed for weaker
rounding guarantees in that specific context because nobody knows how
to achieve correct rounding efficiently in all cases in that context -
and the members of the 754 committee later said they regretted
allowing this exception).

Bottom line here:  for all the 754 designers, and regardless of the
type of x, Decimal(x) should accept x as exactly correct and produce a
decimal object correctly rounded according to the current context
(including setting context flags appropriately - e.g., signaling the
inexact exception if any information in the input was lost due to
rounding).

Now the specific instance of this Mike did pronounce on is
Decimal(string).  It's obvious that the 754 view is that the
assumed-to-be infinitely precise string literal be rounded to the
current context's precision (etc).  But Mike (in email at the time)
explicitly wanted to retain all the string digits in the returned
decimal object, wholly ignoring the current context.

So that's what Python does. Is Decimal(0.1) really different?  The 754
view, once you're used to it, is utterly predictable:  Whatever the
internal representation of 0.1, it's assumed to be infinitely precise,
and you round its value to the decimal context's current precision.
Mike's view is usually the same, but in one specific case of
construction he thought differently.  My guess is that he'd choose to
be consistent with "the rule" for construction, having made an
exception for it once already, than choose to be consistent with the
other decimal operations.

Or we could hark back to REXX's desire to present an arithmetic that
works the way people learned in school.  I'd try that, except I'm not
sure kids today learn arithmetic in school any more beyond learning
how to push buttons ;-)

From cfkaran2 at gmail.com  Sun Mar  9 00:24:06 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sat, 8 Mar 2014 18:24:06 -0500
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <5317FDB6.5000806@canterbury.ac.nz>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
Message-ID: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>

I've written some sample code that can do the type of annotations that I'm suggesting; is there a particular way that code is shared on the list, or should I just put it on the web somewhere?

On Mar 5, 2014, at 11:46 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> CFK wrote:
>> then it shouldn't be too hard for a project to add its own UUID to the annotation dictionary.
> 
> I can't see what benefit there is in bringing UIIDs into
> the picture.
> 
> If you're suggesting that people write the UUIDs directly
> into their code as literals, that's *not* going to fly.
> It would be appallingly ugly and error-prone.
> 
> The only way to make it usable would be to import the
> UUIDs from somewhere under human-readable names. But then
> there's no need to use UUIDs, any set of unique sentinel
> objects will do.
> 
> -- 
> Greg

There are a number of reasons I'm suggesting UUIDs instead of simple strings:

- Since they are truly unique, and since the hex string doesn't form words in any human language, you can enter the string in your favorite search engine to get more information about the string.  The probability of false positives is almost 0, which is something that a string like 'doc' simply won't give you.

- They don't require any coordination between projects.  That means you don't have to worry that the name you choose for your project conflicts with a name that someone else chooses.  You also don't have to worry about changing the name of your project; UUIDs don't carry any inherent meaning to people so you can keep them even when your project changes.

- They have fixed lengths and formats.  Annotations have been around for a while, which means that any automated tooling is going to have to find a way to deal with a new convention.  If a parser can convert all the keys into UUID instances, then that gives it a high confidence that the code is using this convention.  This is much more difficult to do when the universe of keys is more or less arbitrary.

As for being ugly and error prone, I agree that putting it all in by hand would be horrible.  The code that I wrote demonstrates a different way, which only requires decorators.  It is basically a library that does the following:

    import annotizer
    import uuid

    # Load your uuid in some manner into project_ID.
    project_annotizer = annotizer.annotizer(ID)

    # From now on, you can use project_annotizer's decorators everywhere.

    @project_annotizer.parameter_decorator(a="blah")
    def func(a):
        pass

The example code I've written doesn't use strings, it uses uuid.UUID instances directly, but that is easily changed.  With some work, it can be extended for a large variety of useful annotations, all while hiding the complexities of dealing with UUIDs completely.  

Thanks,
Cem Karan

From harrismh777 at gmail.com  Sat Mar  8 21:23:23 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 12:23:23 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
Message-ID: <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com>



On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote:
 

>     y=x.__round__(15)     <======  how to set the round within context so 
> that works for float literals & decimals
>     with localcontext(ctx=None) as cmngr:
> {snip}
>


Is this possible..? 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/84f6fa98/attachment.html>

From alexander.belopolsky at gmail.com  Sun Mar  9 00:36:18 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Mar 2014 18:36:18 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
Message-ID: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 11:54 AM, Guido van Rossum <guido at python.org> wrote:
>
> I'll try to respond to Mark Dickinson's second message (and nothing else
that happens in the thread since last night), because (a) it concisely
summarizes his position and (b) brings up a new strawman.
>
> On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>
>> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com>
wrote:
>>>
>>>
>>> I see three sane options for float to Decimal conversion:
>>>
>>> 1. Raise an exception.
>>> 2. Round to the nearest Decimal value using the current context for
that round operation.
>>> 3. Do what we're currently doing, and do an exact conversion.
>
>
> I think you're writing this entirely from the POV of an expert in
floating point. And I'm glad we have experts like you around! I don't
consider myself an expert at all, but I do think I have something to bring
to the table -- insight the experience of non-expert users.
>

This reminded me a discussion we had with Mark Dickinson on the bug
tracker: "When a user is entering 0.6112295, she means 0.6112295, not
0x1.38f312b1b36bdp-1 or
0.61122949999999998116351207499974407255649566650390625 which are exact
values of the underlying binary representation of 0.6112295."

http://bugs.python.org/issue8860#msg108601

The topic was the conversion from binary floats to timedelta which is
effectively a fixed point decimal with 6 decimal places after the point.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/535d4dd1/attachment.html>

From rosuav at gmail.com  Sun Mar  9 00:38:37 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 10:38:37 +1100
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
Message-ID: <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com>

On Sun, Mar 9, 2014 at 10:24 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
> There are a number of reasons I'm suggesting UUIDs instead of simple strings:

But against that is that they're extremely long. Most other places
where long hex strings are used, humans won't use the full thing. With
git and hg, revisions are identified by hash - but you can use a
prefix, and in most repos, 6-8 hex digits will uniquely identify
something. When humans look at SSH key fingerprints, how many actually
read the whole thing? That's why randart was produced. Tell me, which
of these is different?

def f(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
    pass

def asdf(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
    pass

def hello_world(x:"f0e4c2f76c58916ec258f246851bea891d14d4247a2fc3e18694461b1816e13b"):
    pass

def testing(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
    pass

def longer_function_name(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
    pass

As uniqueness guarantors, they're a bit unwieldy, and that makes them
pretty unhelpful.

ChrisA

From harrismh777 at gmail.com  Sun Mar  9 00:33:15 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 15:33:15 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>
Message-ID: <a86e5574-21f4-48f3-a973-8f39c2371681@googlegroups.com>



On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote:
 

> So your goal is that if someone is a TI89 expert but a Python novice, they 
> can easily port a function from their TI89 to Python (or maybe even write a 
> TI89-like desktop calculator program) with your library? That doesn't seem 
> too unreasonable of a goal. 
>

Your points are all mostly valid, taking all assumptions into account. Some 
of these
folks actually have TI89's but don't really know how to use them yet.  Some 
of them
have TI84+, or TI83,  some TI Nspire... some others.  Mostly the comparison 
between
the calc and python isn't the point. The point is using python to do the 
trig (or whatever)
and along the way introducing programming.  Its just one use case that is 
not too
different than a thousand other "average" or naive user cases, where the 
folks are 
still on the learning curve but need to be introduced to computers and 
computer 
science as well as the maths.  This will also be good for folks who know 
what they
are doing and will now be able to do it just a little more efficiently. 
 For folks who 
really know what they are doing spelling issues are not even a problem. 
 So, we can
help newbie/naive users without impacting advanced users like yourself. 

The bottom line is trying to eliminate as few surprises as possible, making 
the learning
curve as easy and fun as possible. The point is not to make a calculator; 
 rather to 
make a decimal floating point package for python that is flexible. 
Experienced users 
can use it for all kinds of purposes, and newbies can use it too. The 
experts on this
list, like you, will have no problem using the package (if you want) and 
folks that are
brand new to python will be able to use it too. 

At some point the decimal concepts will come up, and the decimal literal 
will come in
very handy then. For folks using 3.3.4 or below obviously the caveats are 
going to 
need to be explained in the documentation... and my functions are going to 
need to
take in floats and do something meaningful with them, as you saw in the 
square root 
routine. Its a little more hassle for me to code and maintain, but its less 
"surprise" 
impact on users will be worth it (esp for young wanna bee naivet? types).

marcus 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/bba8d014/attachment-0001.html>

From tjreedy at udel.edu  Sun Mar  9 00:48:50 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Mar 2014 18:48:50 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
Message-ID: <lfga94$4tq$1@ger.gmane.org>

On 3/7/2014 9:02 PM, Guido van Rossum wrote:
> On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano

> What is the "suggested change" here? If it's "default float literals to
> Decimal" I agree. But if the suggestion is my "Decimal(<float>) should
> be implemented as Decimal(repr(<float>))" I don't think most scientists
> care (few of them use Decimal, they stay entirely in the realm of binary
> floating point).

As a scientist, I either would not care, or might like it if it reduced 
the pressure to switch to decimal floats. I would however, consider 
changing the proposal to producing the best n-digit decimal, where n is 
the current context precision, or at least the default number (greater 
than 17?). This might be easily done by making n a parameter instead of 
a constant in the current internal function used to produce repr(float).

>     Python's done 97% of the work for you: the Decimal module, and the cmd
>     module which makes building interactive command-line oriented
>     applications easy. You can even simulate the Python interactive
>     interpreter.

Idle already does that.

>     In other words, if you don't want to convert from floats to Decimal,
>     there is *absolutely no reason* why you should.

It might be possible to experiment with this in Idle. It already does 
some parsing of user input as typed in order to color code it, though 
that may not recognize float literals. If not, let float_lit = <re that 
recognizes float literals>. Then use re.sub to replace 'xx.yy', etc, 
with 'D("xx.yy")' before sending user input to the subprocess. When 
auto-decimal mode is set, display and send 'from decimal import Decimal 
as D' to the user-process and change the prompt to, for instance

dec> ...


-- 
Terry Jan Reedy


From harrismh777 at gmail.com  Sun Mar  9 00:44:25 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 15:44:25 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
Message-ID: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>



On Saturday, March 8, 2014 5:36:18 PM UTC-6, Alexander Belopolsky wrote:
 

> This reminded me a discussion we had with Mark Dickinson on the bug 
> tracker: "When a user is entering 0.6112295, she means 0.6112295, not 
> 0x1.38f312b1b36bdp-1 or 
> 0.61122949999999998116351207499974407255649566650390625 which are exact 
> values of the underlying binary representation of 0.6112295."
>
> http://bugs.python.org/issue8860#msg108601
>

Yes, and I certainly don't want to start up the whole underlying discussion 
again, but very simply put,  that's it !

There are two user issues:

   1) maths should not give surprises

   2) inputs should have a reasonable "expected" interpretation


marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/d18d28dc/attachment.html>

From greg.ewing at canterbury.ac.nz  Sun Mar  9 01:05:53 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Mar 2014 13:05:53 +1300
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
Message-ID: <531BB061.2060009@canterbury.ac.nz>

Cem Karan wrote:
> There are a number of reasons I'm suggesting UUIDs instead of simple strings:

I'm not talking about strings, I'm talking about
objects created and exported by the module defining
the annotations, and compared by identity.

The Python module namespace then ensures they have
unique names within any given program. That's all
you need, because there's no requirement to persist
them from one program execution to another.

-- 
Greg

From alexander.belopolsky at gmail.com  Sun Mar  9 01:06:39 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Mar 2014 19:06:39 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
Message-ID: <CAP7h-xbh3FukXCBTPBX_-9AM_-1UUum0z4KXp8i7fwP2i2cEnw@mail.gmail.com>

On Sat, Mar 8, 2014 at 6:44 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:

> There are two user issues:
>
>    1) maths should not give surprises
>
>    2) inputs should have a reasonable "expected" interpretation
>

I am with Mark H. on this.  While I appreciate the theoretical
underpinnings of the status quo, I am yet to see data stored in binary that
did not originate from decimal at some point.  I think 99.999% of all
instances of 0x1.199999999999ap+0 in numerical data come from a conversion
of 1.1 from decimal to float rather than an a result of a computation that
is correct to 16+ decimal places.

In a well-designed system, simple things should be simple and difficult
things should be possible.  Decimal(x) = Decimal(str(x)) is simple and what
most users expect and ultimately need.  Expert users will always
have Decimal.from_float, contexts, round and other power tools at their
disposal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/3daf6b7b/attachment.html>

From rosuav at gmail.com  Sun Mar  9 01:07:21 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 11:07:21 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfga94$4tq$1@ger.gmane.org>
References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <lfga94$4tq$1@ger.gmane.org>
Message-ID: <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com>

On Sun, Mar 9, 2014 at 10:48 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> It might be possible to experiment with this in Idle. It already does some
> parsing of user input as typed in order to color code it, though that may
> not recognize float literals. If not, let float_lit = <re that recognizes
> float literals>. Then use re.sub to replace 'xx.yy', etc, with 'D("xx.yy")'
> before sending user input to the subprocess. When auto-decimal mode is set,
> display and send 'from decimal import Decimal as D' to the user-process and
> change the prompt to, for instance

I played with the possibility of an AST transform, but by the time
it's compiled into an AST, it's too late - the string has already been
floated. A regex would be a fairly hefty hammer, I think.

ChrisA

From harrismh777 at gmail.com  Sat Mar  8 22:54:11 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 13:54:11 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
Message-ID: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>



On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote:

It looks like you're trying to emulate a pocket calculator here. The 
> question is, why are you accepting floats in the first place if that's your 
> goal?
>

That's fair... if it were me, or you, I wouldn't.   In fact, if it were 
just me, or just you,
I don't think any of us would be talking about this--- I wouldn't even have 
brought it up.

I am advocating for people--- I am going to say, " if you want just a tad 
more than double
just to check things out, please use the decimal module and import pdeclib. 
 These folks
are what Guido called newbies, I'll call them naive users, and these folks 
are using 
a calculator (like the TI89) and they will be prone to keying in 
 sqrt(2.345) /  

Its just easier than explaining to them (at this point) all the stuff they 
need to understand to use
the functions by making sure they feed decimals to the function; either by 
quoting their
numeric inputs, or even using a decimal literal, &c. They know what the 
functions do,
and will want to use them to get more digits, but may not remember the 
rules to get the
"correct" digits.  So I'm advocating for them, to make it easier.  

If it were just me,  I'd just make sure the functions got decimals and be 
done with it. 

If you look at my square root function, its really not right; but its 
closer.  For instance if you give it 
1/3 it still is only slightly better than before. I don't have all the junk 
after the 16 th 
digit, but, I only have .33333333 to 15 places.  So, believe me, I know 
there is no
one truly good answer here. Just trying to make it easier for the most 
people.

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/8b8a914b/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 01:21:59 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 16:21:59 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>
Message-ID: <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com>



On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote:
>
> {snip}


Many folks "play" on the interactive python terminal;  and some python 
experienced users
and developers forget this. For instance the QPython people did not provide 
a terminal 
or interactive python with their first release because they were focused on 
their product 
being a script reader and forgot that normal people experiment on the 
interactive console, for 
calculation, and other things.   We had a discussion the other night about 
the decimal distribution
(numerical digit distribution) of the digits 0-9 in the number PI. The 
conjecture is that as the 
number of digits increased the distribution evens out, and PI is for all 
intents and purposes 
a random number generator (for large numbers of digits).  But what does it 
look like to users
who have never seen it, and how hard is it to code up?   Well, its two 
lines of python code, 
and inexperienced users can not believe that:  below:

>>> ================================ RESTART 
================================
>>> from pdeclib import *
>>> dscale(1010)
42
>>> pi=get_PI()
>>> sPI=repr(pi)[:1002]
>>> 
>>> for n in range(10):
print(n, sPI.count(str(n)))

0 92
1 114
2 102
3 103
4 92
5 97
6 93
7 95
8 100
9 104
 
Well, there it is; the distribution of the digits of PI in the first 1002, 
precisely;  all from the console.

marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/92d2c7ee/attachment.html>

From cfkaran2 at gmail.com  Sun Mar  9 01:34:32 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sat, 8 Mar 2014 19:34:32 -0500
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
 <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com>
Message-ID: <970F7B20-6FA8-4B43-91F1-292440CF8FCB@gmail.com>


On Mar 8, 2014, at 6:38 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Mar 9, 2014 at 10:24 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
>> There are a number of reasons I'm suggesting UUIDs instead of simple strings:
> 
> But against that is that they're extremely long. Most other places
> where long hex strings are used, humans won't use the full thing. With
> git and hg, revisions are identified by hash - but you can use a
> prefix, and in most repos, 6-8 hex digits will uniquely identify
> something. When humans look at SSH key fingerprints, how many actually
> read the whole thing? That's why randart was produced. Tell me, which
> of these is different?
> 
> def f(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def asdf(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def hello_world(x:"f0e4c2f76c58916ec258f246851bea891d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def testing(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def longer_function_name(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> As uniqueness guarantors, they're a bit unwieldy, and that makes them
> pretty unhelpful.

Agreed... for human beings.  However, the purpose isn't just for people, its for automated systems that use annotations in possibly incompatible ways.  My original example was the problem of docstrings between projects like PLY (http://www.dabeaz.com/ply/) and Sphinx (http://sphinx-doc.org/).  For this pair of projects, you must choose to either document your code, or use the parser; not both.  My proposal circumvents the problem.  Nick Coghlan suggested using decorators instead, which I implemented.  In my test code (posted below), you can stack decorators, which completely hide the UUID from human eyes.  If an automated tool requires the annotations, it can look for its own UUID.

BTW, my apologies if the test code isn't 100% perfect; I threw it together to make sure that the decorator idea would work well.  Also, I'm using actual UUID instances instead of strings; it was easier to see what I was doing that way, but would need to be changed in real code.

Thanks,
Cem Karan

#!/usr/bin/env python
# -*- coding: utf-8 -*-

__docformat__ = "restructuredtext en"

import inspect
import uuid
import pprint

##############################################################################
##############################################################################
### Helper classes
##############################################################################
##############################################################################


class _parameter_decorator(object):
    """
    This class adds a documentation string to the annotations of a function or
    class instance's parameter.  You can add multiple documentation strings,
    or you can add one at a time.  Note that it is intended to only be used by
    the annotizer class.
    """
    def __init__(self, ID, *args, **kwargs):
        self._ID = ID
        self._args = args
        self._kwargs = kwargs
    # End of __init__()

    def __call__(self, f):
        sig = inspect.signature(f)

        func_args = {x for x in sig.parameters}
        decorator_args = {x for x in self._kwargs}
        keys = func_args.intersection(decorator_args)

        for key in keys:
            if key not in f.__annotations__:
                f.__annotations__[key] = dict()
            if self._ID not in f.__annotations__[key]:
                f.__annotations__[key][self._ID] = dict()
            f.__annotations__[key][self._ID]["doc"] = self._kwargs[key]

        return f
    # End of __call__()
# End of class _parameter_decorator

class _return_decorator(object):
    """
    This class adds a documentation string to the annotations of a function or
    class instance's return value.
    """
    def __init__(self, ID, *args, **kwargs):
        self._ID = ID
        self._args = args
        self._kwargs = kwargs
    # End of __init__()

    def __call__(self, f):
        sig = inspect.signature(f)

        key = 'return'
        if sig.return_annotation == inspect.Signature.empty:
            f.__annotations__[key] = dict()
        if self._ID not in f.__annotations__[key]:
            f.__annotations__[key][self._ID] = dict()
        f.__annotations__[key][self._ID]["doc"] = self._args[0]

        return f
    # End of __call__()
# End of class _return_decorator

##############################################################################
##############################################################################
### annotizer
##############################################################################
##############################################################################


class annotizer(object):
    """
    This assists in making annotations by providing decorators that are aware
    of your project's internal `UUID <http://en.wikipedia.org/wiki/UUID>`_.
    Using this class will ensure that your annotations are separate from the
    annotations that others use, even if the keys they use are the same as
    your keys.  Thus, projects **A** and **B** can both use the key ''doc''
    while annotating the same function, without accidentally overwriting the
    other project's use.
    """
    def __init__(self, ID,
                 parameter_decorator_class=_parameter_decorator,
                 return_decorator_class=_return_decorator):
        """
        :param ID: This is your project's UUID.  The easiest way to generate
            this is to use the
            `uuid <http://docs.python.org/3/library/uuid.html`_ module, and
            then store the ID somewhere convenient.
        :type ID: ``str`` that can be used to initialize a
            `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_
            instance, or a
            `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_
            instance
        """
        if isinstance(ID, uuid.UUID):
            self._ID = ID
        else:
            self._ID = uuid.UUID(ID)

        self.parameter_decorator_class = parameter_decorator_class
        self.return_decorator_class = return_decorator_class
    # End of __init__()

    def ID():
        doc = ("This is the ID of your project.  It is a " +
               "`UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_" +
               "instance.")
        def fget(self):
            return self._ID
        return locals()
    ID = property(**ID())

    def parameter_decorator_class():
        doc = ("Instances of this class may be used to decorate")
        def fget(self):
            return self._parameter_decorator_class
        def fset(self, value):
            self._parameter_decorator_class = value
        def fdel(self):
            self._parameter_decorator_class = _parameter_decorator
        return locals()
    parameter_decorator_class = property(**parameter_decorator_class())

    def return_decorator_class():
        doc = "The return_decorator_class property."
        def fget(self):
            return self._return_decorator_class
        def fset(self, value):
            self._return_decorator_class = value
        def fdel(self):
            self._return_decorator_class = _return_decorator
        return locals()
    return_decorator_class = property(**return_decorator_class())

    def parameter_decorator(self, *args, **kwargs):
        decorator = self.parameter_decorator_class(self.ID, *args, **kwargs)
        return decorator
    # End of parameter_decorator()

    def return_decorator(self, *args, **kwargs):
        decorator = self.return_decorator_class(self.ID, *args, **kwargs)
        return decorator
    # End of return_decorator()
# End of class annotizer

##############################################################################
##############################################################################
### Main
##############################################################################
##############################################################################

if __name__ == "__main__":
    ID1 = uuid.uuid1()
    an1 = annotizer(ID1)
    ID2 = uuid.uuid1()
    an2 = annotizer(ID2)

    @an1.parameter_decorator(a="a", b="b", c="c")
    @an2.parameter_decorator(a="A", b="B", c="C")
    @an1.return_decorator("Doesn't return anything of value")
    @an2.return_decorator("Does not return a value")
    def func(a,b,c):
        print("a = {0!s}, b = {1!s}, c = {2!s}".format(a,b,c))

    pprint.pprint(func.__annotations__)

The output is:

$ ./annotizer.py 
{'a': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'a'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'A'}},
 'b': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'b'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'B'}},
 'c': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'c'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'C'}},
 'return': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': "Doesn't return anything of value"},
            UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'Does not return a value'}}}

From harrismh777 at gmail.com  Sun Mar  9 01:35:57 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 16:35:57 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com>
 <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com>
Message-ID: <392731c6-4dfa-4cce-8289-bd9a61701ddd@googlegroups.com>

On Saturday, March 8, 2014 6:21:59 PM UTC-6, Mark H. Harris wrote:

Actually this is it, because I keep forgetting that repr(D) includes 
"Decimal " in the string, must 
use str()
>>> from pdeclib import *
>>> dscale(1010)
42
>>> pi=get_PI()
>>> sPI=str(pi)[:1002]
>>> for n in range(10):
      print(n, sPI.count(str(n)))
      
0 93
1 116
2 103
3 103
4 93
5 97
6 94
7 95
8 101
9 106
>>>  
      Anyway,  its just fun....
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/0da5ab2a/attachment.html>

From steve at pearwood.info  Sun Mar  9 01:39:25 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 9 Mar 2014 11:39:25 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
Message-ID: <20140309003925.GC28804@ando>

On Sat, Mar 08, 2014 at 03:44:25PM -0800, Mark H. Harris wrote:

> There are two user issues:
> 
>    1) maths should not give surprises
>    2) inputs should have a reasonable "expected" interpretation

Computer maths cannot fail to give surprises.

py> x = 1/Decimal(3)
py> sum([x, x, x]) == 1
False

"No surprises" is simply an impossible request.



-- 
Steven

From steve at pearwood.info  Sun Mar  9 01:44:36 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 9 Mar 2014 11:44:36 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com>
References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <lfga94$4tq$1@ger.gmane.org>
 <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com>
Message-ID: <20140309004436.GD28804@ando>

On Sun, Mar 09, 2014 at 11:07:21AM +1100, Chris Angelico wrote:
> On Sun, Mar 9, 2014 at 10:48 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> > It might be possible to experiment with this in Idle. It already does some
> > parsing of user input as typed in order to color code it, though that may
> > not recognize float literals. If not, let float_lit = <re that recognizes
> > float literals>. Then use re.sub to replace 'xx.yy', etc, with 'D("xx.yy")'
> > before sending user input to the subprocess. When auto-decimal mode is set,
> > display and send 'from decimal import Decimal as D' to the user-process and
> > change the prompt to, for instance
> 
> I played with the possibility of an AST transform, but by the time
> it's compiled into an AST, it's too late - the string has already been
> floated. A regex would be a fairly hefty hammer, I think.

Great minds think alike :-)

An AST transformation is possible:


import ast

class MyTransformer(ast.NodeTransformer):
    def visit_Num(self, node):
        # Transform float literals to Decimal("literal").
        if isinstance(node.n, float):
            quoted_literal = ast.Str(repr(node.n))
            new_node =ast.Call(
                    func=ast.Name(id='Decimal', ctx=ast.Load()),
                    args=[quoted_literal],
                    )
            return ast.copy_location(new_node, node)
        return node

node = ast.parse('2.5 + 5**0.5', mode='eval')
print("Before:")
ast.dump(node)
print("After:")
ast.dump(MyTransformer().generic_visit(node))



I tried to find some way to hook that transformation into the code 
module, so as to give an intereactive interpreter with decimal floats, 
but couldn't work out how to do so.


-- 
Steven

From rosuav at gmail.com  Sun Mar  9 02:46:53 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 12:46:53 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309004436.GD28804@ando>
References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <20140307101235.GR28804@ando>
 <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com>
 <20140308010526.GV28804@ando>
 <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com>
 <lfga94$4tq$1@ger.gmane.org>
 <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com>
 <20140309004436.GD28804@ando>
Message-ID: <CAPTjJmqHHfuvf0zExHOKQ3ODs9zMhKGMwJ9H8-tJQi=SkOq=rg@mail.gmail.com>

On Sun, Mar 9, 2014 at 11:44 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Great minds think alike :-)
>
> An AST transformation is possible:
>
>
> import ast
>
> class MyTransformer(ast.NodeTransformer):
>     def visit_Num(self, node):
>         # Transform float literals to Decimal("literal").
>         if isinstance(node.n, float):
>             quoted_literal = ast.Str(repr(node.n))

Somewhat possible, but it's kinda too late at that point; you've
already gone via float. It's of some use, perhaps, but it doesn't
actually let you "do more". I was hoping to get the source string
before it got floated, and to make a decimal out of it instead. But by
that time, it's already actually become a float, and the AST node.n
is, as you demonstrate here, an actual float.

I tried substituting in a Decimal instead of building up a call to
Decimal("..."), but compile() won't touch it:

>>> ast.dump(node)
"Expression(body=BinOp(left=Num(n=Decimal('2.5')), op=Add(),
right=BinOp(left=Num(n=5), op=Pow(), right=Num(n=Decimal('0.5')))))"
>>> compile(node,"<...>","eval")
Traceback (most recent call last):
  File "<pyshell#15>", line 1, in <module>
    compile(node,"<...>","eval")
TypeError: non-numeric type in Num

Awwwww :(

ChrisA

From cfkaran2 at gmail.com  Sun Mar  9 02:54:07 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sat, 8 Mar 2014 20:54:07 -0500
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <531BB061.2060009@canterbury.ac.nz>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
 <531BB061.2060009@canterbury.ac.nz>
Message-ID: <33D3ACF7-35BB-4AC7-9843-F680A089A547@gmail.com>


On Mar 8, 2014, at 7:05 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Cem Karan wrote:
>> There are a number of reasons I'm suggesting UUIDs instead of simple strings:
> 
> I'm not talking about strings, I'm talking about
> objects created and exported by the module defining
> the annotations, and compared by identity.
> 
> The Python module namespace then ensures they have
> unique names within any given program. That's all
> you need, because there's no requirement to persist
> them from one program execution to another.
> 
> -- 
> Greg

I see your point, and I think you're right.  I can see a lot of really good additional tricks we can play (adding in metadata to the sentinel, etc.).  I've spent the time since I saw this message trying to come up with good counterarguments and I honestly can't come up with any good ones.  I can only think of two counterarguments, both of which are really weak.  First, if you want to mock the module, you don't need the module's sentinel; you just need the UUID, and then you never need to load the module at all.  Second, there are cases where you want to warn people that although the signature of a function has remained the same, the semantics have shifted.  In this case, a module user might want to key off of the UUID.  If the ID changes, then the meaning has shifted. Both of those counterarguments are really weak though, so unless anyone can think of a reason NOT to use sentinels as Greg suggests, I'd like to shift my proposal to using sentinels instead of UUIDs.

Thanks,
Cem Karan

From harrismh777 at gmail.com  Sun Mar  9 02:58:18 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 17:58:18 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309003925.GC28804@ando>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
Message-ID: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>



On Saturday, March 8, 2014 6:39:25 PM UTC-6, Steven D'Aprano wrote:
 

> py> x = 1/Decimal(3) 
> py> sum([x, x, x]) == 1 
> False 
>
> "No surprises" is simply an impossible request. 
>

hi Steven,  that's a straw man.   Your argument presumes that False is a 
surprise.  I am
not in the least surprised.  As, I am also not surprised by this:

>>> dscale(32)
10100
>>> d(1)/3
Decimal('0.33333333333333333333333333333333')   <=== not a surprise, 
expected
>>> d(1)/3 + d(1)/3 +d(1)/3
Decimal('0.99999999999999999999999999999999')   <=== not a surprise either, 
quite expected
>>> 

And following:

>>> dscale(42)
32
>>> dSum= d(1)/3 + d(1)/3 +d(1)/3
>>> dscale(38)
42
>>> +dSum     <=====   at this point I expect to see the following line, 
just as I see it./
Decimal('1.0000000000000000000000000000000000000')
>>> 

 Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/81b05f1b/attachment-0001.html>

From alexander.belopolsky at gmail.com  Sun Mar  9 03:12:53 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Mar 2014 21:12:53 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309003925.GC28804@ando>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
Message-ID: <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>

On Sat, Mar 8, 2014 at 7:39 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> Computer maths cannot fail to give surprises.
>
> py> x = 1/Decimal(3)
> py> sum([x, x, x]) == 1
> False
>
> "No surprises" is simply an impossible request.
>

Maybe, but there is nothing impossible in eliminating this particular
surprise:

q)x:1 % 3f
q)x
0.3333333
q)1 = x + x + x
1b

http://en.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems)

I am not suggesting that Python should behave like Q in this respect, just
showing that this particular wart is not present in all languages.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ce74a2d7/attachment.html>

From tim.peters at gmail.com  Sun Mar  9 03:19:54 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 8 Mar 2014 20:19:54 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
Message-ID: <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com>

[Oscar Benjamin <oscar.j.benjamin at gmail.com>]
> ...
> I don't believe that a standards-compliant decimal module
> has any wiggle room to invent a new kind of rounding (which I think
> would be required to achieve what you suggest).

That would be true if the standard explicitly forbade extensions (any
optional operation or behavior not specified by the standard).  But
there are no standards like that ;-)  To comply with a standard, there
just needs to be _some_ way to "spell" all the behaviors the standard
mandates.  For example, in a programming language no relevant 754-ish
standard says that the infix binary operator "*" has to implement
754-ish multiplication.  So far as standard compliance goes, "*" could
mean "ignore the second argument and add 1.7 to the first argument,
rounding toward 7" - just so long as there's *some* way to spell
"754-ish multiplication".

Python (or any other implementation) is quite free to invent all the
rounding modes (or any other kinds of operations, or modifications to
standard behaviors) it likes.  That kind of experimentation is where
ideas for the _next_ iteration of the standard comes from.

From harrismh777 at gmail.com  Sun Mar  9 03:26:51 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 18:26:51 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com>
Message-ID: <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com>



On Saturday, March 8, 2014 8:19:54 PM UTC-6, Tim Peters wrote

That would be true if the standard explicitly forbade extensions (any 
> optional operation or behavior not specified by the standard).  But 
> there are no standards like that ;-)  To comply with a standard, there 
> just needs to be _some_ way to "spell" all the behaviors the standard 
> mandates. 


The IEEE 854 1987  is just such a standard (Mike Cowlishaw's extension).

The generalized  IEEE 754 2008  standard might not have as much wiggly
room.  
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/7c2ca224/attachment.html>

From rosuav at gmail.com  Sun Mar  9 03:46:21 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 9 Mar 2014 13:46:21 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
Message-ID: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>

On Sun, Mar 9, 2014 at 12:58 PM, Mark H. Harris <harrismh777 at gmail.com> wrote:
> hi Steven,  that's a straw man.   Your argument presumes that False is a
> surprise.  I am
> not in the least surprised.  As, I am also not surprised by this:
>
>>>> dscale(32)
> 10100
>>>> d(1)/3
> Decimal('0.33333333333333333333333333333333')   <=== not a surprise,
> expected
>>>> d(1)/3 + d(1)/3 +d(1)/3
> Decimal('0.99999999999999999999999999999999')   <=== not a surprise either,
> quite expected
>>>>

I was trying to construct a proof from memory that would support this
case, but I got tangled, so I cheated and went to Wikipedia for
something similar.

Let's start with two variables, a and b, which we shall suppose are equal.

# Postulation? Or is this called an axiom? I don't remember.
a = b
# Wave hands vigorously and conclude that:
a = 0

Are you surprised? Probably. Normally, two variables being equal
doesn't prove the value of either. Without knowing what actual
handwaving went on in there, that is and should be a surprising
result. Now, here's the handwaving, spelled out in full:

# As above, start here
a = b
# Multiply both sides by a
aa = ba
# Subtract bb (aka b squared, ASCII-friendly) from both sides
aa - bb = ba - bb
# Factor (both sides separately)
(a + b) (a - b) = b (a - b)
# Cancel out the multiplication by (a - b)
a + b = b
# Subtract b from both sides
a = 0

And there you are. To be unsurprised by the result, you have to know
exactly what happened in between, and how it's different from the
conventional rules of mathematics. Someone who already understands all
that may well not be surprised, but someone who's expecting sane
real-number arithmetic is in for a shock.

ChrisA

From tim.peters at gmail.com  Sun Mar  9 03:48:18 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 8 Mar 2014 20:48:18 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com>
 <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com>
Message-ID: <CAExdVNmw2zDZ3ghzSXAgNSST66xLF-j57j+4sYR2zJfPRyn6Qw@mail.gmail.com>

[Tim]
>> That would be true if the standard explicitly forbade extensions (any
>> optional operation or behavior not specified by the standard).  But
>> there are no standards like that ;-)  To comply with a standard, there
>> just needs to be _some_ way to "spell" all the behaviors the standard
>> mandates.

[Mark H. Harris]
> The IEEE 854 1987  is just such a standard (Mike Cowlishaw's extension).
>
> The generalized  IEEE 754 2008  standard might not have as much wiggly
> room.

I have the 2008 version of the standard; it's thoroughly ordinary in
these respects.

As usual, it's not even "a language" that's normally said to be
compliant with the standard.  Instead:

    Conformance to this standard is a property of a specific
    implementation of a specific programming environment, rather
    than of a language specification.

Although:

    However a language standard could also be said to conform to this standard
    if it were constructed so that every conforming implementation of
that language
    also conformed automatically to this standard.

So far as languages go, there's very little a standard of this nature
_can_ say!  Languages vary enormously in syntax and conventions.
Indeed, for example, the standard "spells" multiplication as

    multiplication(x, y)

That doesn't mean a conforming implementation has to supply a
2-argument function named "multiplication".  It just means a
conforming implementation has to supply _some_ way to spell what the
standard defines via "multiplication(x, y)".

Etc.

From harrismh777 at gmail.com  Sun Mar  9 04:05:04 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 19:05:04 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
Message-ID: <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com>



On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote:
 

> # As above, start here 
> a = b 
> # Multiply both sides by a 
> aa = ba 
> # Subtract bb (aka b squared, ASCII-friendly) from both sides 
> aa - bb = ba - bb 
> # Factor (both sides separately) 
> (a + b) (a - b) = b (a - b) 
> # Cancel out the multiplication by (a - b)      <======= if you divide by 
> zero, the universe will explode/  
> a + b = b 
> # Subtract b from both sides 
> a = 0 
>

I found a proof once that proved that 1==2,  but I can't find it...  let 
you know.

Anyway (laughs)   thanks for the grins

marcus 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/e3bd52f5/attachment.html>

From steve at pearwood.info  Sun Mar  9 04:15:16 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 9 Mar 2014 14:15:16 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
Message-ID: <20140309031516.GF28804@ando>

On Sat, Mar 08, 2014 at 09:12:53PM -0500, Alexander Belopolsky wrote:
> On Sat, Mar 8, 2014 at 7:39 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> > Computer maths cannot fail to give surprises.
> >
> > py> x = 1/Decimal(3)
> > py> sum([x, x, x]) == 1
> > False
> >
> > "No surprises" is simply an impossible request.
> >
> 
> Maybe, but there is nothing impossible in eliminating this particular
> surprise:

Of course not. Numeric surprises are like bubbles under wallpaper: you 
can push them around, move them from place to place, but you cannot 
eliminate them entirely. (The analogy is not perfect: with wallpaper, 
you can push the bubble out to the edge, and at least sometimes, 
eliminate it.) By using Decimal, you eliminate one class of surprises 
("why does my number not equal the decimal digits I typed in?") but at 
the cost of creating new surprises ("why is the average of x and y not 
always between x and y?").


> q)x:1 % 3f
> q)x
> 0.3333333
> q)1 = x + x + x
> 1b

If I were to write:

y:0.3333333

I'd expect y to equal x, since that's what x looks like. If it didn't, 
that would be a surprise. If it did, that would be a surprise too, 
because when I went to school, adding y three times ought to get 
0.9999999 and not 1.

Since Q is based on APL, I'm guessing that the surprise here is 
actually in the = equals operator. Is it doing a fuzzy comparison? If 
so, that's also surprising, since that means you can have triplets of 
numbers such that a = b and b = c but a != c.

> http://en.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems)
> 
> I am not suggesting that Python should behave like Q in this respect, just
> showing that this particular wart is not present in all languages.


-- 
Steven

From tim.peters at gmail.com  Sun Mar  9 04:21:15 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 8 Mar 2014 21:21:15 -0600
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com>
Message-ID: <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com>

[Mark H. Harris]
> I found a proof once that proved that 1==2,  but I can't find it...  let you
> know.

Dead easy:

>>> 1e100 + 1 == 1e100 + 2
True

Now just subtract 1e100 from both sides ;-)

From alexander.belopolsky at gmail.com  Sun Mar  9 04:41:14 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Mar 2014 22:41:14 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309031516.GF28804@ando>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
 <20140309031516.GF28804@ando>
Message-ID: <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com>

On Sat, Mar 8, 2014 at 10:15 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> Of course not. Numeric surprises are like bubbles under wallpaper: you
> can push them around, move them from place to place, but you cannot
> eliminate them entirely. (The analogy is not perfect: with wallpaper,
> you can push the bubble out to the edge, and at least sometimes,
> eliminate it.)
>

Great analogy!  I'll start using it with wallpaper covering floor,
ceiling as well as all four walls.


>
> Since Q is based on APL, I'm guessing that the surprise here is
> actually in the = equals operator. Is it doing a fuzzy comparison?
>

Exactly.


> If so, that's also surprising, since that means you can have triplets of
> numbers such that a = b and b = c but a != c.
>

Yes, but such triplets are not  as easy to stumble upon as 1/3 + 1/3 + 1/3
and lack of transitivity is not something that novices are likely to
complain
about.

You are absolutely right about not being able to eliminate all bubbles, but
we can move them away from the center of the wall to the corners and under
the wall-pictures.

In my view

>>> Decimal(1.1)
Decimal('1.100000000000000088817841970012523233890533447265625')

is a big bubble right in the middle of the wall.

The most obvious conversion from float to Decimal should implement the most
commonly needed operation.  I cannot imagine anyone needing to convert
floats to 53-digit decimals in a real life application.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/30480e3a/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 04:33:17 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 19:33:17 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com>
 <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com>
Message-ID: <b3963d58-6321-4b9c-bbcc-0a8637901ef9@googlegroups.com>



On Saturday, March 8, 2014 9:21:15 PM UTC-6, Tim Peters wrote:
 

> >>> 1e100 + 1 == 1e100 + 2 
> True 
>
> Now just subtract 1e100 from both sides ;-) 


~nice. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/fd7e546c/attachment-0001.html>

From harrismh777 at gmail.com  Sun Mar  9 04:48:47 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 19:48:47 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
 <20140309031516.GF28804@ando>
 <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com>
Message-ID: <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com>



On Saturday, March 8, 2014 9:41:14 PM UTC-6, Alexander Belopolsky wrote:
>
> In my view
>
> >>> Decimal(1.1)
> Decimal('1.100000000000000088817841970012523233890533447265625')
>
> is a big bubble right in the middle of the wall.
>
> The mos
>

hi Alexander,   um,  no...   that's a hole in the plaster,   with a mouse 
chewing 
his way through the wallpaper...     

;-) 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/2c4ef05e/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 04:27:02 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 19:27:02 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
Message-ID: <b0f98886-5a26-4355-8f9d-9046fca309c2@googlegroups.com>



On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote:
>
> # As above, start here 
> a = b 
> # Multiply both sides by a 
> aa = ba 
> # Subtract bb (aka b squared, ASCII-friendly) from both sides 
> aa - bb = ba - bb 
> # Factor (both sides separately) 
> (a + b) (a - b) = b (a - b)        <========of course all you really 
> proved is that  zero == zero
> # Cancel out the multiplication by (a - b) 
> a + b = b 
> # Subtract b from both sides 
> a = 0 


But then again, it really is a straw man (on steroids) because nobody is 
really surprised by
the absurdity

   postulate:  a=b
           {magic}
   conclusion:  a=0

At least no one should be surprised by an absurdity.  On the other hand 
once they  see
the proof run by them, um, quickly--- slight of hand, as it were, they 
might be surprised
if they don't catch it right away. 

{still laughing}

love it
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/75f0434a/attachment.html>

From alexander.belopolsky at gmail.com  Sun Mar  9 05:06:04 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 8 Mar 2014 23:06:04 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
 <20140309031516.GF28804@ando>
 <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com>
 <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com>
Message-ID: <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com>

On Sat, Mar 8, 2014 at 10:48 PM, Mark H. Harris <harrismh777 at gmail.com>wrote:
>
>
> hi Alexander,   um,  no...   that's a hole in the plaster,   with a mouse
> chewing
> his way through the wallpaper...
>


>
>
.. and what would you call this?

>>> int(6.02e23)
601999999999999995805696

http://en.wikipedia.org/wiki/Avogadro_constant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/672994fe/attachment.html>

From steve at pearwood.info  Sun Mar  9 06:02:07 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 9 Mar 2014 16:02:07 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
Message-ID: <20140309050207.GG28804@ando>

On Sat, Mar 08, 2014 at 01:54:11PM -0800, Mark H. Harris wrote:

> I am advocating for people--- I am going to say, " if you want just a 
> tad more than double just to check things out, please use the decimal 
> module and import pdeclib. These folks are what Guido called newbies, 
> I'll call them naive users, and these folks are using a calculator 
> (like the TI89) and they will be prone to keying in sqrt(2.345) /

When we get newbies asking why Python does not behave exactly like Java, 
we explain to them that Python is not Java and describe the differences. 
We do not try to make Python into a clone of Java. The same applies to 
newies who complain that Python does not behave exactly like Ruby, or 
PHP, or Perl, or Lisp.

I don't think TI-89 should be the exception. I don't think it is a good 
idea to try to make a general purpose programming language like Python 
into a (mathematical) clone of the TI-89 calculator just to satisfy the 
tiny minority of users who are numerically naive TI-89 users.


> Its just easier than explaining to them (at this point) all the stuff 
> they need to understand to use the functions

When you put it like that, it sounds like you are arguing, not that it 
is better for the users to avoid learning about floating point maths, 
but that it's easier for *you* if you don't need to explain floating 
point maths. That's okay. Point them to the python-list or tutor mailing 
lists, and the regulars there will be more than happy to teach them, to 
the best of our abilities.


> by making sure they feed decimals to the function; either by quoting 
> their numeric inputs, or even using a decimal literal, &c. They know 
> what the functions do, and will want to use them to get more digits, 
> but may not remember the rules to get the "correct" digits.  So I'm 
> advocating for them, to make it easier.
[...]
> Just trying to make it easier for the most people.

Who are these people you are trying to help? The user-base for Python is 
developers, and people trying to learn to be developers. Python is not 
an end-user application. Python is a programming language, and the 
people using it should be treated as if they were programmers.

I think that there are very few non-programmer, numerically-naive, TI-89 
users working with Python, and I think that shaping Python specifically 
for them is a mistake. It is completely appropriate to build an 
interactive calculator using Decimals aimed at the TI-89-using audience, 
and I applaud that. But that's an application written in Python, not 
Python itself.


> If it were just me,  I'd just make sure the functions got decimals and be 
> done with it. 

You would do that even though the majority of numeric users of Python 
would be opposed to such a change?



-- 
Steven

From harrismh777 at gmail.com  Sun Mar  9 07:09:40 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 22:09:40 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309050207.GG28804@ando>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
Message-ID: <5b6ee23e-974c-4eec-aeb1-5dfaaf79130d@googlegroups.com>



On Saturday, March 8, 2014 11:02:07 PM UTC-6, Steven D'Aprano wrote:

I don't think TI-89 should be the exception. I don't think it is a good 
> idea to try to make a general purpose programming language like Python 
> into a (mathematical) clone of the TI-89 calculator just to satisfy the 
> tiny minority of users who are numerically naive TI-89 users. 
>

hi Steven,  yeah, in some ways I wish I'd never mentioned the TI89, because 
I've
been misinterpreted (for which I am sorry). I mentioned the TI89 because it 
uses
decimal floating points by default , I think (as I believe python should 
also). I mentioned the
TI89 mainly as a "calculator" and that is the one that popped into my 
head... the
reason for mentioning it is that folks who poke a number into a calculator 
don't quote
the number, they don't add a "d" to the number, and they expect the 
calculator to 
interpret their number correctly.  ( it was a usability reference ) 

>
>
> > Its just easier than explaining to them (at this point) all the stuff 
> > they need to understand to use the functions 
>
> When you put it like that, it sounds like you are arguing, not that it 
> is better for the users to avoid learning about floating point maths, 
> but that it's easier for *you* if you don't need to explain floating 
> point maths. That's okay. Point them to the python-list or tutor mailing 
> lists, and the regulars there will be more than happy to teach them, to 
> the best of our abilities. 
>

    Well, its that too for sure, but its certainly more. Its a way for maths
and other explanations to be brought on-line in stages (learning mods) that
are progressive, and yet have the students be able to get the right results
from the beginning and not to be frustrated.  Again, a usability thing. 


> Who are these people you are trying to help? The user-base for Python is 
> developers, and people trying to learn to be developers. Python is not 
> an end-user application. Python is a programming language, and the 
> people using it should be treated as if they were programmers. 
>

    Unfortunately, I think that is the saddest thing I have heard you say 
in almost
seven years of knowing you. The earliest beginnings of Python (ABC at CWI) 
was to
"Stamp out BASIC" .   BASIC was designed at Dartmouth in 1964 to help 
average users and other scientists to be able to use a computer WITHOUT
having to be a developer (computer scientist, nor programmer).  Python has
the opportunity to be the BASIC of the 21st century (and beyond). Part of 
the
reason Python has NOT had a greater general acceptance is that it is 
becoming
a programming language for experts. This should not generally true, 
fortunately.  I mean
that Python is actually a VERY good language for newbies to start with, and 
its
a VERY good language for average people and other scientists to interact 
with 
a computer in sophisticated ways WITHOUT having to be a computer scientist
nor programmer.

>
> I think that there are very few non-programmer, numerically-naive, TI-89 
> users working with Python, and I think that shaping Python specifically 
> for them is a mistake. It is completely appropriate to build an 
> interactive calculator using Decimals aimed at the TI-89-using audience, 
> and I applaud that. But that's an application written in Python, not 
> Python itself. 
>

      Yes, and no.  Again, the TI89 has really nothing to do with it... I 
am advocating
for general average usability of the language by the masses; whether they 
are
software engineers or not, whether they are computer scientists or not, 
whether they
are have a PhD in mathematics or not. Python should be a both | and;   new 
folks 
should feel welcome and encouraged, and experienced folks should feel the 
power 
in their hands. Those ruby slippers should be the magic in the castle, and 
the way home
for Dorothy.  There are actually a large number of people trying out Python 
for the first
time, and many professional people and academic (like Matlab or Mathematica 
users) who are switching
to Python because of its power and flexibility (not to mention freedom). 
Both sets of "people" need to be welcomed
and cared for.  I honestly think that is one of the richest personal traits 
of the BDfL, and in
my observation (of course he should speak for himself) he seems to advocate 
for a both |and (at 
least that is how he makes me feel about it, when I have heard him speak 
publicly).

> If it were just me,  I'd just make sure the functions got decimals and be 
> > done with it. 
>
> You would do that even though the majority of numeric users of Python 
> would be opposed to such a change? 
>

     I am not sure I understand this question.  All I'm saying is that as a 
programmer, I am 
willing to take on constraints (spelling, rules, syntax, heaps of 
knowledge) in stride (as a 
software engineer that's my job) which I do not feel average people and 
other scientists
should have to worry about.  Just like at Dartmouth in 1964... a physicist 
should be able
to sit down to a python terminal and in very short order (with some small 
doc, tutorial) be
able to be talking to Python rapidly. The experience should not be 
frustrating, the experience
should in fact be as intuitive as possible, and it might also be fun...   
very seriously I believe this. 

     Python is a computer language.  But, it should not be C++ (we already 
have one of those), and
it should not be Java (for heaven's sake, certainly not that) and it should 
not be GNU C nor Assembler,
nor should it require a PhD in computer science to use it !  Python 
programming can and should be a rich 
and rewarding "liberal art"  which may be learned by anyone (even children) 
to accomplish real
world tasks with a rapid development cycle and a minimal of head banging 
(whatever I mean by that, its
all debatable). 

     I ran into this at IBM all the time too...  sooner or later developers 
become arrogant and self-absorbed.
IBM engineers are sometimes their own best customers, and their own worst 
enemies.  I am hoping that
does not happen in the Python community.  Python community has a very rich 
and talented group of 
engineers, mathematicians, coders, coordinators, doc writers, pep writers, 
testers, and a super BDfL. 
It would be sad IMHO for the Python community to implode (that is, build a 
superstructure or play-box
for developers, forgetting what they are developing, and for whom).

    This is my philosophy, and I don't expect that everyone will share it 
(that's just not realistic). I do hope
that Python receives wider adoption in the future, and I do hope that more 
people (esp women, kids, young
people) will choose computer science (through Python education) and that 
more people will actually 
reach out and and learn programming as a liberal art.  I am sick frankly, 
of our point-and-grunt culture (here in 
the States, and elsewhere).   Python is a way out of  point-and-grunt,  and 
a way forward into thinking with
a computer as a liberal art--- and as fine art. It is the power of thinking 
in the 21st century, and it is the power
of communication, collaboration, and cooperation between people and nations 
in the 21st century. I mean it.

marcus

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/f5b8b8f3/attachment-0001.html>

From python at 2sn.net  Sun Mar  9 07:21:07 2014
From: python at 2sn.net (Alexander Heger)
Date: Sun, 9 Mar 2014 17:21:07 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309050207.GG28804@ando>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
Message-ID: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>

> I think that there are very few non-programmer, numerically-naive, TI-89
> users working with Python,

students.

For numerical applications there is Numpy.

> and I think that shaping Python specifically
> for them is a mistake. It is completely appropriate to build an
> interactive calculator using Decimals aimed at the TI-89-using audience,
> and I applaud that. But that's an application written in Python, not
> Python itself.

The decimal module really should only be used for financial
applications; there should be a separate high-precision float type.

Another note on the 'd' notation - this might be confused with the
FORTRAN 'd' for double precision number.  Would you write, 1.2e3d or
1.2d3?

-Alexander

From harrismh777 at gmail.com  Sun Mar  9 07:57:27 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 22:57:27 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com>
 <20140309031516.GF28804@ando>
 <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com>
 <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com>
 <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com>
Message-ID: <d769697b-a5fe-418a-a985-ba89f663da60@googlegroups.com>



On Saturday, March 8, 2014 10:06:04 PM UTC-6, Alexander Belopolsky wrote

.. and what would you call this?
>  
> >>> int(6.02e23)
> 601999999999999995805696         <=====   a big fat surprise, and WAY not 
> enough atoms / mole     :)
>

Well, the number of atoms|molecules in one mole of gas at standard 
temperature 
and pressure is actually more precisely   6.02214129x10^23  (depending on 
who you talk to)

However, back in the day--- Chem 201 UMKC 1975,  I set the cursor of my 
dual base 
log log slide rule (Pickett)  just to the right of  6  and then kept track 
of the decimal place
myself in my notebook...    ;-)

... but now I have python:   whoohoo!

>>> d(6.02214129e23)
Decimal('6.02214129E+23')
>>> av=d(6.02214129e23)
>>> int(av)
602214129000000000000000       <=====    yousah like this better?
>>> 

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/45ca7942/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 08:10:41 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 23:10:41 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
 <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
Message-ID: <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com>



On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote:
>
> {snip}


hi Alexander, Steve,  this is one of Mike Cowlishaw's papers regarding 
decimal 
floating point, hardware decimal floating point, and the IEEE 854 1987 
standard,
and other topics.  He mentions the TI89, and others, that use decimal 
floating
point arithmetic, with citations. 

http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf

Its a good read.

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/c7cdbd3d/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 07:35:00 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sat, 8 Mar 2014 22:35:00 -0800 (PST)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
 <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
Message-ID: <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com>



On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote:
 

> The decimal module really should only be used for financial 
> applications; there should be a separate high-precision float type. 
>

   No, no, no....   It may and should be used for financial calculations,
          so true,
but it may ALSO be used for a general purpose high precision (and) 
arbitrary precision decimal floating point. Otherwise it might as well have
been patterned after the fixed width standard.  It was patterned after 
Cowlishaw's extension. 

>
> Why do you say only financial?


 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/16b5b976/attachment.html>

From steve at pearwood.info  Sun Mar  9 08:32:29 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 9 Mar 2014 18:32:29 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <531B92C4.3040502@stoneleaf.us>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us>
Message-ID: <20140309073229.GJ28804@ando>

On Sat, Mar 08, 2014 at 01:59:32PM -0800, Ethan Furman wrote:

> Python 3.4.0b3+ (default:aab7258a31d3, Feb  7 2014, 10:48:46)
> [GCC 4.7.3] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> --> from decimal import Decimal as D
> --> 9017.0109812864350067128347806
> 9017.010981286436
> --> D(9017.0109812864350067128347806)
> Decimal('9017.01098128643570817075669765472412109375')
> 
> In case my question isn't obvious, the direct float got me 16 digits, while 
> the Decimal float got me 42 digits.  Why is the Decimal more "accurate" 
> that the float it came from?

That's an interesting question. It does appear to be a lot more digits 
than needed. Here's that float exactly, in hex:

py> 9017.010981286436.hex()
'0x1.19c8167d5b50ep+13'

Let's convert it to decimal:


py> digits = list('19c8167d5b50e')
py> for i, c in enumerate(digits):
...     digits[i] = '0123456789abcdef'.index(c)
...
py> d = Decimal(0)
py> for n in digits[::-1]:
...     d += n
...     d /= 16
...
py> d += 1
py> d *= 2**13
py> d
Decimal('9017.010981286435708170756694')

Note that this round-trips back to float correctly:

py> assert float(d) == 9017.010981286436
py>


So I must admit, I'm not sure where the extra digits come from:

Decimal('9017.01098128643570817075669765472412109375')
Decimal('9017.010981286435708170756694')

but perhaps my calculation was rather naive.



-- 
Steven

From python at 2sn.net  Sun Mar  9 10:37:32 2014
From: python at 2sn.net (Alexander Heger)
Date: Sun, 9 Mar 2014 20:37:32 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
 <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
 <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com>
Message-ID: <CAN3CYHz-L5fKy2D7pZ4BAJw=h_89EW+KC9TF_jmCaC6OBkW6kA@mail.gmail.com>

>> Why do you say only financial?

Arbitrary precision is good - but why should it be done in a way that
is not natural to how machines work?
But I admit, I have not looked at the code how it is implemented.
Maybe it is done well (only saving integer part and an exponent).  I
only recall the BCD standard used in Turbo Pascal - which was not so
great - and really mostly for financial use.

But there is no point in having fractions in decimal for computation,
unless the result has to have a good representation in a specific
base.  Something you can send to a tax office.  The decimal package
solves problems specific to financial.  And, yes, it can be used for
other applications lacking other arbitrary precision classes in Python
standard class library - but it would not be my first choice for other
applications.

Hence I propose to have a different arbitrary precision package for
applications that do not require 'decimal' precision. As for class
hierarchy it likely should be a base class for the decimal package.
(Then you could even add other bases as well.)

From p.f.moore at gmail.com  Sun Mar  9 13:24:41 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 9 Mar 2014 12:24:41 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <20140305135646.GB28804@ando>
 <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com>
 <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com>
 <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com>
 <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com>
 <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com>
 <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com>
 <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
Message-ID: <CACac1F_BaUaX+w3xkRo0kJ9TjzS9EisKhnAjE3AiTEVwgRfMMA@mail.gmail.com>

On 8 March 2014 23:36, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
> This reminded me a discussion we had with Mark Dickinson on the bug tracker:
> "When a user is entering 0.6112295, she means 0.6112295, not
> 0x1.38f312b1b36bdp-1 or
> 0.61122949999999998116351207499974407255649566650390625 which are exact
> values of the underlying binary representation of 0.6112295."

However, when a user enters 0.6112295, she presumably *also* means she
wants a float value (assuming she understands how literals in Python
work). What the user *doesn't* necessarily realise is that those two
desires are mutually contradictory (there *is* no float value for the
exact value 0.6112295). So the question becomes, what should we do
when the user asks for two mutually-incompatible things.

One option is to raise an exception. I presume everyone agrees that
this is not a sensible option :-) [0.1 is a syntax error, but 0.5 is
OK...]
Another option is to fail to satisfy the user's desire for exactly
0.6112295. But do the best we can given the limitations of floats.
That's current behaviour.
A third option would be to fail to satisfy the user's implied desire
for a float:
  - either by always giving a Decimal (which is a variation on the
original "decimal as the default non-integer type" proposal)
  - or by giving an exactly equivalent float if possible but a Decimal
otherwise (which means seemingly-similar literals have different
types)

The only one of these that feels palatable to me is the current behaviour.

The problem of course is that what the user *actually* wants is
context-sensitive behaviour - they don't *see* a float being passed to
a constructor, but rather a Decimal (or timedelta, or whatever)
literal being written using a slightly-clumsy functional notation.
That's why explicit decimal literals are a good fit for the actual
user issue here. (It also hints at a benefit to more general
user-defined literals, but let's not go there :-))

Paul

From rob.cliffe at btinternet.com  Sun Mar  9 13:31:28 2014
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sun, 09 Mar 2014 12:31:28 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com>
 <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com>
 <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
Message-ID: <531C5F20.4090000@btinternet.com>


On 08/03/2014 18:49, Mark Dickinson wrote:
>
>
> >>> x = 49534541648432951
>
> >>> y = x + 2.0
>
> >>> x < y
>
> True
>
> >>> Decimal(x) > Decimal(repr(y))  # simulating proposed meaning of 
> Decimal(y)
>
> True
>
>
>
You don't need to add 2.0 to x to see this effect.  You can write
     y = float(x)
or even
     y = x - 3.0
and you get the same value for y.

So in fact you can see

 >>> x  <  x - 3.0
True

which might be surprising at first sight.

[Just for amusement :-) ]
Rob Cliffe

From stefan at bytereef.org  Sun Mar  9 13:36:41 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 9 Mar 2014 13:36:41 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309073229.GJ28804@ando>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando>
Message-ID: <20140309123641.GA30184@sleipnir.bytereef.org>

Steven D'Aprano <steve at pearwood.info> wrote:
> Note that this round-trips back to float correctly:
> 
> py> assert float(d) == 9017.010981286436

Many inexact values map to the same float.


> So I must admit, I'm not sure where the extra digits come from:
> 
> Decimal('9017.01098128643570817075669765472412109375')
> Decimal('9017.010981286435708170756694')

Without looking at your calculation, I guess it is rounding:

>>> 9017.010981286436.as_integer_ratio()
(2478577105427079, 274877906944)
>>> Decimal(2478577105427079) / 274877906944
Decimal('9017.010981286435708170756698')
>>> getcontext().prec = 100
>>> Decimal(2478577105427079) / 274877906944
Decimal('9017.01098128643570817075669765472412109375')



Stefan Krah



From ron3200 at gmail.com  Sun Mar  9 15:28:53 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 09 Mar 2014 09:28:53 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309123641.GA30184@sleipnir.bytereef.org>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando>
 <20140309123641.GA30184@sleipnir.bytereef.org>
Message-ID: <lfhtqn$5kq$1@ger.gmane.org>



On 03/09/2014 07:36 AM, Stefan Krah wrote:
> Steven D'Aprano<steve at pearwood.info>  wrote:
>> >Note that this round-trips back to float correctly:
>> >
>> >py> assert float(d) == 9017.010981286436

> Many inexact values map to the same float.

To get the same result as Stevens calculation, the context needs to be set 
to 16.

 >>> from decimal import *
 >>> setcontext(Context(16))
 >>> Decimal(2478577105427079) / 274877906944
Decimal('9017.010981286436')


When converting a float to Decimal... shouldn't the smallest (closest to 
zero) value that round trips be correct?  It seems like anything else is an 
implicit rounding up.  We don't add .5 when we convert an int to a float.

The rounding may be due to the financial need to average out small errors 
rather than have them accumulate.   ie.. ROUND_HALF_EVEN,

 >>> from decimal import *
 >>> getcontext()
Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, 
capitals=1, clamp=0, flags=[], traps=[InvalidOperation, DivisionByZero, 
Overflow])

Which is why it's suggested decimal should only be used for financial 
calculations.  But it's only the context that differs I think.  Decimals 
default context does seem to be for financial calculations.  Maybe there 
should be some easy to set contexts could be added.

     setcontext(MATH)
     setcontext(FINANCE)
     setcontext(SCIENCE)


Or even...

     setcontext(CALCULATOR)


>> > So I must admit, I'm not sure where the extra digits come from:
>> >
>> > Decimal('9017.01098128643570817075669765472412109375')
>> > Decimal('9017.010981286435708170756694')

> Without looking at your calculation, I guess it is rounding:

>>>> >>> 9017.010981286436.as_integer_ratio()
> (2478577105427079, 274877906944)
>>>> >>> Decimal(2478577105427079) / 274877906944
> Decimal('9017.010981286435708170756698')
>>>> >>> getcontext().prec = 100
>>>> >>> Decimal(2478577105427079) / 274877906944
> Decimal('9017.01098128643570817075669765472412109375')

Something that bothers me about the decimal module is how the rest of the 
numbers are converted.  (Not the ones entered in by hand.)

If we should enter x = Decimal('some number') to get the exact value, how 
does Decimal get y in xy = x * y if y is a float?  Would it be the same as 
Decimal(y)?  (Without quotes)

Cheers,
    Ron


From solipsis at pitrou.net  Sun Mar  9 15:59:56 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 9 Mar 2014 15:59:56 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us>
Message-ID: <20140309155956.75ad1ca5@fsol>

On Sat, 08 Mar 2014 13:59:32 -0800
Ethan Furman <ethan at stoneleaf.us> wrote:
> So, how is this justified?
> 
> Python 3.4.0b3+ (default:aab7258a31d3, Feb  7 2014, 10:48:46)
> [GCC 4.7.3] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> --> from decimal import Decimal as D
> --> 9017.0109812864350067128347806
> 9017.010981286436
> --> D(9017.0109812864350067128347806)
> Decimal('9017.01098128643570817075669765472412109375')
> 
> In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits.  Why is 
> the Decimal more "accurate" that the float it came from?

Both representations need to uphold the round-tripping property:

  float(str(some_float)) == some_float
  Decimal(str(some_decimal)) == some_decimal

However, since Decimal has arbitrary precision (as opposed to a fixed
number of bits), the number of digits needed in the repr() can be much
higher in order to uphold the round-tripping property:

  >>> float('1.1') == 1.1
  True
  >>> decimal.Decimal('1.1') == decimal.Decimal(1.1)
  False

Moreover, the Decimal constructor strives to uphold the property that
Decimal(some_number) == some_number (using the required internal
precision).

Therefore, decimal.Decimal(1.1) is the exact decimal value of the
binary floating-point number *best approaching* the decimal literal '1.1':

  >>> decimal.Decimal(1.1)
  Decimal('1.100000000000000088817841970012523233890533447265625')
  >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1
  True

Indeed, the difference is less than a binary floating-point mantissa's
resolution:

  >>> decimal.Decimal(1.1) - decimal.Decimal('1.1')
  Decimal('8.881784197001252323389053345E-17')
  >>> 2**(-sys.float_info.mant_dig)
  1.1102230246251565e-16

However, since Decimal has such a high potential resolution, changing even one
faraway digit will break the equality:

  >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265626') == 1.1
  False

  (I changed the trailing '5' to a '6')

Which is why Decimal(1.1) has to be so precise, and so has its repr() too.

Of course, if you convert those literals to floats, they turn out to convert to
the same binary floating-point number:

  >>> 1.1 == 1.100000000000000088817841970012523233890533447265625
  True

float() can therefore afford to have a "user-friendly" repr() in some
cases where Decimal can't.

Regards

Antoine.



From solipsis at pitrou.net  Sun Mar  9 16:05:14 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 9 Mar 2014 16:05:14 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
Message-ID: <20140309160514.3d86ae24@fsol>

On Sun, 9 Mar 2014 16:02:07 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> 
> I don't think TI-89 should be the exception. I don't think it is a good 
> idea to try to make a general purpose programming language like Python 
> into a (mathematical) clone of the TI-89 calculator just to satisfy the 
> tiny minority of users who are numerically naive TI-89 users.

I'm surprised: are physical calculators (TI-89) still in use today?

Regards

Antoine.



From ron3200 at gmail.com  Sun Mar  9 16:19:00 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 09 Mar 2014 10:19:00 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309155956.75ad1ca5@fsol>
References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com>
 <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com>
 <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com>
 <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us> <20140309155956.75ad1ca5@fsol>
Message-ID: <lfi0om$34u$1@ger.gmane.org>



On 03/09/2014 09:59 AM, Antoine Pitrou wrote:
> Therefore, decimal.Decimal(1.1) is the exact decimal value of the
> binary floating-point number*best approaching*  the decimal literal '1.1':
>
>    >>> decimal.Decimal(1.1)
>    Decimal('1.100000000000000088817841970012523233890533447265625')
>    >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1
>    True
>
> Indeed, the difference is less than a binary floating-point mantissa's
> resolution:

Yes, I realized that using the number closest to zero, is an implicit 
rounding down, after I posted.

The increases resolution accounted for the slightly less results in the 
examples.  But because it was using the value closest to the decimal 
representation (the quoted value) instead of the exact float value.  The 
higher resolution might be slightly higher or lower in that case and have 
extra, or even possibly (but more rarely) fewer digits. (is that right?)

The decimal is more accurate in that manner.  But more picky about quality 
too.  Looks like there is a way to set that in the context.. So that (value 
+- some error amount) is used in the equality tests.  Anyway need to go... 
  Check back later this evening.

Cheers,
    Ron



From tim.peters at gmail.com  Sun Mar  9 16:52:30 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 9 Mar 2014 10:52:30 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309073229.GJ28804@ando>
References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com>
 <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com>
 <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com>
 <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando>
Message-ID: <CAExdVNkDm6x65O=YYJ48=GUPiVMP3eZutVo9j-T++L8Z7JOOLw@mail.gmail.com>

[Ethan Furman]
>> Python 3.4.0b3+ (default:aab7258a31d3, Feb  7 2014, 10:48:46)
>> [GCC 4.7.3] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>> --> from decimal import Decimal as D
>> --> 9017.0109812864350067128347806
>> 9017.010981286436
>> --> D(9017.0109812864350067128347806)
>> Decimal('9017.01098128643570817075669765472412109375')
>>
>> In case my question isn't obvious, the direct float got me 16 digits, while
>> the Decimal float got me 42 digits.  Why is the Decimal more "accurate"
>> that the float it came from?

[Steven D'Aprano]
> That's an interesting question. It does appear to be a lot more digits
> than needed. Here's that float exactly, in hex:
>
> py> 9017.010981286436.hex()
> '0x1.19c8167d5b50ep+13'
>
> Let's convert it to decimal:
>
>
> py> digits = list('19c8167d5b50e')
> py> for i, c in enumerate(digits):
> ...     digits[i] = '0123456789abcdef'.index(c)
> ...
> py> d = Decimal(0)
> py> for n in digits[::-1]:
> ...     d += n
> ...     d /= 16
> ...
> py> d += 1
> py> d *= 2**13
> py> d
> Decimal('9017.010981286435708170756694')
>
> ,,,
> So I must admit, I'm not sure where the extra digits come from:
>
> Decimal('9017.01098128643570817075669765472412109375')
> Decimal('9017.010981286435708170756694')
>
> but perhaps my calculation was rather naive.

The calculation is fine, but recall that decimal calculations are
limited to 28 significant digits by default.  It's no coincidence that
len('9017.010981286435708170756694') == 29 (28 digits + a decimal
point).  The answer has "suffered" multiple rounding errors to make it
fit in 28 digits.

Boost the context precision before you start, and you'll get more
digits.  Keep boosting it, and you'll eventually get
9017.01098128643570817075669765472412109375 followed by (zero or more)
trailing zeroes.

A more direct way is to view this as an integer problem:  mathematically,

0x1.19c8167d5b50ep+13 is
0x119c8167d5b50e / 2**(4*13 - 13) =
0x119c8167d5b50e / 2**39 =  (multiplying top and bottom by 5**39)
0x119c8167d5b50e * 5**39 / 10**39

and now it's in the form of an integer numerator divided by a power of
10.  The numerator is:

>>>  0x119c8167d5b50e * 5**39
9017010981286435708170756697654724121093750

From harrismh777 at gmail.com  Sun Mar  9 19:53:34 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sun, 9 Mar 2014 11:53:34 -0700 (PDT)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309160514.3d86ae24@fsol>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol>
Message-ID: <49e2c298-5bee-4797-8dd9-3d8fc7ea0a0b@googlegroups.com>



On Sunday, March 9, 2014 10:05:14 AM UTC-5, Antoine Pitrou wrote:
>
> > tiny minority of users who are numerically naive TI-89 users. 
>
> I'm surprised: are physical calculators (TI-89) still in use today? 
>

Oh my, yes.  They are the premier programmable graphing calculator
today (not the only one, or course) that supports advanced computation
(calc, linear algebra, &c) and with not only polar and rect coord graphing
but also 3D graphing.  Its native language is BASIC, in the calculator 
supports a plethora  of applications for science, business, academia, 
engineering, and also a host of organizer apps like calendar, notepad, 
spread sheet (yes, really) and more. Its a phenomenal device, really.

The TI84+ is also still marketed heavily; and it has been updated lately
with a color screen. It is not sufficient for college engineering work, but 
its fine for high school and general programming/graphing. Its language
is also BASIC.

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/31c880f4/attachment.html>

From harrismh777 at gmail.com  Sun Mar  9 20:00:14 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sun, 9 Mar 2014 12:00:14 -0700 (PDT)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com>
References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando>
 <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com>
 <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com>
Message-ID: <db797bdc-d6a5-4bd7-8fc7-a5c0d5c18ad0@googlegroups.com>



On Sunday, March 9, 2014 1:10:41 AM UTC-6, Mark H. Harris wrote:
 

>   this is one of Mike Cowlishaw's papers regarding decimal 
> floating point, hardware decimal floating point, and the IEEE 854 1987 
> standard,
> and other topics.  He mentions the TI89, and others, that use decimal 
> floating
> point arithmetic, with citations. 
>
> http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf
>

In the course of this thread I have mentioned Michael Cowlishaw's Rexx, and 
my
work at IBM;  for one thing adding a library of decimal floating point 
routines for
use with VM370 Rexx which provides the scientific and transcendental 
functions
for the now arcane 370 system.  I found one of the scripts in my archive 
and 
uploaded it to code.google.com for historical purposes and perspective. If 
anyone
is interested it can be found here:

https://code.google.com/p/rexxdecimallibrary/


Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/59f4d99a/attachment-0001.html>

From steve at pearwood.info  Sun Mar  9 20:01:46 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 10 Mar 2014 06:01:46 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309160514.3d86ae24@fsol>
References: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol>
Message-ID: <20140309190145.GA12595@ando>

On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote:
> On Sun, 9 Mar 2014 16:02:07 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
> > 
> > I don't think TI-89 should be the exception. I don't think it is a good 
> > idea to try to make a general purpose programming language like Python 
> > into a (mathematical) clone of the TI-89 calculator just to satisfy the 
> > tiny minority of users who are numerically naive TI-89 users.
> 
> I'm surprised: are physical calculators (TI-89) still in use today?

I don't know about other places, but in Australia graphing calculators, 
including the TI-89 series, are required for all high school school 
students doing maths.



-- 
Steven

From harrismh777 at gmail.com  Sun Mar  9 20:32:03 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Sun, 9 Mar 2014 12:32:03 -0700 (PDT)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140309190145.GA12595@ando>
References: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com>
 <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com>
 <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com>
 <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com>
 <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com>
 <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol>
 <20140309190145.GA12595@ando>
Message-ID: <0b19b5d8-e776-4820-9922-7058e7bd3b10@googlegroups.com>



On Sunday, March 9, 2014 2:01:46 PM UTC-5, Steven D'Aprano wrote:
>
> On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote: 
> > I'm surprised: are physical calculators (TI-89) still in use today? 
>
> I don't know about other places, but in Australia graphing calculators, 
> including the TI-89 series, are required for all high school school 
> students doing maths. 


The same is true in the States too; well at least in Southeast Minnesota 
where
I reside.  Also, I have a son who is planning on engineering school at Iowa
State next fall;  they require the  TI89 {preferably} or equiv for their 
course work.

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/bb98c393/attachment.html>

From guido at python.org  Sun Mar  9 20:34:12 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 9 Mar 2014 12:34:12 -0700
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
Message-ID: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>

I would dearly like to put this thread to rest, as it has strayed mightily
from the topic of improvements to Python, and all points of view have been
amply defended. I'm hoping to hear from Cowlishaw, but I expect he'll side
with one of Mark Dickinson's proposals. I hope that somebody will take up
the task to write a PEP about introducing a decimal literal; that sounds
like an obtainable goal, less controversial than changing Decimal(<float>).

I did do some more thinking about how the magic repr() affects the
distribution of values, and came up with an example of sorts that might
show what it does. We've mostly focused on simple like 1.1, but to
understand the distribution issue it's better to look at a very large value.

I took 2**49 as an example and added a random fraction. When printed this
always gives a single digit past the decimal point, e.g. 562949953421312.5.
Then I measured the distribution of the last digit. What I found matched my
prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 occurred with roughly equal
probability (1/8th). So 3 and 7 are completely missing.

The explanation is simple enough: using the (current) Decimal class it's
easy to see that there are only 8 possible actual values, whose fractional
part is a multiple of 1/8. IOW the exact values end in .000, .125, .250,
.375, .500, .625, .750, .875. (*) The conclusion is that there are only 3
bits represented after the binary point, and repr() produces a single digit
here, because that's all that's needed to correctly round back to the 8
possible values. So it picks the digit closest to each of the possible
values, and when there are two possibilities it picks one. I don't know how
it picks, but it is reproducible -- in this example it always chooses .2 to
represent .250, and .8 to represent .750. The exact same thing happens
later in the decimal expansion for smaller numbers.

I think that the main objection to this distribution is that, while the
exact distribution is evenly spaced (the possibilities are 1/8th away from
each other), the rounded distribution has some gaps of 1/10th and some of
1/5th. I am not enough of a statistician to know whether this matters (the
distribution of *digits* in the exact values is arguably less randomized
:-) but at least this example clarified my understanding of the phenomenon
we're talking about when we discuss distribution of values.

________
(*) I was momentarily startled to find that the set of Decimals produced
contained 9 items, until I realized that some random() call must have
produced a value close enough to 1 to be rounded up.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/fb4ac37c/attachment.html>

From oscar.j.benjamin at gmail.com  Sun Mar  9 21:07:39 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sun, 9 Mar 2014 20:07:39 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
Message-ID: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>

On 9 March 2014 19:34, Guido van Rossum <guido at python.org> wrote:
>
> I hope that somebody will take up the task to write a PEP about introducing
> a decimal literal; that sounds like an obtainable goal, less controversial
> than changing Decimal(<float>).

I said I'd have a go at this. It's not as simple as it might seem though.

The simplest proposal would be to say that 3.14d is equivalent to
Decimal('3.14'). The fact the the constructor is independent of the
context is good here. It means that the value of the literal can be
determined statically which is good for trying to understand what code
does and also for constant folding etc.

The problem though is with things like +3.14d or -3.14d. Python the
language treats the + and - signs as not being part of the literal but
as separate unary operators. This is unimportant for int, float and
imaginary literals because there unary + is a no-op and unary - is
exact. For decimal this is not the same as +Decimal('3.14') is not the
same as Decimal('+3.14'):

    >>> from decimal import Decimal, getcontext
    >>> getcontext().prec = 3
    >>> Decimal('+1234')  # Exact
    Decimal('1234')
    >>> +Decimal('1234')  # Rounded
    Decimal('1.23E+3')
    >>> Decimal('-1234')  # Exact
    Decimal('-1234')
    >>> -Decimal('1234')  # Rounded
    Decimal('-1.23E+3')

I think that this behaviour would be surprising from a literal. I
would prefer to be able to say that -1234d was equivalent to
Decimal('-1234d') and would be guaranteed not to round. I just don't
really know if that's trivially easy or really difficult given the
current grammar specification and parse/compiler infrastructure.


Oscar

From guido at python.org  Sun Mar  9 21:39:01 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 9 Mar 2014 13:39:01 -0700
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
Message-ID: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>

Any solutions you are going to come up with would cause other anomalies
such as -(5d) not being equal to -5d. My vote goes to treating + and - as
the operators they are and telling people that if they want a negative
constant that exceeds the context's precision they're going to have to use
Decimal('-N'), rather than adding a terrible hack to the parser (and hence
to the language spec).


BTW Do you why the default precision is 28?


On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> On 9 March 2014 19:34, Guido van Rossum <guido at python.org> wrote:
> >
> > I hope that somebody will take up the task to write a PEP about
> introducing
> > a decimal literal; that sounds like an obtainable goal, less
> controversial
> > than changing Decimal(<float>).
>
> I said I'd have a go at this. It's not as simple as it might seem though.
>
> The simplest proposal would be to say that 3.14d is equivalent to
> Decimal('3.14'). The fact the the constructor is independent of the
> context is good here. It means that the value of the literal can be
> determined statically which is good for trying to understand what code
> does and also for constant folding etc.
>
> The problem though is with things like +3.14d or -3.14d. Python the
> language treats the + and - signs as not being part of the literal but
> as separate unary operators. This is unimportant for int, float and
> imaginary literals because there unary + is a no-op and unary - is
> exact. For decimal this is not the same as +Decimal('3.14') is not the
> same as Decimal('+3.14'):
>
>     >>> from decimal import Decimal, getcontext
>     >>> getcontext().prec = 3
>     >>> Decimal('+1234')  # Exact
>     Decimal('1234')
>     >>> +Decimal('1234')  # Rounded
>     Decimal('1.23E+3')
>     >>> Decimal('-1234')  # Exact
>     Decimal('-1234')
>     >>> -Decimal('1234')  # Rounded
>     Decimal('-1.23E+3')
>
> I think that this behaviour would be surprising from a literal. I
> would prefer to be able to say that -1234d was equivalent to
> Decimal('-1234d') and would be guaranteed not to round. I just don't
> really know if that's trivially easy or really difficult given the
> current grammar specification and parse/compiler infrastructure.
>
>
> Oscar
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/23c9e87b/attachment-0001.html>

From tim.peters at gmail.com  Sun Mar  9 22:33:41 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 9 Mar 2014 16:33:41 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
Message-ID: <CAExdVN=XyYZhQ4R9rm21N7CJxAEmtLriU7YUktFmR1vHgXoNvA@mail.gmail.com>

[Guido]
> ...
> I did do some more thinking about how the magic repr() affects the
> distribution of values, and came up with an example of sorts that might show
> what it does. We've mostly focused on simple like 1.1, but to understand the
> distribution issue it's better to look at a very large value.
>
> I took 2**49 as an example and added a random fraction. When printed this
> always gives a single digit past the decimal point, e.g. 562949953421312.5.
> Then I measured the distribution of the last digit. What I found matched my
> prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 occurred with roughly equal
> probability (1/8th). So 3 and 7 are completely missing.
>
> The explanation is simple enough: using the (current) Decimal class it's
> easy to see that there are only 8 possible actual values, whose fractional
> part is a multiple of 1/8. IOW the exact values end in .000, .125, .250,
> .375, .500, .625, .750, .875. (*) The conclusion is that there are only 3
> bits represented after the binary point, and repr() produces a single digit
> here, because that's all that's needed to correctly round back to the 8
> possible values. So it picks the digit closest to each of the possible
> values, and when there are two possibilities it picks one. I don't know how
> it picks, but it is reproducible -- in this example it always chooses .2 to
> represent .250, and .8 to represent .750.

Just the usual round-to-nearest/even.  0.25 and 0.75 are exactly
halfway, so round to the closest even retained digit (0.2 and 0.8).

> ...
> (*) I was momentarily startled to find that the set of Decimals produced
> contained 9 items, until I realized that some random() call must have
> produced a value close enough to 1 to be rounded up.

Not rare!  About half the trailing zeroes are accounted for by
rounding up fractions >= 0.9375 (1-1/16), and the other half from
rounding down fractions <= 0.0625 (1/16).  For example, here's the
full distribution across 10,000 tries:

562949953421312.0   631
562949953421312.1  1230
562949953421312.2  1224
562949953421312.4  1270
562949953421312.5  1245
562949953421312.6  1287
562949953421312.8  1271
562949953421312.9  1206
562949953421313.0   636

Except that .3 and .7 are missing (and 2 values must be missing
regardless of rounding rules), there's no bias toward higher or lower
numbers.  There is a bias toward trailing even digits, but that's an
intended consequence of the "even" in "round-to-nearest/even".

BTW, I still like changing Decimal(a_float) to work with
repr(a_float).  It's more-than-less unprincipled, but it would reduce
surprises for non-experts.  Ya, I know they may eventually bump into a
non-monotonic case (etc), but by the time they can _spell_ "monotonic"
they won't be newbies anymore ;-)

For a while I hung out on stackoverflow answering "Python fp is
broken!" complaints.  One thing I learned is that while the
questioners always had a shallow (to be charitable) understanding of
fp issues, exceedingly few had the slightest interest in gaining
deeper knowledge.  They just wanted their immediate problem to go
away, happy to accept any quick hack.  Will they get burned again
later?  Sure.  But you can't teach the unteachable.  What you can do
is delay the day they realize they're in trouble again ;-)

From stefan at bytereef.org  Sun Mar  9 22:57:15 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 9 Mar 2014 22:57:15 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
References: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
Message-ID: <20140309215714.GA3282@sleipnir.bytereef.org>

Guido van Rossum <guido at python.org> wrote:
> I would dearly like to put this thread to rest, as it has strayed mightily from
> the topic of improvements to Python, and all points of view have been amply
> defended. I'm hoping to hear from Cowlishaw, but I expect he'll side with one
> of Mark Dickinson's proposals. I hope that somebody will take up the task to
> write a PEP about introducing a decimal literal; that sounds like an obtainable
> goal, less controversial than changing Decimal(<float>).

One more thing that hasn't been mentioned yet: Java's BigDecimal constructor
appears to perform exact double -> Decimal conversions, unless a context
is passed:

http://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html


"""
BigDecimal(double val)
Translates a double into a BigDecimal which is the exact decimal representation of the double's binary floating-point value.
BigDecimal(double val, MathContext mc)
Translates a double into a BigDecimal, with rounding according to the context settings.
"""


I strongly suspect that this matches Mike Cowlishaw's preference.



Stefan Krah




From saghul at gmail.com  Sun Mar  9 23:02:16 2014
From: saghul at gmail.com (=?UTF-8?B?U2HDumwgSWJhcnJhIENvcnJldGfDqQ==?=)
Date: Sun, 09 Mar 2014 23:02:16 +0100
Subject: [Python-ideas] Extend socket.accept to put accepted socket in
 non-blocking mode
In-Reply-To: <20140307114039.720efff9@fsol>
References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol>
Message-ID: <531CE4E8.704@gmail.com>

On 03/07/2014 11:40 AM, Antoine Pitrou wrote:
> On Fri, 07 Mar 2014 10:37:58 +0100
> Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
> wrote:
>>
>> accept4 allows us to do this with a single syscall, and it made it into
>> FreeBSD 10, so that's another system that could benefit from this
>> optimization.
>
> If it doesn't, then perhaps the configure script needs to be fixed.
>

I checked and the condigure script seems to do the right thing, so 
FreeBSD 10 should just use accpet4 without nay change.

-- 
Sa?l Ibarra Corretg?
bettercallsaghul.com

From tjreedy at udel.edu  Sun Mar  9 23:22:29 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 09 Mar 2014 18:22:29 -0400
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
Message-ID: <lfipj6$3iv$1@ger.gmane.org>

On 3/9/2014 3:34 PM, Guido van Rossum wrote:
> I would dearly like to put this thread to rest, as it has strayed
> mightily from the topic of improvements to Python, and all points of
> view have been amply defended. I'm hoping to hear from Cowlishaw, but I
> expect he'll side with one of Mark Dickinson's proposals. I hope that
> somebody will take up the task to write a PEP about introducing a
> decimal literal; that sounds like an obtainable goal, less controversial
> than changing Decimal(<float>).
>
> I did do some more thinking about how the magic repr() affects the
> distribution of values, and came up with an example of sorts that might
> show what it does. We've mostly focused on simple like 1.1, but to
> understand the distribution issue it's better to look at a very large value.
>
> I took 2**49 as an example and added a random fraction. When printed
> this always gives a single digit past the decimal point, e.g.
> 562949953421312.5. Then I measured the distribution of the last digit.
> What I found matched my prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9
> occurred with roughly equal probability (1/8th). So 3 and 7 are
> completely missing.
>
> The explanation is simple enough: using the (current) Decimal class it's
> easy to see that there are only 8 possible actual values, whose
> fractional part is a multiple of 1/8. IOW the exact values end in .000,
> .125, .250, .375, .500, .625, .750, .875. (*) The conclusion is that
> there are only 3 bits represented after the binary point, and repr()
> produces a single digit here, because that's all that's needed to
> correctly round back to the 8 possible values. So it picks the digit
> closest to each of the possible values, and when there are two
> possibilities it picks one. I don't know how it picks, but it is
> reproducible -- in this example it always chooses .2 to represent .250,
> and .8 to represent .750. The exact same thing happens later in the
> decimal expansion for smaller numbers.

It is doing 'round to even'.

> I think that the main objection to this distribution is that, while the
> exact distribution is evenly spaced (the possibilities are 1/8th away
> from each other), the rounded distribution has some gaps of 1/10th and
> some of 1/5th. I am not enough of a statistician to know whether this
> matters (the distribution of *digits* in the exact values is arguably
> less randomized :-) but at least this example clarified my understanding
> of the phenomenon we're talking about when we discuss distribution of
> values.

The fact that the rounding rule produces one round down an one up and a 
symmetric distribution about 5 of non-0 final digits ameliorates any 
statistical problems (this is the reason for 'round to even'). If we add 
0 as a final digit, we must remember (as you noted in your footnote) 
that this is sometimes a result of rounding down to 0 and sometimes a 
result of rounding up to 10 (with the 1 added into the preceding digit).

I think the statistical issue you framed is this. We have numbers 
between 0 and 1 with a distribution that has a mean and variance. The 
uniform [0,1] distribution (mean .5, variance 1/12) is the simplest, 
though it may or may not be the most realistic in any particular 
situation.  We have two procedures.

1. Round each number to the nearest n/10, from 0/10 to 10/10.

2. Round each number to the nearest m/8 (0 <= m <= 8) and thence to some 
n/10 as described above.

If we compare the mean and variance of the rounded numbers, how much 
bias is introduced by the rounding process and is the bias introduced by 
the second enough more than by the first to worry about.

For a uniform distribution, there is no bias in the mean of rounded 
numbers. As I write this, I suspect there will be a small bias in the 
variance, and slightly more in the second, but not enough that I would 
worry about it. Let's see.

 >>> from statistics import pvariance as pvar
 >>> p1 = (0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10)
 >>> p2 = (0, 1, 1, 2, 2, 4, 4, 5, 5, 6, 6,  8, 8, 9, 9, 10)
 >>> print(1/12, pvar(p1)/100, pvar(p2)/100)
0.08333333333333333 0.085 0.09625

Hmmm. The removal of the .3s and .7s biases the digits away from .5, 
making the variance increase more than I expected. However, if I were 
looking as numbers from 0.0 to 10.0 (instead of 1.0), about half the .x 
double rounded digits would be moved toward 5.0 rather than away. Ditto 
for rounding 0 to 1.0 to 2 digits instead of 1. Either way, I would 
expect the double rounding bias to be quite reduced.

 >>> def end37(n):
	return (n % 10) in (3, 7)

 >>> p3 = list(range(100)) + list(range(1, 101))
 >>> p4 = list(i for i in range(100) if not end37(i)) + list(i for i in 
range(1, 101) if not end37(i))
 >>> print(1/12, pvar(p3)/10000, pvar(p4)/10000)
0.08333333333333333 0.08335 0.0834625

 >>> p5 = list(range(1000)) + list(range(1, 1001))
 >>> p6 = list(i for i in range(1000) if not end37(i)) + list(i for i in 
range(1, 1001) if not end37(i))
 >>> print(1/12, pvar(p5)/1000000, pvar(p6)/1000000)
0.08333333333333333 0.0833335 0.083334625

For any realistic number of significant digits, variance bias for 
uniformly distributed numbers is not an issue.

-- 
Terry Jan Reedy


From dickinsm at gmail.com  Mon Mar 10 09:32:16 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Mar 2014 08:32:16 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
Message-ID: <CAAu3qLUJDMrEk7RmrLrpdbJrhPZvzpaV6n8EcbRYqHr8+-BCrQ@mail.gmail.com>

On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin
<oscar.j.benjamin at gmail.com>wrote:

> The problem though is with things like +3.14d or -3.14d. Python the
> language treats the + and - signs as not being part of the literal but
> as separate unary operators. This is unimportant for int, float and
> imaginary literals because there unary + is a no-op and unary - is
> exact. For decimal this is not the same as +Decimal('3.14') is not the
> same as Decimal('+3.14'):
>

Another option would be to change the behaviour of + and - on Decimal
objects by binding `__pos__` and `__neg__` to the specification's 'copy'
and 'copy-negate' operations instead of (as currently) to the 'plus' and
'minus' operations.  Those behaviours are more in keeping with the way that
+ and - operate on other objects.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/565967ec/attachment-0001.html>

From oscar.j.benjamin at gmail.com  Mon Mar 10 11:07:11 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 10:07:11 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com>
 <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com>
 <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com>
 <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com>
 <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com>
 <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com>
 <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com>
 <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
Message-ID: <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>

On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote:
> On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>
>> The problem though is with things like +3.14d or -3.14d. Python the
>> language treats the + and - signs as not being part of the literal but
>> as separate unary operators. This is unimportant for int, float and
>> imaginary literals because there unary + is a no-op and unary - is
>> exact. For decimal this is not the same as +Decimal('3.14') is not the
>> same as Decimal('+3.14'):

> Any solutions you are going to come up with would cause other anomalies such
> as -(5d) not being equal to -5d. My vote goes to treating + and - as the
> operators they are and telling people that if they want a negative constant
> that exceeds the context's precision they're going to have to use
> Decimal('-N'), rather than adding a terrible hack to the parser (and hence
> to the language spec).

I think it's reasonable that -(5d) not be equal to -5d. Expert users
would exploit this behaviour the same way that they currently exploit
unary + for rounding to context. I don't think that non-expert users
would write that by mistake. I'm also not that bothered about +5d
rounding to context since the + is optional and current users of the
decimal module would probably expect that to round.

What I dislike about this is that it would mean that negative decimal
literals were essentially unsafe. It seems okay if you assume that the
precision is 28 or higher but users can set it to a lower value and
decimal contexts are unscoped so they have action at a distance on
other code. So if I have a module libmod.py with:

    # libmod.py

    def libfunc():
        a = -1.23465789d
        return f(a)

Then I have a script:

    # script.py

    from decimal import localcontext
    import libmod

    with localcontext() as ctx:
         ctx.prec = 5
         b = libmod.libfunc()

I think we could easily get into a situation where the author of
libmod just assumes that the decimal context has enough precision and
the author of script.py doesn't realise the effect that their change
of context has on libmod.

These kinds of things are generally problematic when using the decimal
module and the solution is typically that libfunc needs to take
control of the context. I think that it would be particularly
surprising with decimal literals though. The change in the precision
of "a" above might be a harmless reduction in the precision of the
result or it could lead to an infinite loop or basically anything else
depending on what libmod uses the value for.

> BTW Do you why the default precision is 28?

I have no idea. The standards I've read describe a few particular 9
digit contexts that should be provided by a conforming implementation
and decimal provides them but neither is the default:

>>> import decimal
>>> decimal.BasicContext.prec
9
>>> decimal.ExtendedContext.prec
9

AFAIK there's no standard that suggests 28 digits. There are other
common contexts such  as decimal32 (7 digits), decimal64 (16 digits),
and decimal128 (34 digits) and these are the contexts that Java's
BigDecimal provides (as well as an "unlimited" context).


Oscar

From oscar.j.benjamin at gmail.com  Mon Mar 10 11:16:00 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 10:16:00 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was:
 Re: Python Numbers as Human Concept Decimal System]
Message-ID: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>

On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>
> wrote:
>>
>> The problem though is with things like +3.14d or -3.14d. Python the
>> language treats the + and - signs as not being part of the literal but
>> as separate unary operators. This is unimportant for int, float and
>> imaginary literals because there unary + is a no-op and unary - is
>> exact. For decimal this is not the same as +Decimal('3.14') is not the
>> same as Decimal('+3.14'):
>
> Another option would be to change the behaviour of + and - on Decimal
> objects by binding `__pos__` and `__neg__` to the specification's 'copy' and
> 'copy-negate' operations instead of (as currently) to the 'plus' and 'minus'
> operations.  Those behaviours are more in keeping with the way that + and -
> operate on other objects.

Hi Mark and thanks for responding,

Do you think it's really feasible to make such a change? The use of
unary + for rounding to context has been a feature of the decimal
module for as long as I've been using it. It's shown in the docs both
in the Recipes section and the FAQ.

I just advised Mark Harris to do this and it know appears ~20 times in
his pdeclib module:
https://code.google.com/p/pythondecimallibrary/source/browse/pdeclib.py#207

I would expect that a change such as this would break quite a lot of
code (some of mine would break) so there would have to be some kind of
compatibility mechanism or deprecation process or something but I
can't think of a good way of doing it.


Oscar

From oscar.j.benjamin at gmail.com  Mon Mar 10 11:54:54 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 10:54:54 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
Message-ID: <CAHVvXxSMxPPYvMMHNQr1aRZKytd+GyE9qv9LndyF2M4bcYogAQ@mail.gmail.com>

On 10 March 2014 10:16, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote:
>> On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>
>> wrote:
>>>
>>> The problem though is with things like +3.14d or -3.14d. Python the
>>> language treats the + and - signs as not being part of the literal but
>>> as separate unary operators. This is unimportant for int, float and
>>> imaginary literals because there unary + is a no-op and unary - is
>>> exact. For decimal this is not the same as +Decimal('3.14') is not the
>>> same as Decimal('+3.14'):
>>
>> Another option would be to change the behaviour of + and - on Decimal
>> objects by binding `__pos__` and `__neg__` to the specification's 'copy' and
>> 'copy-negate' operations instead of (as currently) to the 'plus' and 'minus'
>> operations.  Those behaviours are more in keeping with the way that + and -
>> operate on other objects.
>
> Hi Mark and thanks for responding,
>
> Do you think it's really feasible to make such a change? The use of
> unary + for rounding to context has been a feature of the decimal
> module for as long as I've been using it. It's shown in the docs both
> in the Recipes section and the FAQ.

On the other hand the issue is primarily with __neg__ rather than
__pos__as I see it because a leading + is unnecessary and I really
just want a safe way of spelling a negative value. I can't see any
documented reference to the behaviour of Decimal.__neg__ and it's
probably used less often with the same intent.

So perhaps only __neg__ could be changed. This would still be a
backward-incompatible change but I think it would have significantly
less breakage.


Oscar

From steve at pearwood.info  Mon Mar 10 11:59:43 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 10 Mar 2014 21:59:43 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
Message-ID: <20140310105942.GB12595@ando>

On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:
> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote:
> > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> >>
> >> The problem though is with things like +3.14d or -3.14d. Python the
> >> language treats the + and - signs as not being part of the literal but
> >> as separate unary operators.

Is that still true? Possibly the peephole optimizer has changed the 
situation?

Using Python 1.5:

>>> from dis import dis
>>> dis(compile("-5", "", "single"))
          0 SET_LINENO          0

          3 SET_LINENO          1
          6 LOAD_CONST          0 (5)
          9 UNARY_NEGATIVE
         10 PRINT_EXPR
         11 LOAD_CONST          1 (None)
         14 RETURN_VALUE


but using Python 2.4:

py> dis(compile("-5", "", "single"))
  1           0 LOAD_CONST               0 (-5)
              3 PRINT_EXPR
              4 LOAD_CONST               1 (None)
              7 RETURN_VALUE

(versions more recent than 2.4 also use a constant).


> >> This is unimportant for int, float and
> >> imaginary literals because there unary + is a no-op and unary - is
> >> exact. For decimal this is not the same as +Decimal('3.14') is not the
> >> same as Decimal('+3.14'):
> 
> > Any solutions you are going to come up with would cause other anomalies such
> > as -(5d) not being equal to -5d. My vote goes to treating + and - as the
> > operators they are and telling people that if they want a negative constant
> > that exceeds the context's precision they're going to have to use
> > Decimal('-N'), rather than adding a terrible hack to the parser (and hence
> > to the language spec).
> 
> I think it's reasonable that -(5d) not be equal to -5d.

Did you leave the word "not" out of that sentence? :-)

I don't think that it is reasonable for -(5d) to not equal -5d. I think 
that's an awful interface, and terribly surprising. I don't think that 
there are any situations were an otherwise redundant pair of parens 
changes behavious. E.g. x*(y) is the same as x*y.

(Parens are sometimes used to disambiguate syntax or change precedence. 
That's different, since the parens aren't redundant.)

Considering that the motivation for this change is to make it easier for 
newbies and numerically naive users, I really do not relish having to 
explain to newbies why -5d and -(5d) are different.


-- 
Steven

From oscar.j.benjamin at gmail.com  Mon Mar 10 13:12:56 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 12:12:56 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310105942.GB12595@ando>
References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
Message-ID: <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com>

On 10 March 2014 10:59, Steven D'Aprano <steve at pearwood.info> wrote:
> On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:
>> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote:
>> > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>> >>
>> >> The problem though is with things like +3.14d or -3.14d. Python the
>> >> language treats the + and - signs as not being part of the literal but
>> >> as separate unary operators.
>
> Is that still true? Possibly the peephole optimizer has changed the
> situation?

Yes it does. It also does the same for "complex literals" even though
the language formally only defines imaginary literals. The question is
how to define exactly the grammar. Then once defined if the result is
independent of any context the optimiser can safely constant fold
decimal literals in the same way.

The grammar is here:
http://docs.python.org/3.4/reference/grammar.html

The relevant part is here:

factor: ('+'|'-'|'~') factor | power
power: atom trailer* ['**' factor]
atom: ('(' [yield_expr|testlist_comp] ')' |
       '[' [testlist_comp] ']' |
       '{' [dictorsetmaker] '}' |
       NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')

So if the specification is that decimal literals go into the NUMBER
part just like int, float and imaginary literals do then the + or -
are at a separate place. The expression -5 ** 2 is evaluated as -(5 **
2) which is -25 rather than (-5) ** 2 which is +25. It may be possible
but it's certainly not straight-forward to rewrite the grammar so that
-25d becomes D('-25') without affecting the normal precedence rules.

>> I think it's reasonable that -(5d) not be equal to -5d.
>
> Did you leave the word "not" out of that sentence? :-)

It's reasonable if you're an experienced user of the decimal module
but yeah okay it's not really reasonable in a general sense...

> I don't think that it is reasonable for -(5d) to not equal -5d. I think
> that's an awful interface, and terribly surprising. I don't think that
> there are any situations were an otherwise redundant pair of parens
> changes behavious. E.g. x*(y) is the same as x*y.
>
> (Parens are sometimes used to disambiguate syntax or change precedence.
> That's different, since the parens aren't redundant.)
>
> Considering that the motivation for this change is to make it easier for
> newbies and numerically naive users, I really do not relish having to
> explain to newbies why -5d and -(5d) are different.

Yes but newbies won't write -(5d). Experts might if it had the
semantics they want. The current behaviour of the Decimal type is
already surprising in this regard so unless __pos__ and __neg__ are
changed there will always be cases that surprise people.


Oscar

From ncoghlan at gmail.com  Mon Mar 10 13:57:56 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 10 Mar 2014 22:57:56 +1000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310105942.GB12595@ando>
References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
Message-ID: <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>

On 10 Mar 2014 21:06, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:
> > On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote:
> > > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <
oscar.j.benjamin at gmail.com> wrote:
> > >>
> > >> The problem though is with things like +3.14d or -3.14d. Python the
> > >> language treats the + and - signs as not being part of the literal
but
> > >> as separate unary operators.
>
> Is that still true? Possibly the peephole optimizer has changed the
> situation?
>
> Using Python 1.5:
>
> >>> from dis import dis
> >>> dis(compile("-5", "", "single"))
>           0 SET_LINENO          0
>
>           3 SET_LINENO          1
>           6 LOAD_CONST          0 (5)
>           9 UNARY_NEGATIVE
>          10 PRINT_EXPR
>          11 LOAD_CONST          1 (None)
>          14 RETURN_VALUE
>
>
> but using Python 2.4:
>
> py> dis(compile("-5", "", "single"))
>   1           0 LOAD_CONST               0 (-5)
>               3 PRINT_EXPR
>               4 LOAD_CONST               1 (None)
>               7 RETURN_VALUE
>
> (versions more recent than 2.4 also use a constant).

It actually makes the discrepancy worse, rather than better - the optimiser
is just doing constant folding at compile time, so it would just be
rounding using the context that applied at compile time. That context
dependence creates problems for the entire *notion* of constant folding
decimal literals - you can't constant fold them at compile time at all,
because the context may be wrong.

I think users of decimal literals will just need to deal with the risk of
unexpected rounding, as the alternatives are even more problematic.

Cheers,
Nick.

>
>
> > >> This is unimportant for int, float and
> > >> imaginary literals because there unary + is a no-op and unary - is
> > >> exact. For decimal this is not the same as +Decimal('3.14') is not
the
> > >> same as Decimal('+3.14'):
> >
> > > Any solutions you are going to come up with would cause other
anomalies such
> > > as -(5d) not being equal to -5d. My vote goes to treating + and - as
the
> > > operators they are and telling people that if they want a negative
constant
> > > that exceeds the context's precision they're going to have to use
> > > Decimal('-N'), rather than adding a terrible hack to the parser (and
hence
> > > to the language spec).
> >
> > I think it's reasonable that -(5d) not be equal to -5d.
>
> Did you leave the word "not" out of that sentence? :-)
>
> I don't think that it is reasonable for -(5d) to not equal -5d. I think
> that's an awful interface, and terribly surprising. I don't think that
> there are any situations were an otherwise redundant pair of parens
> changes behavious. E.g. x*(y) is the same as x*y.
>
> (Parens are sometimes used to disambiguate syntax or change precedence.
> That's different, since the parens aren't redundant.)
>
> Considering that the motivation for this change is to make it easier for
> newbies and numerically naive users, I really do not relish having to
> explain to newbies why -5d and -(5d) are different.
>
>
> --
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/64852efb/attachment.html>

From dickinsm at gmail.com  Mon Mar 10 14:30:47 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Mar 2014 13:30:47 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
Message-ID: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>

On Mon, Mar 10, 2014 at 10:16 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com
> wrote:

> On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote:
> >
> > Another option would be to change the behaviour of + and - on Decimal
> > objects by binding `__pos__` and `__neg__` to the specification's 'copy'
> and
> > 'copy-negate' operations instead of (as currently) to the 'plus' and
> 'minus'
> > operations.  Those behaviours are more in keeping with the way that +
> and -
> > operate on other objects.
>
> Hi Mark and thanks for responding,
>
> Do you think it's really feasible to make such a change?


Probably not. :-(  As you point out, there are significant backward
compatibility issues.  It's the solution I'd adopt if those concerns didn't
exists, and the one I'd still look for a way to get to even *with* those
concerns, but the level of breakage may make this out of the question.  I
doubt we're going to get to Decimal literals without breaking at least
*some* code, but this may be too much breakage.

I guess I have ulterior motives; (ab)using '+' to mean 'round to the
current context' has always felt way too implicit for my tastes, and I'd
jump at the chance to see it go away.  The unconventional behaviour with
respect to signed zeros bugs me, too.

>>> -0.0

-0.0

>>> -Decimal(0.0)  # negative sign mysteriously disappears.

Decimal('0')

It's perfectly legitimate, and part of the standard, but ... ugh.  I argued
with Mike Cowlishaw about it at one point.  I lost.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/a32fd8db/attachment-0001.html>

From dickinsm at gmail.com  Mon Mar 10 14:38:49 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Mar 2014 13:38:49 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com>
References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com>
Message-ID: <CAAu3qLWd6fNYW-DhehAOJKaXjk9kg3Oa1cHHOX-+nPsK4RJv5A@mail.gmail.com>

On Mon, Mar 10, 2014 at 12:12 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com
> wrote:

> On 10 March 2014 10:59, Steven D'Aprano <steve at pearwood.info> wrote:
> > On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:
> >> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote:
> >> > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <
> oscar.j.benjamin at gmail.com> wrote:
> >> >>
> >> >> The problem though is with things like +3.14d or -3.14d. Python the
> >> >> language treats the + and - signs as not being part of the literal
> but
> >> >> as separate unary operators.
> >
> > Is that still true? Possibly the peephole optimizer has changed the
> > situation?
>
> Yes it does. It also does the same for "complex literals" even though
> the language formally only defines imaginary literals.


... and don't forget the Python 2.x-only hack for negation of integers:

>>> type(-9223372036854775808)

<type 'int'>

>>> type(-(9223372036854775808))

<type 'long'>

which means that it's not true that Python 2.x behaves "as if" there were
no negative literals.  Python 3 is cleaner in this respect.  I guess this
shows that we *could* in theory reintroduce such a hack for negation of
decimal literals, but I agree with everyone else so far that that would be
a bad idea.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/22c26861/attachment.html>

From random832 at fastmail.us  Mon Mar 10 14:49:48 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 10 Mar 2014 09:49:48 -0400
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310105942.GB12595@ando>
References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com>
 <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com>
 <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
Message-ID: <1394459388.28493.92665413.408BEE02@webmail.messagingengine.com>

On Mon, Mar 10, 2014, at 6:59, Steven D'Aprano wrote:
> Is that still true? Possibly the peephole optimizer has changed the 
> situation?

The optimizer does not dictate the specification; rather, the optimizer
can do this because this operation is defined to have this result. If
+3.14d has a different value than 3.14d, then the optimizer should load
a different constant for these expressions.

From stefan at bytereef.org  Mon Mar 10 14:53:42 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 10 Mar 2014 14:53:42 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
Message-ID: <20140310135342.GA10968@sleipnir.bytereef.org>

[My apologies for being terse, I don't have much time to follow this
discussion right now.]


Nick Coghlan <ncoghlan at gmail.com> wrote:
> I think users of decimal literals will just need to deal with the risk of
> unexpected rounding, as the alternatives are even more problematic.

That is why I think we should seriously consider moving to IEEE semantics
for a decimal literal.  Among other things:

  - Always round the literal inputs.

  - Supply IEEE contexts.

  - Make Decimal64 the default.

  - Add the missing rounding modes to sqrt() and exp().

  - Keep the ability to create exact Decimals through the constructor when
    no context is passed.

  - Make the Decimal constructor use the context for rounding if it is passed.

  - ...

 
The existing specification is largely compatible with IEEE 754-2008:

  http://speleotrove.com/decimal/dascope.html


We can still support setting irregular (non IEEE) contexts.



Stefan Krah



From random832 at fastmail.us  Mon Mar 10 14:55:09 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 10 Mar 2014 09:55:09 -0400
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
Message-ID: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>

On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote:
> I guess I have ulterior motives; (ab)using '+' to mean 'round to the
> current context' has always felt way too implicit for my tastes, and I'd
> jump at the chance to see it go away.

It occurs to me that there is an unprecedented opportunity to make warts
"go away" - making Decimal a "first-class citizen" suggests moving it
from the decimal module to a built-in type. Any backwards-incompatible
change you want to make, you can give programs importing decimal the old
behavior, and reserve the new behavior for the new type in builtins.

Then remove the module from Python 4000.

From stefan at bytereef.org  Mon Mar 10 14:58:24 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 10 Mar 2014 14:58:24 +0100
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
Message-ID: <20140310135824.GB10968@sleipnir.bytereef.org>

Mark Dickinson <dickinsm at gmail.com> wrote:
> >>> -Decimal(0.0) ?# negative sign mysteriously disappears.
> 
> Decimal('0')

Unless the rounding mode is ROUND_FLOOR. :(

>>> getcontext().rounding = ROUND_FLOOR
>>> -Decimal(0.0)
Decimal('-0')
>>>


Stefan Krah



From oscar.j.benjamin at gmail.com  Mon Mar 10 15:05:22 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 14:05:22 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
Message-ID: <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com>

On 10 March 2014 13:53, Stefan Krah <stefan at bytereef.org> wrote:
> [My apologies for being terse, I don't have much time to follow this
> discussion right now.]
>
>
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I think users of decimal literals will just need to deal with the risk of
>> unexpected rounding, as the alternatives are even more problematic.
>
> That is why I think we should seriously consider moving to IEEE semantics
> for a decimal literal.  Among other things:
>
>   - Always round the literal inputs.
>
>   - Supply IEEE contexts.
>
>   - Make Decimal64 the default.
>
>   - Add the missing rounding modes to sqrt() and exp().
>
>   - Keep the ability to create exact Decimals through the constructor when
>     no context is passed.
>
>   - Make the Decimal constructor use the context for rounding if it is passed.
>
>   - ...
>
>
> The existing specification is largely compatible with IEEE 754-2008:
>
>   http://speleotrove.com/decimal/dascope.html
>
>
> We can still support setting irregular (non IEEE) contexts.

This is what I've been thinking about. Most non-expert users will be
very happy with Decimal64 and a single fixed context. There could be a
separate type called decimal64 in builtins that always used the same
standard context. Literals could create decimal64 instances.

The semantics of code that uses this type and decimal literals would
be independent of any arithmetic context which is good not just for
constant folding but for understanding. There would be no need to
explain what an arithmetic context is to new users. You can just say:
"here's a type that represents decimal values with 16 digits. It
sometimes needs to round if the result of a calculation exceeds 16
digits so it uses the standard decimal rounding mode XXX."

Where this would get complicated is for people who also use the
Decimal type. They'd need to keep track of which objects were of which
type and so decimal literals might seem more annoying than useful.


Oscar

From oscar.j.benjamin at gmail.com  Mon Mar 10 15:21:22 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 14:21:22 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
Message-ID: <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com>

On 10 March 2014 13:55,  <random832 at fastmail.us> wrote:
> On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote:
>>
>> I guess I have ulterior motives; (ab)using '+' to mean 'round to the
>> current context' has always felt way too implicit for my tastes, and I'd
>> jump at the chance to see it go away.

I tend to agree. How would you prefer to spell it? I guess something like:

    return d.contextround()

I've sometimes used it in a polymorphic kind of way so that a function
does the right thing with Decimal but also works with float/int etc.
(with the + just being a no-op). That's not really an argument against
your point though.

> It occurs to me that there is an unprecedented opportunity to make warts
> "go away" - making Decimal a "first-class citizen" suggests moving it
> from the decimal module to a built-in type. Any backwards-incompatible
> change you want to make, you can give programs importing decimal the old
> behavior, and reserve the new behavior for the new type in builtins.

That's also possible. I wonder what a good name for a new Decimal type
would be. It's unfortunate that calling it 'decimal' would sort of
conflict with people doing 'import decimal'.


Oscar

From dickinsm at gmail.com  Mon Mar 10 15:22:19 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Mar 2014 14:22:19 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
Message-ID: <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>

On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote:

> That is why I think we should seriously consider moving to IEEE semantics
> for a decimal literal.


I think that would make a *lot* of sense.

-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/299bac63/attachment.html>

From random832 at fastmail.us  Mon Mar 10 15:42:26 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 10 Mar 2014 10:42:26 -0400
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
 <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com>
Message-ID: <1394462546.13875.92685197.059D8531@webmail.messagingengine.com>

On Mon, Mar 10, 2014, at 10:21, Oscar Benjamin wrote:
> That's also possible. I wonder what a good name for a new Decimal type
> would be. It's unfortunate that calling it 'decimal' would sort of
> conflict with people doing 'import decimal'.

I don't think the conflict is a big deal. You can shadow builtins with
globals, and on the off chance that you want to use both, you can import
it as a different name. Any module that imports decimal is going to be
one that doesn't know or care about the built-in type.

To me, the more dicey aspect is what happens if you mix the types - if
you pick a type for mixed operations to convert to, then passing in the
'wrong' type will slowly 'infect' that module's data with a type which
has behavior subtly different from what it expects. You could disallow
mixed operations like it does now for floats, I suppose.

From guido at python.org  Mon Mar 10 16:01:03 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Mar 2014 08:01:03 -0700
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
Message-ID: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>

On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

> On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote:
>
> That is why I think we should seriously consider moving to IEEE semantics
>> for a decimal literal.
>
>
> I think that would make a *lot* of sense.
>

What's the proposal? Just using decimal64 for decimal literals, or
introducing decimal64 as a new builtin type? (I could get behind either
one.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/fcafca28/attachment.html>

From steve at pearwood.info  Mon Mar 10 16:19:23 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 11 Mar 2014 02:19:23 +1100
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
	[Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
Message-ID: <20140310151923.GD12595@ando>

On Mon, Mar 10, 2014 at 09:55:09AM -0400, random832 at fastmail.us wrote:
> On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote:
> > I guess I have ulterior motives; (ab)using '+' to mean 'round to the
> > current context' has always felt way too implicit for my tastes, and I'd
> > jump at the chance to see it go away.
> 
> It occurs to me that there is an unprecedented opportunity to make warts
> "go away" - making Decimal a "first-class citizen" suggests moving it
> from the decimal module to a built-in type. Any backwards-incompatible
> change you want to make, you can give programs importing decimal the old
> behavior, and reserve the new behavior for the new type in builtins.

Historically, I don't think Python has changed the API for types when 
they've moved from a module to a built-in. I can't think of any example 
except set, but apart from a change in name from set.Set to set, I don't 
think there were any changes.

Any other precedents?

On the other hand, the built-in doesn't need to be a replacement for the 
decimal module, but rather an alternative solution. If decimal64 was a 
built-in, presumably it wouldn't need contexts and would be a lot 
simpler. The decimal module would still be available for those with 
advanced needs.


-- 
Steven

From steve at pearwood.info  Mon Mar 10 16:21:49 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 11 Mar 2014 02:21:49 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org>
References: <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
Message-ID: <20140310152149.GE12595@ando>

On Mon, Mar 10, 2014 at 02:53:42PM +0100, Stefan Krah wrote:

> That is why I think we should seriously consider moving to IEEE semantics
> for a decimal literal.  Among other things:

Okay, I'm confused. I thought the decimal module had IEEE-754 semantics, 
and so I assumed that so would any decimal literal. What have I missed?



-- 
Steven

From steve at pearwood.info  Mon Mar 10 16:31:27 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 11 Mar 2014 02:31:27 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com>
References: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com>
Message-ID: <20140310153127.GF12595@ando>

On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:

> This is what I've been thinking about. Most non-expert users will be
> very happy with Decimal64 and a single fixed context. There could be a
> separate type called decimal64 in builtins that always used the same
> standard context. Literals could create decimal64 instances.
> 
> The semantics of code that uses this type and decimal literals would
> be independent of any arithmetic context which is good not just for
> constant folding but for understanding. There would be no need to
> explain what an arithmetic context is to new users. You can just say:
> "here's a type that represents decimal values with 16 digits. It
> sometimes needs to round if the result of a calculation exceeds 16
> digits so it uses the standard decimal rounding mode XXX."

All this sounds quite good to me. What's the catch? :-)


> Where this would get complicated is for people who also use the
> Decimal type. They'd need to keep track of which objects were of which
> type and so decimal literals might seem more annoying than useful.

Hmmm. I don't think this should necessarily be complicated, or at least 
no more complicated than dealing with any other mixed numeric types. If 
you don't want to mix them, don't mix them :-) 

(Perhaps there could be a Decimal context flag to allow/disallow mixed 
Decimal/decimal64 operations?)

I would expect that decimal64 and decimal.Decimal would be separate 
types. They might share parts of the implementation under the hood, but 
I don't think it would be necessary or useful to have decimal64 inherit 
from Decimal, or visa versa.


-- 
Steven

From oscar.j.benjamin at gmail.com  Mon Mar 10 16:54:16 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 15:54:16 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310152149.GE12595@ando>
References: <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org> <20140310152149.GE12595@ando>
Message-ID: <CAHVvXxRA9vjMe77tY7dp8vwENEHC-+XNo34uAKErZuyyyzMn-g@mail.gmail.com>

On 10 March 2014 15:21, Steven D'Aprano <steve at pearwood.info> wrote:
> On Mon, Mar 10, 2014 at 02:53:42PM +0100, Stefan Krah wrote:
>
>> That is why I think we should seriously consider moving to IEEE semantics
>> for a decimal literal.  Among other things:
>
> Okay, I'm confused. I thought the decimal module had IEEE-754 semantics,
> and so I assumed that so would any decimal literal. What have I missed?

IEEE 754 (2008) defines fixed width decimal types decimal64 and
decimal128. It also refers to "extended and extendable" with
"extendable" meaning something like the current Decimal type.

I think that Stefan is proposing that decimal literals should be
parsed as if the result needed to be stored in a fixed width decimal64
type. The standard defines all the "Details of conversion between
floating-point data and external character sequences" which would
regulate how this could be done.

Just parsing the number as decimal64 and using a decimal64 type
context by default is not enough to prevent the IMO confusing action
at a distance effect of Decimal contexts though. It also wouldn't
solve the issue around negative literals if a context with less
precision or exponent range than decimal64 was used.

If there were a separate type whose context always corresponded to
decimal64 then unary + and - on a literal would have the meaning that
people would expect and it would be possible to determine exactly what
the code would do without needing to place caveats on what the decimal
context is. I think that this would be the best approach if we're
aiming at non-expert users.


Oscar

From oscar.j.benjamin at gmail.com  Mon Mar 10 17:00:38 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 16:00:38 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310153127.GF12595@ando>
References: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com>
 <20140310153127.GF12595@ando>
Message-ID: <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com>

On 10 March 2014 15:31, Steven D'Aprano <steve at pearwood.info> wrote:
> On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:
>
>> This is what I've been thinking about. Most non-expert users will be
>> very happy with Decimal64 and a single fixed context. There could be a
>> separate type called decimal64 in builtins that always used the same
>> standard context. Literals could create decimal64 instances.
>>
>> Where this would get complicated is for people who also use the
>> Decimal type. They'd need to keep track of which objects were of which
>> type and so decimal literals might seem more annoying than useful.
>
> Hmmm. I don't think this should necessarily be complicated, or at least
> no more complicated than dealing with any other mixed numeric types. If
> you don't want to mix them, don't mix them :-)

Exactly. It just means that you wouldn't want to use the literals in
code that actually uses the decimal module. Consider:

if some_condition:
    x = 1d
else:
    x = Decimal(some_string)
# ...
y = x / 3

So now x / 3 rounds differently depending on whether x is a decimal64
or a Decimal. I probably don't want that. The solution: coerce to
Decimal. But then why did I bother with the Decimal literal anyway?
Decimal(1d) is hardly better than Decimal('1').


Oscar

From wolfgang.maier at biologie.uni-freiburg.de  Mon Mar 10 18:48:21 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Mon, 10 Mar 2014 17:48:21 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and float/decimal
	properties [Was: Re: Python Numbers as Human Concept Decimal System]
Message-ID: <loom.20140310T181321-988@post.gmane.org>

Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah:

> [My apologies for being terse, I don't have much time to follow this
discussion right now.]

> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I think users of decimal literals will just need to deal with the risk of
unexpected rounding, as the alternatives are even more problematic.

> That is why I think we should seriously consider moving to IEEE semantics
for a decimal literal.  Among other things:
> - Always round the literal inputs.
> - Supply IEEE contexts.
> - Make Decimal64 the default.
> - Add the missing rounding modes to sqrt() and exp().
> - Keep the ability to create exact Decimals through the constructor when
no context is passed.
> - Make the Decimal constructor use the context for rounding if it is passed.
> - ...

While I find this discussion about decimal literals extremely interesting,
in my opinion, such a literal should have an underlying completely new
numerical type, if it is really supposed to be for inexperienced users.

Most of the discussions right now concern rounding issues that occur after
the decimal point, but I think an at least equally big problem is rounding
*to the left* of it as in (using current float):

>>> 1e50 + 1000
1e+50

Importantly, Decimal is no cure here:

>>> Decimal(10**50) + Decimal(100)
Decimal('1.000000000000000000000000000E+50')

(of course, you can debate context settings to make this particular example
work, but, in general, it happens with big enough numbers.)

The solution for this example is using ints of course:

>>> 10**50 + 100
100000000000000000000000000000000000000000000000100

, but obviously this works only for whole numbers, so there currently is no
built-in way to make this example work correctly:

>>> 10**50 - 9999999999999999.5
1e+50

(or with Decimal:
>>> Decimal(10**50) - Decimal('9999999999999999.5')
Decimal('1.000000000000000000000000000E+50')
).

If we are discussing a new number literal I would like to have it cope with
this and my suggestion is a new numeric type that's sort of a hybrid between
int and either Decimal or float, i.e., a type that behaves like int for
digits left of the decimal point, but may truncate to the right.
In pure Python, something similar (but not a literal form of course) could
be implemented as a class that stores the left digits as an int and the
right digits as a float/Decimal internally. Calculations involving this
class would be slow due to the constant need of shifting digits between the
integer?and the float part, and might be too slow for many users even when
written in C, but its behavior would meet the expectation of inexperienced
people better than the existing types.
Going back to Mark Harris' initial proposal of unifying numeric types (which
I am not trying to support in any way here), such a type would even allow to
unify int and float since an ints could be considered a subset of the new
type with a fractional part of zero.

Cheers,
Wolfgang




From guido at python.org  Mon Mar 10 18:56:20 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Mar 2014 10:56:20 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140310T181321-988@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
Message-ID: <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>

On Mon, Mar 10, 2014 at 10:48 AM, Wolfgang Maier <
wolfgang.maier at biologie.uni-freiburg.de> wrote:

> Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah:
>
> > [My apologies for being terse, I don't have much time to follow this
> discussion right now.]
>
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> I think users of decimal literals will just need to deal with the risk
> of
> unexpected rounding, as the alternatives are even more problematic.
>
> > That is why I think we should seriously consider moving to IEEE semantics
> for a decimal literal.  Among other things:
> > - Always round the literal inputs.
> > - Supply IEEE contexts.
> > - Make Decimal64 the default.
> > - Add the missing rounding modes to sqrt() and exp().
> > - Keep the ability to create exact Decimals through the constructor when
> no context is passed.
> > - Make the Decimal constructor use the context for rounding if it is
> passed.
> > - ...
>
> While I find this discussion about decimal literals extremely interesting,
> in my opinion, such a literal should have an underlying completely new
> numerical type, if it is really supposed to be for inexperienced users.
>
> Most of the discussions right now concern rounding issues that occur after
> the decimal point, but I think an at least equally big problem is rounding
> *to the left* of it as in (using current float):
>
> >>> 1e50 + 1000
> 1e+50
>
> Importantly, Decimal is no cure here:
>
> >>> Decimal(10**50) + Decimal(100)
> Decimal('1.000000000000000000000000000E+50')
>
> (of course, you can debate context settings to make this particular example
> work, but, in general, it happens with big enough numbers.)
>
> The solution for this example is using ints of course:
>
> >>> 10**50 + 100
> 100000000000000000000000000000000000000000000000100
>
> , but obviously this works only for whole numbers, so there currently is no
> built-in way to make this example work correctly:
>
> >>> 10**50 - 9999999999999999.5
> 1e+50
>
> (or with Decimal:
> >>> Decimal(10**50) - Decimal('9999999999999999.5')
> Decimal('1.000000000000000000000000000E+50')
> ).
>
> If we are discussing a new number literal I would like to have it cope with
> this and my suggestion is a new numeric type that's sort of a hybrid
> between
> int and either Decimal or float, i.e., a type that behaves like int for
> digits left of the decimal point, but may truncate to the right.
> In pure Python, something similar (but not a literal form of course) could
> be implemented as a class that stores the left digits as an int and the
> right digits as a float/Decimal internally. Calculations involving this
> class would be slow due to the constant need of shifting digits between the
> integer?and the float part, and might be too slow for many users even when
> written in C, but its behavior would meet the expectation of inexperienced
> people better than the existing types.
> Going back to Mark Harris' initial proposal of unifying numeric types
> (which
> I am not trying to support in any way here), such a type would even allow
> to
> unify int and float since an ints could be considered a subset of the new
> type with a fractional part of zero.
>

I think what you're proposing here is variant of fixed-point numbers. The
representation you seem to be looking for is an arbitrary-precision integer
plus an exponent. A question for you: how would you treat results like 1/3?

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/e51f1b29/attachment-0001.html>

From wolfgang.maier at biologie.uni-freiburg.de  Mon Mar 10 19:03:12 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Mon, 10 Mar 2014 18:03:12 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and
	float/decimal properties [Was: Re: Python Numbers as Human
	Concept Decimal System]
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
Message-ID: <loom.20140310T185941-251@post.gmane.org>

Guido van Rossum <guido at ...> writes:
 
> 
> 
> I think what you're proposing here is variant of fixed-point numbers. The
representation you seem to be looking for is an arbitrary-precision integer
plus an exponent. A question for you: how would you treat results like 1/3?
> 

No, I don't think this is what I'm proposing. I said digits left of the
decimal point should behave like a Python integer, i.e., have arbitrary
precision, but to the right of it things should look like a float or Decimal
with fixed precision. So 1/3 would just look like float(1/3) for example.

Best,
Wolfgang


From stefan at bytereef.org  Mon Mar 10 19:09:13 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 10 Mar 2014 19:09:13 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
References: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
Message-ID: <20140310180913.GA13362@sleipnir.bytereef.org>

Guido van Rossum <guido at python.org> wrote:
> On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>     On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote:
> 
>         That is why I think we should seriously consider moving to IEEE
>         semantics
>         for a decimal literal.
> 
>     I think that would make a *lot* of sense.
> 
> What's the proposal? Just using decimal64 for decimal literals, or introducing
> decimal64 as a new builtin type? (I could get behind either one.)

IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows
multiple discrete contexts: Decimal32, Decimal64, Decimal128, ...

In theory this goes on ad infinitum, but the precision increases much
slower than the exponents.  Mike Cowlishaw's spec fills in the gaps by
allowing arbitrary contexts.  Additionally there are some minor differences,
but if we make moderate changes to Decimal, we get strict IEEE behavior.


So my specific proposal was:

  1) Make those changes to Decimal (we can call the module decimal2
     if backwards compatibility rules it out). The most important
     change relevant to this subthread is rounding all *literals* on
     input while preserving exact construction via Decimal(value).

  2) Keep the arbitrary context facility.

  3) Set IEEEContext(Decimal64) as the default. Users will most
     likely primarily use Decimal32, Decimal64 and Decimal128.
     (Many users will likely never change the context at all.)

  4) Optional: Also use the function names from IEEE 754-2008.


With these changes most users would /think/ that Decimal is a
fixed width Decimal64 type.  Advanced users can still change the
context.  I know for a fact that some users really like the
option of increasing the precision temporarily.


I have to think about the other solution (decimal64 only). At first
glance it seems too restrictive, since I imagine users would at least
want Decimal128, too. Additionally there is no speed benefit.


Stefan Krah



From guido at python.org  Mon Mar 10 19:09:46 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Mar 2014 11:09:46 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140310T185941-251@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
Message-ID: <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>

On Mon, Mar 10, 2014 at 11:03 AM, Wolfgang Maier <
wolfgang.maier at biologie.uni-freiburg.de> wrote:

> Guido van Rossum <guido at ...> writes:
>
> >
> >
> > I think what you're proposing here is variant of fixed-point numbers. The
> representation you seem to be looking for is an arbitrary-precision integer
> plus an exponent. A question for you: how would you treat results like 1/3?
> >
>
> No, I don't think this is what I'm proposing. I said digits left of the
> decimal point should behave like a Python integer, i.e., have arbitrary
> precision, but to the right of it things should look like a float or
> Decimal
> with fixed precision. So 1/3 would just look like float(1/3) for example.
>

I'm sorry, I still don't understand your proposal. Suppose I have two
numbers following that description and I divide them, and suppose the
result cannot be represented as a decimal or binary fraction. What is your
algorithm for deciding how many digits after the decimal point to keep? (Or
how many bits after the binary point -- are you proposing a binary or a
decimal representation?) Is the number of bits/digits to keep a constant,
or does it vary by how many digits/bits are to the left of the point? And
what about multiplications? Should they always produce an exact result or
should they truncate if the result requires more digits/bits behind the
point than a certain number? (If the latter, how is that limit determined?)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/316439d2/attachment.html>

From tim.peters at gmail.com  Mon Mar 10 20:08:49 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 10 Mar 2014 14:08:49 -0500
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <20140310135824.GB10968@sleipnir.bytereef.org>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <20140310135824.GB10968@sleipnir.bytereef.org>
Message-ID: <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com>

[Mark Dickinson <dickinsm at gmail.com>]
>> >>> -Decimal(0.0)  # negative sign mysteriously disappears.
>>
>> Decimal('0')

[Stefan Krah <stefan at bytereef.org>]
> Unless the rounding mode is ROUND_FLOOR. :(
>
> >>> getcontext().rounding = ROUND_FLOOR
> >>> -Decimal(0.0)
> Decimal('-0')

I may be missing something, but our docs don't appear to be clear
about what unary minus means in Python's decimal implementation.

Looks like Mike C's intent was that

    +x

act like

    Decimal(0) + x

and

    -x

like

    Decimal(0) - x

Then all the examples "make sense" (given that you think the
standards' rules for adding/subtracting signed zeros make sense).

Can't win!  That -x loses the sign when x is a positive 0 (under most
rounding modes) is jarring; but that the results of "-x" and "0-x" may
not be the same would also be jarring.

Not a fan of signed zeroes :-(

From dickinsm at gmail.com  Mon Mar 10 20:30:49 2014
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Mar 2014 19:30:49 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <20140310135824.GB10968@sleipnir.bytereef.org>
 <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com>
Message-ID: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com>

On Mon, Mar 10, 2014 at 7:08 PM, Tim Peters <tim.peters at gmail.com> wrote:

> Looks like Mike C's intent was that
>
>     +x
>
> act like
>
>     Decimal(0) + x
> [...]


Yes, I'm pretty sure that's what he intended.  It's clearly spelled out in
the standard: "the operations plus(a) and minus(a) (where a and b refer to
any numbers) are calculated as the operations add(?0?, a) and subtract(?0?,
b)respectively", but not in our docs.

Can't win!  That -x loses the sign when x is a positive 0 (under most
> rounding modes) is jarring; but that the results of "-x" and "0-x" may
> not be the same would also be jarring.
>

Hmm. I find the second option a lot less jarring than the first, but maybe
that's because I'm used to it (and because I'm too young to really remember
non-IEEE floating-point formats; the only one I really encountered was the
Sinclair ZX81's 40-bit format).

Not a fan of signed zeroes :-(
>

Yeah, me neither, but with IEEE 754 world domination it looks like they're
here to stay.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/26f2442a/attachment-0001.html>

From oscar.j.benjamin at gmail.com  Mon Mar 10 21:51:20 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 20:51:20 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140310180913.GA13362@sleipnir.bytereef.org>
References: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <20140310180913.GA13362@sleipnir.bytereef.org>
Message-ID: <CAHVvXxQwqpgmdVeDh0PFVrxNg5Q3FmGExmNQGZgSSJuWOQGB6Q@mail.gmail.com>

On 10 March 2014 18:09, Stefan Krah <stefan at bytereef.org> wrote:
> Guido van Rossum <guido at python.org> wrote:
>> On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>     On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote:
>>
>>         That is why I think we should seriously consider moving to IEEE
>>         semantics
>>         for a decimal literal.
>>
>>     I think that would make a *lot* of sense.
>>
>> What's the proposal? Just using decimal64 for decimal literals, or introducing
>> decimal64 as a new builtin type? (I could get behind either one.)
>
> IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows
> multiple discrete contexts: Decimal32, Decimal64, Decimal128, ...
>
> In theory this goes on ad infinitum, but the precision increases much
> slower than the exponents.  Mike Cowlishaw's spec fills in the gaps by
> allowing arbitrary contexts.  Additionally there are some minor differences,
> but if we make moderate changes to Decimal, we get strict IEEE behavior.
>
>
> So my specific proposal was:
>
>   1) Make those changes to Decimal (we can call the module decimal2
>      if backwards compatibility rules it out). The most important
>      change relevant to this subthread is rounding all *literals* on
>      input while preserving exact construction via Decimal(value).
>
>   2) Keep the arbitrary context facility.
>
>   3) Set IEEEContext(Decimal64) as the default. Users will most
>      likely primarily use Decimal32, Decimal64 and Decimal128.
>      (Many users will likely never change the context at all.)
>
>   4) Optional: Also use the function names from IEEE 754-2008.

I generally agree with the above except that...

> With these changes most users would /think/ that Decimal is a
> fixed width Decimal64 type.  Advanced users can still change the
> context.

This is the problem I have with this particular proposal. Users would
think that it's a fixed-width type and then write code that naively
makes that assumption. Then the code blows up when someone else
changes the arithmetic context. I don't think we should encourage
non-expert users to think that they can safely rely on this behaviour
without actually making it safe to rely on.

>  I know for a fact that some users really like the
> option of increasing the precision temporarily.

Agreed. I do this often. But I think it's the kind of thing that
happens in a special library like Mark's pcdeclib rather than
application code for a casual user. The option will always be there to
promote your decimal64s to the Big Decimal type and do all your
calculations in whichever precision you want.

> I have to think about the other solution (decimal64 only). At first
> glance it seems too restrictive, since I imagine users would at least
> want Decimal128, too. Additionally there is no speed benefit.

Decimal128 seems fine to me. I just think it should be a true
fixed-width type. The benefits of this are:
1) Naive code doesn't get broken by different contexts.
2) I can tell by looking at a snippet exactly what it does without
needing to wonder (or ask) whether or not the context has been fiddled
with.
3) I can show code that uses decimal literals and the decimal128
constructor and guarantee that it works without caveats.
4) Since the meaning of any expression is known at compile time it is
amenable to constant folding.
5) Unary + is a no-op and - is exact (as users would expect) so
negative literals will have the same meaning regardless of the current
context.


Oscar

From oscar.j.benjamin at gmail.com  Mon Mar 10 21:58:18 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 10 Mar 2014 20:58:18 +0000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <20140310135824.GB10968@sleipnir.bytereef.org>
 <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com>
 <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com>
Message-ID: <CAHVvXxT7ocoA2+q3d_j1YQN4zUXW_HRkLozELgbXjNz0nYcXzQ@mail.gmail.com>

On 10 March 2014 19:30, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Mon, Mar 10, 2014 at 7:08 PM, Tim Peters <tim.peters at gmail.com> wrote:
>
>> Can't win!  That -x loses the sign when x is a positive 0 (under most
>> rounding modes) is jarring; but that the results of "-x" and "0-x" may
>> not be the same would also be jarring.
>
> Hmm. I find the second option a lot less jarring than the first,

I agree. Needing to call copy_negate just to flip the sign when unary
- is exact for every other numeric seems very unfortunate to me.


Oscar

From ethan at stoneleaf.us  Mon Mar 10 21:57:09 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 10 Mar 2014 13:57:09 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
Message-ID: <531E2725.5090907@stoneleaf.us>

On 03/10/2014 11:09 AM, Guido van Rossum wrote:
>
> I'm sorry, I still don't understand your proposal.

I think he's saying make a new type similarly to complex, only instead of two floats to make a complex number, have a 
(long) int and a decimal float to make this new type.  The long int portion would have infinite precision, the float 
portion would have, say, 16 digits (or whatever).

--
~Ethan~

From rosuav at gmail.com  Mon Mar 10 22:51:00 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 11 Mar 2014 08:51:00 +1100
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <531E2725.5090907@stoneleaf.us>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
Message-ID: <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>

On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> I think he's saying make a new type similarly to complex, only instead of
> two floats to make a complex number, have a (long) int and a decimal float
> to make this new type.  The long int portion would have infinite precision,
> the float portion would have, say, 16 digits (or whatever).

That's plausible as a representation, and looks tempting, but basic
arithmetic operations become more complicated. Addition and
subtraction just need to worry about carries, but multiplication forks
out into four multiplications (int*int, int*frac, frac*int,
frac*frac), and division becomes similarly complicated. Would it
really be beneficial?

ChrisA

From tim.peters at gmail.com  Mon Mar 10 22:54:05 2014
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 10 Mar 2014 16:54:05 -0500
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <20140310135824.GB10968@sleipnir.bytereef.org>
 <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com>
 <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com>
Message-ID: <CAExdVN=XWsHUg1xjyWxjsV5XgeDWFGaXZgwH9v4BG9LQq7tY3g@mail.gmail.com>

[Mark Dickinson]
> Yes, I'm pretty sure that's what he intended.  It's clearly spelled out in
> the standard: "the operations plus(a) and minus(a) (where a and b refer to
> any numbers) are calculated as the operations add('0', a) and subtract('0',
> b)respectively",

Ah.  The great thing about standards is that there are so many of them
:-(  I was staring at 754-2008, and found nary a mention of "plus" or
"minus" operations.  But I found the quoted text at:

    http://speleotrove.com/decimal/daops.html#refplusmin

where it also says they "correspond to the prefix minus and plus
operators in programming languages".  That doesn't leave much wiggle
room for a conforming implementation.

Next ;-)

From guido at python.org  Mon Mar 10 22:57:48 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Mar 2014 14:57:48 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
 <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
Message-ID: <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com>

On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> > I think he's saying make a new type similarly to complex, only instead of
> > two floats to make a complex number, have a (long) int and a decimal
> float
> > to make this new type.  The long int portion would have infinite
> precision,
> > the float portion would have, say, 16 digits (or whatever).
>
> That's plausible as a representation, and looks tempting, but basic
> arithmetic operations become more complicated. Addition and
> subtraction just need to worry about carries, but multiplication forks
> out into four multiplications (int*int, int*frac, frac*int,
> frac*frac), and division becomes similarly complicated. Would it
> really be beneficial?
>

It looks neither plausible nor tempting to me at all, and I hope that's not
what he meant. It can represent numbers of any magnitude that have lots of
zeros following the decimal point followed by up to 16 digits of precision,
but not numbers that have e.g. lots of ones instead of those zeros -- the
float portion would be used up for the first 16 ones. E.g.

111111111111111111111111111111.000000000000000000000000000000123456789

would be representable exactly but not

111111111111111111111111111111.111111111111111111111111111111123456789

What makes numbers in the vicinity of integers special?

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/2487bed8/attachment-0001.html>

From rosuav at gmail.com  Mon Mar 10 23:29:26 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 11 Mar 2014 09:29:26 +1100
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
 <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
 <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com>
Message-ID: <CAPTjJmp08WNDiF=7w9ktxULr6gi0V6OKExmNpsHDDbAJO6LMDQ@mail.gmail.com>

On Tue, Mar 11, 2014 at 8:57 AM, Guido van Rossum <guido at python.org> wrote:
> It looks neither plausible nor tempting to me at all, and I hope that's not
> what he meant. It can represent numbers of any magnitude that have lots of
> zeros following the decimal point followed by up to 16 digits of precision,
> but not numbers that have e.g. lots of ones instead of those zeros -- the
> float portion would be used up for the first 16 ones. E.g.
>
> 111111111111111111111111111111.000000000000000000000000000000123456789
>
> would be representable exactly but not
>
> 111111111111111111111111111111.111111111111111111111111111111123456789
>
> What makes numbers in the vicinity of integers special?

Hmm, good point. I was thinking this would give a predictable 16
digits of precision after the decimal, but the leading zeroes are
somewhat special. But when I said "tempting" I meant that it looks
initially nice, and then went on to show that it's not so nice on
analysis - which latter part you're also demonstrating.

ChrisA

From wolfgang.maier at biologie.uni-freiburg.de  Mon Mar 10 23:36:59 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Mon, 10 Mar 2014 22:36:59 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and
	float/decimal properties [Was: Re: Python Numbers as Human
	Concept Decimal System]
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
 <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
 <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com>
Message-ID: <loom.20140310T233009-70@post.gmane.org>

Guido van Rossum <guido at ...> writes:

> 
> 
> 
> On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico
<rosuav at gmail.com> wrote:
> On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman
<ethan at stoneleaf.us> wrote:
> 
> 
> > I think he's saying make a new type similarly to complex, only instead of
> > two floats to make a complex number, have a (long) int and a decimal float
> > to make this new type. ?The long int portion would have infinite precision,
> > the float portion would have, say, 16 digits (or whatever).
> That's plausible as a representation, and looks tempting, but basic
> arithmetic operations become more complicated. Addition and
> subtraction just need to worry about carries, but multiplication forks
> out into four multiplications (int*int, int*frac, frac*int,
> frac*frac), and division becomes similarly complicated. Would it
> really be beneficial?
> 
> 
> It looks neither plausible nor tempting to me at all, and I hope that's
not what he meant. It can represent numbers of any magnitude that have lots
of zeros following the decimal point followed by up to 16 digits of
precision, but not numbers that have e.g. lots of ones instead of those
zeros -- the float portion would be used up for the first 16 ones.
E.g.111111111111111111111111111111.000000000000000000000000000000123456789
> 
> would be representable exactly but
not111111111111111111111111111111.111111111111111111111111111111123456789
> 
> 
> 
> 
> 
> What makes numbers in the vicinity of integers special?
> 


I'm afraid it is exactly what I'm proposing. I don't see though how this is
different from current behavior of lets say Decimal.
Assuming default context with prec=28 you currently get:

>>> +Decimal('0.000000000000000000000000000000123456789')
Decimal('1.23456789E-31')

, but with a "consumer 1" (one is enough, actually):

>>> +Decimal('0.100000000000000000000000000000123456789')
Decimal('0.1000000000000000000000000000')

Cheers,
Wolfgang


From wolfgang.maier at biologie.uni-freiburg.de  Mon Mar 10 23:42:38 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Mon, 10 Mar 2014 22:42:38 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and
	float/decimal properties [Was: Re: Python Numbers as Human
	Concept Decimal System]
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
 <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
Message-ID: <loom.20140310T233952-563@post.gmane.org>

Chris Angelico <rosuav at ...> writes:

> 
> On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at ...> wrote:
> > I think he's saying make a new type similarly to complex, only instead of
> > two floats to make a complex number, have a (long) int and a decimal float
> > to make this new type.  The long int portion would have infinite precision,
> > the float portion would have, say, 16 digits (or whatever).
> 
> That's plausible as a representation, and looks tempting, but basic
> arithmetic operations become more complicated. Addition and
> subtraction just need to worry about carries, but multiplication forks
> out into four multiplications (int*int, int*frac, frac*int,
> frac*frac), and division becomes similarly complicated. Would it
> really be beneficial?
> 

That's right, it would complicate calculations exactly as you are pointing
out. That's why I said it might be too slow for most users even when
implemented optimally, but then I'm not sure whether performance should be
the very first thing to discuss.

Best,
Wolfgang





From guido at python.org  Mon Mar 10 23:47:03 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Mar 2014 15:47:03 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140310T233009-70@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
 <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com>
 <loom.20140310T185941-251@post.gmane.org>
 <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com>
 <531E2725.5090907@stoneleaf.us>
 <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com>
 <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com>
 <loom.20140310T233009-70@post.gmane.org>
Message-ID: <CAP7+vJJtpcocJwDNE=h_HNYk23UwSLLFhWY3DcQDYRJ+L93FkQ@mail.gmail.com>

On Mon, Mar 10, 2014 at 3:36 PM, Wolfgang Maier <
wolfgang.maier at biologie.uni-freiburg.de> wrote:

> Guido van Rossum <guido at ...> writes:
>
> >
> >
> >
> > On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico
> <rosuav at gmail.com> wrote:
> > On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman
> <ethan at stoneleaf.us> wrote:
> >
> >
> > > I think he's saying make a new type similarly to complex, only instead
> of
> > > two floats to make a complex number, have a (long) int and a decimal
> float
> > > to make this new type.  The long int portion would have infinite
> precision,
> > > the float portion would have, say, 16 digits (or whatever).
> > That's plausible as a representation, and looks tempting, but basic
> > arithmetic operations become more complicated. Addition and
> > subtraction just need to worry about carries, but multiplication forks
> > out into four multiplications (int*int, int*frac, frac*int,
> > frac*frac), and division becomes similarly complicated. Would it
> > really be beneficial?
> >
> >
> > It looks neither plausible nor tempting to me at all, and I hope that's
> not what he meant. It can represent numbers of any magnitude that have lots
> of zeros following the decimal point followed by up to 16 digits of
> precision, but not numbers that have e.g. lots of ones instead of those
> zeros -- the float portion would be used up for the first 16 ones.
> E.g.111111111111111111111111111111.000000000000000000000000000000123456789
> >
> > would be representable exactly but
> not111111111111111111111111111111.111111111111111111111111111111123456789
> >
> >
> >
> >
> >
> > What makes numbers in the vicinity of integers special?
> >
>
>
> I'm afraid it is exactly what I'm proposing. I don't see though how this is
> different from current behavior of lets say Decimal.
> Assuming default context with prec=28 you currently get:
>
> >>> +Decimal('0.000000000000000000000000000000123456789')
> Decimal('1.23456789E-31')
>
> , but with a "consumer 1" (one is enough, actually):
>
> >>> +Decimal('0.100000000000000000000000000000123456789')
> Decimal('0.1000000000000000000000000000')
>

It is very different. Decimal with prec=28 counts the number of digits from
the first non-zero digit, and the number of digits it gives you after the
decimal point depends on how many digits there are before it. That is a
sane perspective on precision (the total number of significant digits).

But in your proposal the number of digits you get after the point depends
on how close the value is to the nearest integer (in the direction of
zero), not how many significant digits you have in total. That's why my
examples started with lots of ones, not zeros.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/123c15de/attachment.html>

From ncoghlan at gmail.com  Tue Mar 11 00:02:31 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Mar 2014 09:02:31 +1000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
Message-ID: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>

On 11 Mar 2014 01:02, "Guido van Rossum" <guido at python.org> wrote:
>
> On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com>
wrote:
>>
>> On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote:
>>
>>> That is why I think we should seriously consider moving to IEEE
semantics
>>> for a decimal literal.
>>
>>
>> I think that would make a *lot* of sense.
>
>
> What's the proposal? Just using decimal64 for decimal literals, or
introducing decimal64 as a new builtin type? (I could get behind either
one.)

My take is that we're down to two main options:

Stefan: use the existing decimal type, change default context to Decimal64,
round all decimal literals to current context

Oscar: new fixed width decimal64 builtin type, always uses Decimal64
context (allowing constant folding), interoperates with variable context
decimal.Decimal values (producing a decimal.Decimal result)

I lean towards Oscar's proposal, as it removes the hidden context dependent
behaviour and makes the builtin decimals true constant values.

Perhaps Stefan & Oscar would be willing to collaborate on a PEP that sums
up the pros and cons of the two main alternatives (as well as some of the
other variants that were discussed and rejected)?

(Adding the IEEE contexts to the decimal module sounds like a good idea
regardless)

Cheers,
Nick.

>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/1d015b3d/attachment-0001.html>

From ncoghlan at gmail.com  Tue Mar 11 00:06:16 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Mar 2014 09:06:16 +1000
Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__
 [Was: Re: Python Numbers as Human Concept Decimal System]
In-Reply-To: <20140310151923.GD12595@ando>
References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com>
 <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com>
 <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com>
 <20140310151923.GD12595@ando>
Message-ID: <CADiSq7cAHtQ3SXGbSjAOrYRxzDv4h82qRShc4MaggFu3nfgrLw@mail.gmail.com>

On 11 Mar 2014 01:20, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Mon, Mar 10, 2014 at 09:55:09AM -0400, random832 at fastmail.us wrote:
> > On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote:
> > > I guess I have ulterior motives; (ab)using '+' to mean 'round to the
> > > current context' has always felt way too implicit for my tastes, and
I'd
> > > jump at the chance to see it go away.
> >
> > It occurs to me that there is an unprecedented opportunity to make warts
> > "go away" - making Decimal a "first-class citizen" suggests moving it
> > from the decimal module to a built-in type. Any backwards-incompatible
> > change you want to make, you can give programs importing decimal the old
> > behavior, and reserve the new behavior for the new type in builtins.
>
> Historically, I don't think Python has changed the API for types when
> they've moved from a module to a built-in. I can't think of any example
> except set, but apart from a change in name from set.Set to set, I don't
> think there were any changes.

I believe there were a few changes in how it interoperated with other
types, based on lessons learned from the module version (although I forget
the details).

Cheers,
Nick.

>
> Any other precedents?
>
> On the other hand, the built-in doesn't need to be a replacement for the
> decimal module, but rather an alternative solution. If decimal64 was a
> built-in, presumably it wouldn't need contexts and would be a lot
> simpler. The decimal module would still be available for those with
> advanced needs.
>
>
> --
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/62b899d9/attachment.html>

From steve at pearwood.info  Tue Mar 11 02:17:35 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 11 Mar 2014 12:17:35 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com>
References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com>
 <20140310153127.GF12595@ando>
 <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com>
Message-ID: <20140311011735.GJ12595@ando>

On Mon, Mar 10, 2014 at 04:00:38PM +0000, Oscar Benjamin wrote:
> On 10 March 2014 15:31, Steven D'Aprano <steve at pearwood.info> wrote:
> > On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:

> >> Where this would get complicated is for people who also use the
> >> Decimal type. They'd need to keep track of which objects were of which
> >> type and so decimal literals might seem more annoying than useful.

> > Hmmm. I don't think this should necessarily be complicated, or at least
> > no more complicated than dealing with any other mixed numeric types. If
> > you don't want to mix them, don't mix them :-)
> 
> Exactly. It just means that you wouldn't want to use the literals in
> code that actually uses the decimal module. Consider:

If you're writing a function that accepts whatever random numeric type 
the caller passes in, you have to expect that you cannot control the 
behaviour precisely. (You can't control rounding or precision for 
floats; Fractions are exact; the proposed decimal64 will have a fixed 
precision and fixed rounding; decimal.Decimal you can control.) You 
might still write type-agnostic code, but the result you get may depend 
on the input data.

 
> if some_condition:
>     x = 1d
> else:
>     x = Decimal(some_string)

Mixing different types within a single calculation/function, as shown 
above, sounds like a recipe for chaos to me. I wouldn't write it like 
that, any more than I would write:

if some_condition:
    x = 1.1
else:
    x = Decimal("2.2")


> # ...
> y = x / 3
> 
> So now x / 3 rounds differently depending on whether x is a decimal64
> or a Decimal. I probably don't want that.

Then don't do it! :-)


> The solution: coerce to Decimal.

Or write both branches as a Decimal in the first place.


> But then why did I bother with the Decimal literal anyway?
> Decimal(1d) is hardly better than Decimal('1').

Presumably you didn't use a decimal64 literal if you cared about 
controlling the rounding or precision, because that would be silly. (You 
might, on the other hand, use decimal64 literals everywhere in your 
function, if its rounding and precision was sufficient for your needs.) 
However, the caller of your function may pass you a decimal64 instance. 
I don't think there's much advantage to passing integer values with the 
d suffix, but passing fractional values is a different story:

some_function(1.1)

versus

some_function(1.1d)


I think this is a proposal I could get behind: leave the decimal module 
(mostly) as-is, perhaps with a few backwards-compatible tweaks, while 
adding an independent decimal64 or decimal128 built-in type and literal 
with fixed rounding and precision. The new literal type would satisfy 
the use-case Mark Harris is worried about, apart from having to add a 
"d" suffix to literals[1], while the decimal module still allows for 
fine control of the context.

The built-in decimal type should convert floats using their repr, which 
ought to make Guido happy too :-)





[1] Shifting to decimal floats by default is a major 
backwards-incompatible change.


-- 
Steven

From cfkaran2 at gmail.com  Tue Mar 11 02:45:17 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Mon, 10 Mar 2014 21:45:17 -0400
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <531BB061.2060009@canterbury.ac.nz>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
 <531BB061.2060009@canterbury.ac.nz>
Message-ID: <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com>

Is there any further interest in standardized annotations, or should the idea be abandoned?

Thanks,
Cem Karan

On Mar 8, 2014, at 7:05 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Cem Karan wrote:
>> There are a number of reasons I'm suggesting UUIDs instead of simple strings:
> 
> I'm not talking about strings, I'm talking about
> objects created and exported by the module defining
> the annotations, and compared by identity.
> 
> The Python module namespace then ensures they have
> unique names within any given program. That's all
> you need, because there's no requirement to persist
> them from one program execution to another.
> 
> -- 
> Greg
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From tjreedy at udel.edu  Tue Mar 11 07:15:23 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 11 Mar 2014 02:15:23 -0400
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
Message-ID: <lfm9lu$7s3$1@ger.gmane.org>

On 3/10/2014 7:02 PM, Nick Coghlan wrote:

> My take is that we're down to two main options:
>
> Stefan: use the existing decimal type, change default context to
> Decimal64, round all decimal literals to current context
>
> Oscar: new fixed width decimal64 builtin type, always uses Decimal64
> context (allowing constant folding), interoperates with variable context
> decimal.Decimal values (producing a decimal.Decimal result)
>
> I lean towards Oscar's proposal, as it removes the hidden context
> dependent behaviour and makes the builtin decimals true constant values.

Ditto for me. An everyday replacement for binary floats should be about 
as simple as binary floats, and simpler in the sense of having less 
surprising behavior.

-- 
Terry Jan Reedy


From ron3200 at gmail.com  Tue Mar 11 07:36:38 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Tue, 11 Mar 2014 01:36:38 -0500
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140310T181321-988@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
Message-ID: <lfmat9$j4s$1@ger.gmane.org>



On 03/10/2014 12:48 PM, Wolfgang Maier wrote:
> Going back to Mark Harris' initial proposal of unifying numeric types (which
> I am not trying to support in any way here), such a type would even allow to
> unify int and float since an ints could be considered a subset of the new
> type with a fractional part of zero.


This turned out a bit longer than I intended.  I think it is the correct 
approach to what I think is the intent of the number unifications 
suggestions.  Trim it down a bit, and add some examples and it's a good 
start to a PEP.   ;-)


I think adding a new number type isn't going to help that much, although it 
won't hurt either.

(Hoping this is a more practical direction for these discussions.)



General Proposal:
=================

Add a BaseValue class for creating context dependent values like a 
CurrencyValue class, or StatisticValue class.

The Value class should not specify any particular number type.  That is an 
implementation detail of a subclass.

The Value class will define how to round, compare, and track error ranges 
when used in calculations.  And will use ValueContext objects to adjust 
that as needed.

Add a ValueContext class to create context parameter objects.  These 
objects will be used to set a Value class's context for rounding, number of 
significant digits, and tracking error ranges.

The ValueContext objects should be very simple, and loosely defined in the 
same way slice objects are.  They are only used for passing the context 
parameters.  (Like slice objects pass index's, but probably by called 
methods rather than by syntax.)



Introduction:
=============

There have been a number of suggestions to unify numbers in order to make 
them easier to use for less mathematically inclined users to do more 
complex calculations without having to know quite as much about the 
underlying type of number.  There is no magic wand that will make it 
completely painless, but we can make things easier to get right to an 
average programmer with average to good math skills.  (Probably the normal 
group for most beginner programmer students.)

The basic issue I see isn't with the number types, but with how we use 
them.  (And how we can us them better)  There are a lot of parts that need 
to work both independently and together at the same time.

Currently It's up to the programmer to keep track of significant digits 
(and significant error), any rounding that may need to be done, and to do 
it all correctly.  There are many places where it can go wrong, and where 
getting it right requires a good understanding of these concepts within the 
speciality they are being used in.

The Decimal module tries to reduce that work, and also reduce errors, and 
increase overall accuracy, all at the same time.   But that doesn't help 
the existing number types.  Also while just using the decimal type isn't 
that difficult, using the other features it has correctly isn't as easy.

It's not easy to mix and match these ideas and come up with a usable, and 
reusable, interface for doing many similar types of calculations correctly 
as each value in a calculation (or function call), may not need the same 
exact requirements.

Possibly the broadest, but still practical viewpoint, gives three kinds of 
numbers.  A number kind as described here, refers to how they are used, and 
is independent of the way it's stored in the computer.  The number kind is 
what determines some of the requirements of a calculation or comparison.

If we consider that how a number is used as independent (as much as is 
reasonable) from how the number is stored in memory, then maybe the type is 
less important than the context.



Kinds of Values:
================

1. Measurements: Values which have limited accuracy and need both proper 
rounding and tracking of error ranges.

2. Exact numbers:  Values that really do represent exactly what they are. 
Usually smaller sized numbers, and usually not fractional amounts. 
(includes counters and index's for example)

3. Ideal numbers: Values that are not measurements but can be calculated as 
accurately as needed and generally don't require keeping track of error 
because the significant digits can far exceed any measurements.
    - (Unless you are a theoretical physicist.)
    - (Low quality approximations should be considered a measurement,
       even if it could be calculated accurately to any nth digit.)

Ints, and floating point actually do a very nice job of representing 2. and 
3 in most cases.  And they generally don't need to have a contexts 
associated to them, and usually don't need rounding.  For example, we don't 
want to round index's and counters, they are in the exact group.  And we 
can calculate Pi to what ever we need so that any error range in the value 
is less than significant.


Equality Tests:
===============

Another issue is with equality and how a value's exactness effects that. 
For example if we have two values with an error of +- 1.  We can say it has 
a width of 3.  (Or inexactness of 3)

So we might have this case where two values overlap when you consider the 
width of the value.  (it's +- amount)

      (23 +- 1) < (24 +- 1)  -->  False

Of course those would be objects instead of literals.  (weather or not they 
are decimal, int, or float, isn't important.)

This can be extended to all the other comparison operators.  The usual way 
of dealing with it is to do an error range test, which adds a lot of noise 
to an otherwise simple operation.  Especially if you need to put those in 
many different places.  They aren't quite complex enough to justify 
functions, but complex enough to be bothersome and can be a common source 
of errors.

Exactness, (and rounding, as it's directly related to exactness), are 
independent concepts from how the number is stored in a computer.  (Unless 
you use the wrong type of course, but then I would consider it a bug.)



Values vs Numbers:
=================

One approach is to have a Value class, and a ValueContext object.  The 
ValueContext object may be more like a slice object. It would be up to the 
Value class to interpret it so that it can know how to do the operator 
methods correctly...  __round__, __add__, __eq__, etc...

A base Value class could be used to define types such as a Measurement 
class, Or a Currency class, (and others).

The context API needs to be on a separate Value class/type rather than on 
the number in my opinion.  I think trying to make numbers type also be a 
value is what gets us mixed up. The number is just a scaler for the value. 
  It might also be called vector, but that is a technical term that can be 
confused with other concepts.



Benefits of a Value class:
==========================

* Doesn't change any existing number type.

* Value objects work with any number types.

* Value objects can have a repr that is much nicer, and more meaningful 
than the underlying number type.

* Context sub-class's define how they are used (or act) rather than by what 
kind of computer number they may contain.

* Context (slice like objects) that can be shared, and/or passed between 
Value objects as needed to keep track of rounding, significant digits, and 
error range.

* Can be a library module.


If Value objects is something that can be included into python, then a 
simpler decimal64 type without all the context settings could be used as 
it's just another computer number type with a bit more accuracy if you need it.

A Currency Value type could use Decimal, and the type can be swapped out 
later for the decimal64, as that becomes just an implementation detail for 
he Currency Value class.

Users could do something like that now by defining classes, and probably 
have done it within their applications.  I consider that is a supporting 
argument for this approach, rather than an argument against it.  It still 
takes a fair amount of math knowledge how to get it right.

Making context dependent calculations easier to do, and use, would be very 
nice, and modules using these Value objects could possibly work together 
better.



A good test might be how "Value" objects could be used in the statistics 
module.  Or maybe Mark Harris could try out this idea in his new module.

I think this is the sort of thing Mark was initially looking for.  It's 
really not that complicated if the order of dependencies is correctly 
ordered.  (probably he's got something close to this in it already, just 
with a different name.  (Just a guess, but any solution that solves some of 
this may not be that far off.)



BTW, The new literals would be helpful with this too. +1

Cheers,
    Ron












From solipsis at pitrou.net  Tue Mar 11 08:24:49 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Mar 2014 08:24:49 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
Message-ID: <20140311082449.4f0c0fdb@fsol>

On Tue, 11 Mar 2014 09:02:31 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> My take is that we're down to two main options:
> 
> Stefan: use the existing decimal type, change default context to Decimal64,
> round all decimal literals to current context
> 
> Oscar: new fixed width decimal64 builtin type, always uses Decimal64
> context (allowing constant folding), interoperates with variable context
> decimal.Decimal values (producing a decimal.Decimal result)
> 
> I lean towards Oscar's proposal, as it removes the hidden context dependent
> behaviour and makes the builtin decimals true constant values.

Yuck. Is this some kind of joke? We've gone through the trouble of
unifying long and int in Python 3 and now people are proposing two
different Decimal types, one with fixed precision and one with
arbitrary precision?

I'm completely -1 on this.

Regards

Antoine.



From stephen at xemacs.org  Tue Mar 11 10:28:32 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 11 Mar 2014 18:28:32 +0900
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
 <531BB061.2060009@canterbury.ac.nz>
 <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com>
Message-ID: <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp>

Cem Karan writes:

 > Is there any further interest in standardized annotations, or
 > should the idea be abandoned?

Obviously there's interest; standards are a good thing when you're
trying to share.  But not if they end up getting in the way of sharing
because they're too limited or you end up with a bunch of standards
such that no program can conform with all of them.

To avoid the latter, you need to provide an implementation and show
that it's useful by waiting for it to be used.  You're not going to
get a standard in to the stdlib at this point because there's not
enough usage of *any* proposed annotation standard.

If you want to make progress on this, just do it, and worry about
getting it in to the stdlib later.

To see what it takes to go directly into the stdlib, consider the PEP
461 debate.  There was no need to provide an implementation and wait
for usage to follow *because %-formatting for binary was already in
widespread practical use in Python 2*.  It was pretty clear that the
default was going to be "just like Python 2", and that's how it ended
up -- with the exception of "%r" because that would do the wrong thing
in the intended use case (and "%a" does an equivalent right thing).

From cfkaran2 at gmail.com  Tue Mar 11 10:45:33 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Tue, 11 Mar 2014 05:45:33 -0400
Subject: [Python-ideas] Suggestion for standardized annotations
In-Reply-To: <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com>
 <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com>
 <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com>
 <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com>
 <5317FDB6.5000806@canterbury.ac.nz>
 <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com>
 <531BB061.2060009@canterbury.ac.nz>
 <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com>
 <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <3C102356-A26B-46FF-8FDB-9FE00AFD6260@gmail.com>

Oh, I didn't think it would get into the standard library in one shot, that's for sure!  I just wanted to gauge interest to see if I should continue working on it and promoting it.  I'll go ahead and do so, and put it up on pypi.

Thanks,
Cem Karan

On Mar 11, 2014, at 5:28 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Cem Karan writes:
> 
>> Is there any further interest in standardized annotations, or
>> should the idea be abandoned?
> 
> Obviously there's interest; standards are a good thing when you're
> trying to share.  But not if they end up getting in the way of sharing
> because they're too limited or you end up with a bunch of standards
> such that no program can conform with all of them.
> 
> To avoid the latter, you need to provide an implementation and show
> that it's useful by waiting for it to be used.  You're not going to
> get a standard in to the stdlib at this point because there's not
> enough usage of *any* proposed annotation standard.
> 
> If you want to make progress on this, just do it, and worry about
> getting it in to the stdlib later.
> 
> To see what it takes to go directly into the stdlib, consider the PEP
> 461 debate.  There was no need to provide an implementation and wait
> for usage to follow *because %-formatting for binary was already in
> widespread practical use in Python 2*.  It was pretty clear that the
> default was going to be "just like Python 2", and that's how it ended
> up -- with the exception of "%r" because that would do the wrong thing
> in the intended use case (and "%a" does an equivalent right thing).


From oscar.j.benjamin at gmail.com  Tue Mar 11 11:51:28 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 10:51:28 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140311082449.4f0c0fdb@fsol>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
Message-ID: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>

On 11 March 2014 07:24, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 11 Mar 2014 09:02:31 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> My take is that we're down to two main options:
>>
>> Stefan: use the existing decimal type, change default context to Decimal64,
>> round all decimal literals to current context
>>
>> Oscar: new fixed width decimal64 builtin type, always uses Decimal64
>> context (allowing constant folding), interoperates with variable context
>> decimal.Decimal values (producing a decimal.Decimal result)
>>
>> I lean towards Oscar's proposal, as it removes the hidden context dependent
>> behaviour and makes the builtin decimals true constant values.
>
> Yuck. Is this some kind of joke? We've gone through the trouble of
> unifying long and int in Python 3 and now people are proposing two
> different Decimal types, one with fixed precision and one with
> arbitrary precision?
>
> I'm completely -1 on this.

I understand your objection Antoine. There is a big difference though
between integer and non-integer types in this respect though. For
integers is is not hard to define a single integer type that satisfies
the vast majority of use-cases (focusing purely on semantics rather
than detailed implementation aspects and performance). The only
contentious issue is how to handle inexact division. int and long were
not really "unified" in Python 3. The long type *replaced* the int
type as it has those universal integer semantics that are really
wanted.

When it comes to non-integer types there's just no single good way of
doing it. So just in the stdlib we already have float, Fraction and
Decimal. The additional libraries that I use have many more numeric
types than that. The idea here is that for most users decimal128 just
is *the* decimal type. The decimal.Decimal type is IMO overly complex
and should really be for niche use-cases like other non-stdlib
multi-precision numeric types.

Java's BigDecimal is more user friendly in some important ways. It
performs all calculations exactly until you explicitly round. When you
ask it to do something that must be inexact without explicitly
supplying a rounding context it throws an exception. Naturally it is
more verbose than calculations with decimal.Decimal but it is possible
to always know what's going on just by looking at the relevant code.

C# has a (non-standard I think) fixed-width decimal type. Once again
the precision and exponent range of this type are not governed by a
hidden modifiable arithmetic context so it is possible to know what a
snippet of code does before run-time.

C++ is introducing decimal32, decimal64, and decimal128 as fixed-width
decimal floating point types as according to IEEE-754-2008. These will
be fixed-width types whose precision is always known so that a static
analysis can explain what code does (excepting traps and flags - I'm
not clear on how they will work in C++).

The problem as I see it is that if Decimal is brought into the core of
the language in its current format then many people will be confused
by its behaviour. The behaviour of Python's Decimal type is governed
by the hidden global variable that is the arithmetic context so that
even trivial looking code cannot fully be understood without thinking
through all the possible contexts that might be in effect when it is
executed. I've used a few multi-precision libraries and I find Decimal
hard to use because of the action at a distance effect of the contexts
and the fact that it is possible for Decimals to exist whose precision
exceeds the context so that even the expression -x is subject to
rounding.

The idea is here is to make decimal floating point accessible for
non-expert users. I'm concerned that the current Decimal type is
overly complex for this use-case.


Oscar

From ncoghlan at gmail.com  Tue Mar 11 12:46:27 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Mar 2014 21:46:27 +1000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
Message-ID: <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>

On 11 March 2014 20:51, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 11 March 2014 07:24, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Tue, 11 Mar 2014 09:02:31 +1000
>> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> My take is that we're down to two main options:
>>>
>>> Stefan: use the existing decimal type, change default context to Decimal64,
>>> round all decimal literals to current context
>>>
>>> Oscar: new fixed width decimal64 builtin type, always uses Decimal64
>>> context (allowing constant folding), interoperates with variable context
>>> decimal.Decimal values (producing a decimal.Decimal result)
>>>
>>> I lean towards Oscar's proposal, as it removes the hidden context dependent
>>> behaviour and makes the builtin decimals true constant values.
>>
>> Yuck. Is this some kind of joke? We've gone through the trouble of
>> unifying long and int in Python 3 and now people are proposing two
>> different Decimal types, one with fixed precision and one with
>> arbitrary precision?
>>
>> I'm completely -1 on this.
>
> I understand your objection Antoine. There is a big difference though
> between integer and non-integer types in this respect though. For
> integers is is not hard to define a single integer type that satisfies
> the vast majority of use-cases (focusing purely on semantics rather
> than detailed implementation aspects and performance). The only
> contentious issue is how to handle inexact division. int and long were
> not really "unified" in Python 3. The long type *replaced* the int
> type as it has those universal integer semantics that are really
> wanted.
>
> When it comes to non-integer types there's just no single good way of
> doing it. So just in the stdlib we already have float, Fraction and
> Decimal. The additional libraries that I use have many more numeric
> types than that. The idea here is that for most users decimal128 just
> is *the* decimal type. The decimal.Decimal type is IMO overly complex
> and should really be for niche use-cases like other non-stdlib
> multi-precision numeric types.

Right. I believe the appropriate comparison here should be with
builtin floats (which are essentially 64 bit C doubles, with all the
limitations that implies), rather than with the historical int/long
situation.

The reason I see Oscar's proposal as arguably different from the
int/long case is that we *wouldn't* be trying to pretend that builtin
decimals are the same type as the existing variable context decimals.
While they would implicitly interoperate under addition, etc (as
numbers generally do, with variable precision decimals being treated
as the resulting type for mixed arithmetic), there would be no way to
implicitly turn two fixed precision decimals into variable precision
ones.

That is, don't think "int vs long", think "float16 vs float32 vs
float64 vs float128", with Oscar's proposal being that picking just
one decimal floating point precision as *the* floating point
precision, just as Python already does for binary floating point,
makes more sense than trying to provide variable precision support in
a builtin type.

That approach buys us a couple of key benefits:

- they would be true constants, so constant folding, etc, would work
normally, and you could tell from reading the code exactly what it
will do, without needing to worry about the impact of thread local
state
- you could generally use the builtin decimal literals without
learning anything about decimal contexts, because decimal literals
wouldn't be context dependent (although their behaviour would be
formally defined in terms of an IEEE decimal context, for most users
it would just be "this is how Python decimal literals behave")

The guidance to new users would then be *don't* use the decimal
module, use decimal literals instead. If you need more configurable
behaviour, *then* reach for the decimal module. However, the explicit
methods on decimal context objects should probably still be updated to
accept both fixed and variable precision decimals under this model.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From mal at egenix.com  Tue Mar 11 13:33:10 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 11 Mar 2014 13:33:10 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>	<85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>	<CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>	<CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>	<CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>	<CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>	<CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>	<20140310105942.GB12595@ando>	<CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>	<20140310135342.GA10968@sleipnir.bytereef.org>	<CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>	<CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>	<CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>	<20140311082449.4f0c0fdb@fsol>	<CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>
Message-ID: <531F0286.8090306@egenix.com>

On 11.03.2014 12:46, Nick Coghlan wrote:
> ...[Oscar's proposal being that picking just one decimal floating
>     point precision as *the* floating point precision]...
>
> That approach buys us a couple of key benefits:
> 
> - they would be true constants, so constant folding, etc, would work
> normally, and you could tell from reading the code exactly what it
> will do, without needing to worry about the impact of thread local
> state
> - you could generally use the builtin decimal literals without
> learning anything about decimal contexts, because decimal literals
> wouldn't be context dependent (although their behaviour would be
> formally defined in terms of an IEEE decimal context, for most users
> it would just be "this is how Python decimal literals behave")
> 
> The guidance to new users would then be *don't* use the decimal
> module, use decimal literals instead. If you need more configurable
> behaviour, *then* reach for the decimal module. However, the explicit
> methods on decimal context objects should probably still be updated to
> accept both fixed and variable precision decimals under this model.

I think you are leaving out one of the most important use cases
for decimal values: data input and output.

The literals would only appear in programs. However, in most use
cases, you want to read decimal data from some source, process it
and then write it back again. This would most likely also require
using a few decimal literals, but the main source of decimals
would still be constructors that you choose when writing the
tools.

Coming back to floats and the decimal constructor:

In investment banking, for example, people usually work with floats
all the time. You only convert back to decimals at the very end of some
calculation. The reason here being either that the calculations
involve complex operations which are not available for decimals,
trying to keep error intervals small, or a combination of both.
In general, you try to use as much precision as you can afford
(in terms of memory and speed), to keep those error intervals
small.

In accounting, you often use decimals for storing data with
an (usually contractually or law based) agreed upon precision
and rounding logic. However, there situations where you have
to go to floats as well in order to run calculations, e.g.
for interest, taxes, etc.

In both situations, you want to have the decimal constructor
take the float values with full precision and only then apply
the necessary rounding to turn the value into a form which
complies with the agreed upon rules. It's also not uncommon to
add correctional bookings to address rounding issues explicitly
(e.g. when calculating VAT of a large number of individual
items).

In short: you try to prevent rounding from happening as
much as possible and when using it, you use it in a
controlled way.

Based on this, the choice to have the decimal constructor
use full precision when reading floats is a good one, even
though it may not feel right for the novice, the casual
decimal user or as human concept :-)

For decimal literals, I'd argue that if you enter a
value 1.2500d, you are expecting a decimal with 4 decimal
places precision, not 64 or 128 bits :-)

The information about the intended precision is implicit in the
literal. You see this done in exchange rates, stock prices,
prices at your grocery store, etc.

The decimals can then be transformed into ones with higher
precision during calculations and then possibly back to lower
precision, but this is an explicit decision by the system doing
the calculation.

Anyway, just throwing in some additional entropy into this
discussion. Probably not all that helpful, since you're already
converging on two possible solutions :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 11 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-04-09: PyCon 2014, Montreal, Canada ...               29 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From oscar.j.benjamin at gmail.com  Tue Mar 11 13:44:00 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 12:44:00 +0000
Subject: [Python-ideas] %-formatting with Decimals
Message-ID: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>

During recent discussions I came across something unfortunate with
formatting Decimals.

Formatting with .format works as I would want:

>>> from decimal import Decimal as D
>>> '{:1.0e}'.format(D('1.123e+1000'))
'1e+1000'
>>> '{:1.0e}'.format(D('1.123e-1000'))
'1e-1000'
>>> '{:.50f}'.format(D('1.1'))
'1.10000000000000000000000000000000000000000000000000'

But %-formatting coerces to float:

>>> '%1.0e' % D('1.123e-1000')
'0e+00'
>>> '%1.0e' % D('1.123e+1000')
'inf'
>>> '%.50f' % D('1.1')
'1.10000000000000008881784197001252323389053344726562'

Am I doing this wrong? Is this just a limitation of the old-style %
formatting or something that could possibly be improved?


Oscar

From oscar.j.benjamin at gmail.com  Tue Mar 11 14:04:50 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 13:04:50 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <531F0286.8090306@egenix.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>
 <531F0286.8090306@egenix.com>
Message-ID: <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com>

On 11 March 2014 12:33, M.-A. Lemburg <mal at egenix.com> wrote:
> On 11.03.2014 12:46, Nick Coghlan wrote:
>>
>> ...[Oscar's proposal being that picking just one decimal floating
>>     point precision as *the* floating point precision]...
>
> I think you are leaving out one of the most important use cases
> for decimal values: data input and output.
>
> The literals would only appear in programs. However, in most use
> cases, you want to read decimal data from some source, process it
> and then write it back again. This would most likely also require
> using a few decimal literals, but the main source of decimals
> would still be constructors that you choose when writing the
> tools.

True, and the Decimal constructor does what you want. The decimal128
constructor will also do what you want provided you don't need more
than 34 digits. In my proposal you would still be able to trap Inexact
with decimal128 if desired.

> Coming back to floats and the decimal constructor:
>
> In investment banking, for example, people usually work with floats
> all the time. You only convert back to decimals at the very end of some
> calculation. The reason here being either that the calculations
> involve complex operations which are not available for decimals,
> trying to keep error intervals small, or a combination of both.
> In general, you try to use as much precision as you can afford
> (in terms of memory and speed), to keep those error intervals
> small.
>
> In accounting, you often use decimals for storing data with
> an (usually contractually or law based) agreed upon precision
> and rounding logic.

Well this is a situation where you would definitely want all of the
bells and whistles of the full Decimal module. You could still use
decimal literals in this code and they would inter-operate as
expected.

> However, there situations where you have
> to go to floats as well in order to run calculations, e.g.
> for interest, taxes, etc.
>
> In both situations, you want to have the decimal constructor
> take the float values with full precision and only then apply
> the necessary rounding to turn the value into a form which
> complies with the agreed upon rules. It's also not uncommon to
> add correctional bookings to address rounding issues explicitly
> (e.g. when calculating VAT of a large number of individual
> items).
>
> In short: you try to prevent rounding from happening as
> much as possible and when using it, you use it in a
> controlled way.
>
> Based on this, the choice to have the decimal constructor
> use full precision when reading floats is a good one, even
> though it may not feel right for the novice, the casual
> decimal user or as human concept :-)

Don't worry, I think that the suggestion to change the way that
Decimal(float) works is settled now.

> For decimal literals, I'd argue that if you enter a
> value 1.2500d, you are expecting a decimal with 4 decimal
> places precision, not 64 or 128 bits :-)

The decimal32/64/128 types can store the number in this form. Unlike
with binary floats the trailing zeros are not discarded.

> The information about the intended precision is implicit in the
> literal. You see this done in exchange rates, stock prices,
> prices at your grocery store, etc.
>
> The decimals can then be transformed into ones with higher
> precision during calculations and then possibly back to lower
> precision, but this is an explicit decision by the system doing
> the calculation.

The current behaviour will not be changed in this respect. There are
two separate things that you are referring to though. One is the
precision information associated with an individual Decimal e.g. the
number of significant digits even if some are trailing zeros. The
Decimal type preserves this information and so do the decimal64/128
formats:

>>> Decimal('10.1000')
Decimal('10.1000')
>>> Decimal('10.1000e-5')
Decimal('0.000101000')

The other precision is the limiting precision of the current
arithmetic context that is used when rounding. If you have a legal
obligation to make that rounding be just so then you're into what I
would consider to be expert usage of the decimal module.

> Anyway, just throwing in some additional entropy into this
> discussion. Probably not all that helpful, since you're already
> converging on two possible solutions :-)

Thanks!


Oscar

From storchaka at gmail.com  Tue Mar 11 14:15:16 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 11 Mar 2014 15:15:16 +0200
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
Message-ID: <lfn280$851$1@ger.gmane.org>

11.03.14 14:44, Oscar Benjamin ???????(??):
> Am I doing this wrong? Is this just a limitation of the old-style %
> formatting or something that could possibly be improved?

Yes, this is a limitation of the old-style % formatting. Another example:

 >>> '{:.2f}'.format(1.234j)
'0.00+1.23j'
 >>> '%.2f' % 1.234j
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: can't convert complex to float



From solipsis at pitrou.net  Tue Mar 11 14:30:03 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Mar 2014 14:30:03 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
Message-ID: <lfn34f$k4q$1@ger.gmane.org>

Le 11/03/2014 11:51, Oscar Benjamin a ?crit :
>
> When it comes to non-integer types there's just no single good way of
> doing it. So just in the stdlib we already have float, Fraction and
> Decimal. The additional libraries that I use have many more numeric
> types than that. The idea here is that for most users decimal128 just
> is *the* decimal type.

Who are those "most users"? Those who are currently happy with float?

Defining two different but similar types, one for "beginner use" and one 
for "expert use" is a terrible design pattern. People *will* have to 
know about both types. Some libraries will choose to return the beginner 
type, others will choose to return the expert type. Users will have to 
learn the differences, will have to learn how conversions work, etc.

> The decimal.Decimal type is IMO overly complex
> and should really be for niche use-cases like other non-stdlib
> multi-precision numeric types.

What is complex about it?
The notion of a decimal's precision in number of digits is 
understandable by a 10 year old. It is arguably much simpler than 
unicode and character encodings. Besides, for most applications 
Decimal's current precision will be good enough: changing the current 
context (which, I believe, is thread-local, not process-global) is only 
necessary in special cases.

> Java's BigDecimal is more user friendly in some important ways. It
> performs all calculations exactly until you explicitly round.

So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-)

(or perhaps it doesn't have .sqrt()?)

> C++ is introducing decimal32, decimal64, and decimal128 as fixed-width
> decimal floating point types as according to IEEE-754-2008.

C++ has tons of integer types too, but Python has a single one.

Regards

Antoine.



From ncoghlan at gmail.com  Tue Mar 11 14:42:19 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Mar 2014 23:42:19 +1000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfn34f$k4q$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
Message-ID: <CADiSq7c4jpyA1NnXRBd9yfvZ7bcOS0TQdERcE7sYOkBx0nNTzQ@mail.gmail.com>

On 11 Mar 2014 23:31, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> Le 11/03/2014 11:51, Oscar Benjamin a ?crit :
>
>
>> The decimal.Decimal type is IMO overly complex
>> and should really be for niche use-cases like other non-stdlib
>> multi-precision numeric types.
>
>
> What is complex about it?

What is the result of "Decimal('1.0') + Decimal('1e70')"?

Correct answer: insufficient data (since we don't know the current
precision).

That becomes a substantially more questionable answer for "1.0d + 1e70d".
Having apparent numeric literals that can't be constant folded would be a
much bigger design wart than having two different kinds of decimal.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/193e0481/attachment.html>

From oscar.j.benjamin at gmail.com  Tue Mar 11 15:40:09 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 14:40:09 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfn34f$k4q$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
Message-ID: <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>

On 11 March 2014 13:30, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 11/03/2014 11:51, Oscar Benjamin a ?crit :
>>
>> The decimal.Decimal type is IMO overly complex
>> and should really be for niche use-cases like other non-stdlib
>> multi-precision numeric types.
>
> What is complex about it?
> The notion of a decimal's precision in number of digits is understandable by
> a 10 year old.

I agree, that's why I say that we just have a type with a fixed
maximum number of digits. I think lots of people will be able to
understand that.

> It is arguably much simpler than unicode and character
> encodings. Besides, for most applications Decimal's current precision will
> be good enough: changing the current context (which, I believe, is
> thread-local, not process-global) is only necessary in special cases.

The problem is that the context cannot be depended upon so the result
of trivial calculations or even a negative decimal literal will depend
on something that can be changed *anywhere* else in the code. The
expression -1.123456789d will evaluate differently depending on the
context and I think many people will be surprised by that. I expect
that this would lead to naive code that breaks when the context is
changed. Great care needs to be taken to get this right, or otherwise
it should just be considered generally unsafe to change the context.

>> Java's BigDecimal is more user friendly in some important ways. It
>> performs all calculations exactly until you explicitly round.
>
> So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-)
>
> (or perhaps it doesn't have .sqrt()?)

Apparently it does not:
http://stackoverflow.com/questions/13649703/square-root-of-bigdecimal-in-java

But let's take something similar: BigDecimal(1).divide(BigDecimal(3)).
This expression cannot be computed exactly in decimal format so it
raises ArithmeticError. If you want it to round to some precision then
you can provide a context with BigDecimal(1).divide(BigDecimal(3),
mymathcontext). This rounds as specified in mymathcontext and the
result of the expression is not affected by someone else changing a
global arithmetic context.


Oscar

From solipsis at pitrou.net  Tue Mar 11 16:14:34 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Mar 2014 16:14:34 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
Message-ID: <lfn98c$4v9$1@ger.gmane.org>


>> It is arguably much simpler than unicode and character
>> encodings. Besides, for most applications Decimal's current precision will
>> be good enough: changing the current context (which, I believe, is
>> thread-local, not process-global) is only necessary in special cases.
>
> The problem is that the context cannot be depended upon so the result
> of trivial calculations or even a negative decimal literal will depend
> on something that can be changed *anywhere* else in the code. The
> expression -1.123456789d will evaluate differently depending on the
> context and I think many people will be surprised by that. I expect
> that this would lead to naive code that breaks when the context is
> changed. Great care needs to be taken to get this right, or otherwise
> it should just be considered generally unsafe to change the context.

This is a good point, but I still don't think it outweighs the large 
cognitive baggage (and maintenance overhead for library authors as well 
as ourselves) of having two different types doing mostly similar things.

It is also not clear that a decimal type would be really beneficial for 
non-expert users. Most people get by with floats fine.

> But let's take something similar: BigDecimal(1).divide(BigDecimal(3)).
> This expression cannot be computed exactly in decimal format so it
> raises ArithmeticError. If you want it to round to some precision then
> you can provide a context with BigDecimal(1).divide(BigDecimal(3),
> mymathcontext).

But then it becomes, ironically, more cumbersome to spell than
   Decimal(1) / Decimal(3)
(I'm talking about the explicit context, of course, not the .divide() 
method which needn't exist in Python)

Regards

Antoine.



From p.f.moore at gmail.com  Tue Mar 11 16:26:34 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 11 Mar 2014 15:26:34 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfn98c$4v9$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
Message-ID: <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>

On 11 March 2014 15:14, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> The problem is that the context cannot be depended upon so the result
>> of trivial calculations or even a negative decimal literal will depend
>> on something that can be changed *anywhere* else in the code. The
>> expression -1.123456789d will evaluate differently depending on the
>> context and I think many people will be surprised by that. I expect
>> that this would lead to naive code that breaks when the context is
>> changed. Great care needs to be taken to get this right, or otherwise
>> it should just be considered generally unsafe to change the context.
>
> This is a good point, but I still don't think it outweighs the large
> cognitive baggage (and maintenance overhead for library authors as well as
> ourselves) of having two different types doing mostly similar things.

It feels to me as if what we're discovering here is why dynamic
scoping is a bad thing. If context was lexically scoped, all of this
concern about "action at a distance" and user code impacting library
behaviour would go away.

I presume that ship has sailed, though, and there's no practical way
of switching to lexical scoping of decimal contexts...

Paul

From solipsis at pitrou.net  Tue Mar 11 16:45:28 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Mar 2014 16:45:28 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
Message-ID: <lfnb29$sbb$1@ger.gmane.org>

Le 11/03/2014 16:26, Paul Moore a ?crit :
>
> It feels to me as if what we're discovering here is why dynamic
> scoping is a bad thing. If context was lexically scoped, all of this
> concern about "action at a distance" and user code impacting library
> behaviour would go away.

I don't think that's the problem. What you call "dynamic scoping" is a 
feature here, and "lexical scoping" wouldn't solve the issue: simply, 
the parser and compiler run before executing any Python code, so they 
can't see any changes in decimal precision triggered by a library call.

Regards

Antoine.



From mertz at gnosis.cx  Tue Mar 11 17:11:16 2014
From: mertz at gnosis.cx (David Mertz)
Date: Tue, 11 Mar 2014 09:11:16 -0700
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfnb29$sbb$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
 <lfnb29$sbb$1@ger.gmane.org>
Message-ID: <CAEbHw4Zr5zYsPa+sCk2A4cG24TQtOAS-=7aX0w7v-d2x-VsGFg@mail.gmail.com>

I think that the dynamic scoping of getcontext() *is* a problem, but
lexical scoping wouldn't actually solve the issue at hand with decimal
literals.  Even if we imagined a lexically scoped future with local
causality, this code would still produce unpredictable results and would
not allow constant folding:

def add_contst(d):
    decimal.lexical_getcontext().prec(random.randint(1,28))
    return d + 1234.567890d

I'm strongly in favor of Oscar's approach of making a decimal literal be
fixed precision (e.g. decimal128; I prefer that to decimal64).  More
advanced users are welcome to cast that literal to decimal.Decimal with a
particular context, but basic users will find this the least surprising
approach.


On Tue, Mar 11, 2014 at 8:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le 11/03/2014 16:26, Paul Moore a ?crit :
>
>
>> It feels to me as if what we're discovering here is why dynamic
>> scoping is a bad thing. If context was lexically scoped, all of this
>> concern about "action at a distance" and user code impacting library
>> behaviour would go away.
>>
>
> I don't think that's the problem. What you call "dynamic scoping" is a
> feature here, and "lexical scoping" wouldn't solve the issue: simply, the
> parser and compiler run before executing any Python code, so they can't see
> any changes in decimal precision triggered by a library call.
>
> Regards
>
> Antoine.
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/c318ae20/attachment.html>

From oscar.j.benjamin at gmail.com  Tue Mar 11 18:03:01 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 17:03:01 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfnb29$sbb$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
 <lfnb29$sbb$1@ger.gmane.org>
Message-ID: <CAHVvXxRwCsKTZfWzVZW2=XEyWrY-yCj8z=KXz97-LVx_nr2Q=A@mail.gmail.com>

On 11 March 2014 15:45, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le 11/03/2014 16:26, Paul Moore a ?crit :
>
>>
>> It feels to me as if what we're discovering here is why dynamic
>> scoping is a bad thing. If context was lexically scoped, all of this
>> concern about "action at a distance" and user code impacting library
>> behaviour would go away.
>
>
> I don't think that's the problem. What you call "dynamic scoping" is a
> feature here,

I think it would more appropriately be called "global scoping" but yes
it is a feature when used carefully.

> and "lexical scoping" wouldn't solve the issue: simply, the
> parser and compiler run before executing any Python code, so they can't see
> any changes in decimal precision triggered by a library call.

The issue is bigger than constant folding. The problem is about being
able to understand a simple looking expression or block of code. With
the decimal module it becomes necessary to consider many different
cases. What if the precision is less than X or higher than Y? What if
this particular Decimal has a precision exceeding that of the current
context? (The extra precision is not ignored; rounding effectively
occurs *after* each calculation.)

The issue applies to all programmers but consider the case where an
inexperienced user posts code that doesn't work asking for help. On
the one hand you don't know if they've modified the context and can't
be sure that you understand what the code does. On the other hand you
don't want to have to bombard them with technical details about
whether the context was modified or explain how to check if they're
unsure.


Oscar

From oscar.j.benjamin at gmail.com  Tue Mar 11 18:09:37 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 17:09:37 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfn98c$4v9$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
Message-ID: <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com>

On 11 March 2014 15:14, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>>> It is arguably much simpler than unicode and character
>>> encodings. Besides, for most applications Decimal's current precision
>>> will
>>> be good enough: changing the current context (which, I believe, is
>>> thread-local, not process-global) is only necessary in special cases.
>>
>>
>> The problem is that the context cannot be depended upon so the result
>> of trivial calculations or even a negative decimal literal will depend
>> on something that can be changed *anywhere* else in the code. The
>> expression -1.123456789d will evaluate differently depending on the
>> context and I think many people will be surprised by that. I expect
>> that this would lead to naive code that breaks when the context is
>> changed. Great care needs to be taken to get this right, or otherwise
>> it should just be considered generally unsafe to change the context.
>
> This is a good point, but I still don't think it outweighs the large
> cognitive baggage (and maintenance overhead for library authors as well as
> ourselves) of having two different types doing mostly similar things.
>
> It is also not clear that a decimal type would be really beneficial for
> non-expert users. Most people get by with floats fine.

These are good arguments. I guess we'll just have to see what all the
pros and cons look like taken together. (There's always the
possibility of doing nothing of course!)

>> But let's take something similar: BigDecimal(1).divide(BigDecimal(3)).
>> This expression cannot be computed exactly in decimal format so it
>> raises ArithmeticError. If you want it to round to some precision then
>> you can provide a context with BigDecimal(1).divide(BigDecimal(3),
>> mymathcontext).
>
> But then it becomes, ironically, more cumbersome to spell than
>   Decimal(1) / Decimal(3)
> (I'm talking about the explicit context, of course, not the .divide() method
> which needn't exist in Python)

That's why I propose something more along the lines of what the other
languages have done.

I'm not sure if any other languages have the same model as Python's
decimal module except possibly REXX. I don't have any experience of
REXX but from a quick read of some docs the "numeric digits"
instruction seems to have the same global effect as the decimal
module's contexts.


Oscar

From rosuav at gmail.com  Tue Mar 11 18:19:08 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 12 Mar 2014 04:19:08 +1100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com>
Message-ID: <CAPTjJmrVWZ579gQz2VQVBS4fw5n9QsKzbRoCj=u4HtodHtLCSg@mail.gmail.com>

On Wed, Mar 12, 2014 at 4:09 AM, Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> I'm not sure if any other languages have the same model as Python's
> decimal module except possibly REXX. I don't have any experience of
> REXX but from a quick read of some docs the "numeric digits"
> instruction seems to have the same global effect as the decimal
> module's contexts.

Yes, it does. I'm more familiar with numeric digits than with Python's
decimal contexts, but they definitely seem to be doing the same thing.
(Does Python have an equivalent of 'numeric fuzz N', which basically
defines numeric equality as "reduce precision by N digits, round both
numbers, then see if they're equal"?)

ChrisA

From solipsis at pitrou.net  Tue Mar 11 18:43:28 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Mar 2014 18:43:28 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
 <lfnb29$sbb$1@ger.gmane.org>
 <CAHVvXxRwCsKTZfWzVZW2=XEyWrY-yCj8z=KXz97-LVx_nr2Q=A@mail.gmail.com>
Message-ID: <20140311184328.52518261@fsol>

On Tue, 11 Mar 2014 17:03:01 +0000
Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> 
> The issue is bigger than constant folding. The problem is about being
> able to understand a simple looking expression or block of code. With
> the decimal module it becomes necessary to consider many different
> cases. What if the precision is less than X or higher than Y? What if
> this particular Decimal has a precision exceeding that of the current
> context?

Many APIs have context-dependent meaning, especially in a duck-typed
languages such as Python. I don't think decimals are a special-case
here.

OTOH, *if you know that d and e are decimals*, then what e.g. `d + e`
does is pretty clear, regardless of the context. Knowing the precision
isn't necessary for *understanding* what the code does; it may be
necessary to track an unexpectedly "wrong" result, but that's all.

And, besides, I think much of this argument is based on FUD. Probably
in most real-world situations people don't set the decimal context to
some arbitrarily low precision, so I don't know what the *practical*
problem is.

> The issue applies to all programmers but consider the case where an
> inexperienced user posts code that doesn't work asking for help.

Define "doesn't work". If the stated result simply isn't precise
enough (or is too precise for the user's expectation), then it will be
obvious what the cause is! No need for some heroic spelunking here,
AFAICT.

Regards

Antoine.



From stefan at bytereef.org  Tue Mar 11 19:41:59 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 11 Mar 2014 19:41:59 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
Message-ID: <20140311184159.GA948@sleipnir.bytereef.org>

Nick Coghlan <ncoghlan at gmail.com> wrote:
> Stefan: use the existing decimal type, change default context to Decimal64,
> round all decimal literals to current context

I'm open to many options, including doing nothing or somehow deal with
the UNARY_MINUS issue in another way.

First of all, I don't think that having a decimal literal is in any
way essential -- professional users probably deal with Decimals much
like Marc-Andre described and I cannot see why we should add a literal
primarily for inexperienced users.

So, if we add a literal, professional users should be able to use it
in a meaningful way, too.


I understand the benefits of a context with fixed precision, Emax and
Emin, so here are the drawbacks:

  1) It must be possible to set rounding and traps, so the context
     is not really fixed.

  2) Code that mixes Decimal64 and Decimal (virtually all professional
     code) will get hard to read and to understand, since we now have
     two contexts (problems) instead of one.

     Even tracking the flags will be challenging.

  3) We must decide between Decimal64 and Decimal128. The latter is
     quite a bit slower, the former may not have sufficient precision.


Now, to the argument that a context may be changed by a third party
package:

This is inherently true for all global (thread local) state and is really
a problem of the package.  Packages can do that with locales (which would
also affect the "n" formatting for Decimal64), packages can do that right
now with the existing Decimal type.

In fact, for a highly robust package, people should not use the global
context at all [1].  IIRC potential problems with coroutines were pointed
out on python-dev recently.


And that is another problem:  If a package is using fixed Decimal64 literals,
is it then supposed to use only the official global context?

I think that would be a bad idea, unless the official context is completely
frozen.  That on the other hand would deprive users of controlling Overflow,
clamping etc.


Stefan Krah



[1] If you read and implement this, do watch performance a little (method
    lookups can ruin a lot, using the operators is usually faster).


From tinchester at gmail.com  Tue Mar 11 19:46:07 2014
From: tinchester at gmail.com (=?UTF-8?B?VGluIFR2cnRrb3ZpxIc=?=)
Date: Tue, 11 Mar 2014 19:46:07 +0100
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <mailman.40053.1394557800.18129.python-ideas@python.org>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
Message-ID: <531F59EF.8030808@gmail.com>

Hello,

it occured to me today that it's not currently possible to parse ISO 
8601 dates using the datetime.datetime.strftime function (I was parsing 
datetimes generated by Postgres). The problem is in the semicolon in the 
time zone offset.

'1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time 
representation, the likes of which can be generated by the datetime 
module itself (the isoformat method). The %z strftime directive only 
recognizes offsets without the semicolon. The Python docs also direct 
users to inspect the platform strftime documentation; on my system the 
man page clearly states %z is "... The +hhmm or -hhmm numeric timezone", 
again, no semicolon support.

Googling around, most common suggestions are to use a third party 
library (such as dateutil), so either this functionality really doesn't 
exist in a simple form in the datetime module, or is really 
undiscoverable. It seems to me ISO 8601 is significant enough (since 
there are even special methods for it in the datetime module) for it to 
be parsable, in a non-complex way, by the standard library.

I guess it's interesting to point out a new flag was added to Java's 
SimpleDateFormat's date and time patterns in Java 7 - the X directive 
stands for "ISO 8601 time zone" (-08; -0800; -08:00, Z). I've worked 
with this so I happen to know it off the top of my head.

Thanks in advance for comments.

Tin

From guido at python.org  Tue Mar 11 19:57:24 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Mar 2014 11:57:24 -0700
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <531F59EF.8030808@gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
 <531F59EF.8030808@gmail.com>
Message-ID: <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>

Can you show or point us to some examples of the types of dates that cannot
be parsed? Do you happen to have the specification for those? "ISO 8601"
alone doesn't tell me much, there are many different formats endorsed by
the standard, and the standard is hard to read. I can't find it in
Wikipedia either, nor in the Postgres docs.

Is it something that a simple regular expression could extract into pieces
that can be parsed with Python's strftime?


On Tue, Mar 11, 2014 at 11:46 AM, Tin Tvrtkovi? <tinchester at gmail.com>wrote:

> Hello,
>
> it occured to me today that it's not currently possible to parse ISO 8601
> dates using the datetime.datetime.strftime function (I was parsing
> datetimes generated by Postgres). The problem is in the semicolon in the
> time zone offset.
>
> '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time
> representation, the likes of which can be generated by the datetime module
> itself (the isoformat method). The %z strftime directive only recognizes
> offsets without the semicolon. The Python docs also direct users to inspect
> the platform strftime documentation; on my system the man page clearly
> states %z is "... The +hhmm or -hhmm numeric timezone", again, no semicolon
> support.
>
> Googling around, most common suggestions are to use a third party library
> (such as dateutil), so either this functionality really doesn't exist in a
> simple form in the datetime module, or is really undiscoverable. It seems
> to me ISO 8601 is significant enough (since there are even special methods
> for it in the datetime module) for it to be parsable, in a non-complex way,
> by the standard library.
>
> I guess it's interesting to point out a new flag was added to Java's
> SimpleDateFormat's date and time patterns in Java 7 - the X directive
> stands for "ISO 8601 time zone" (-08; -0800; -08:00, Z). I've worked with
> this so I happen to know it off the top of my head.
>
> Thanks in advance for comments.
>
> Tin
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/c42dee5d/attachment-0001.html>

From alexander.belopolsky at gmail.com  Tue Mar 11 20:08:06 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 11 Mar 2014 15:08:06 -0400
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
 <531F59EF.8030808@gmail.com>
 <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>
Message-ID: <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com>

On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum <guido at python.org> wrote:

> Can you show or point us to some examples of the types of dates that
> cannot be parsed? Do you happen to have the specification for those?


This is a known limitation. We don't have an strftime code for TZ in hh:mm
format.  GNU date uses ':z' for this.

See http://bugs.python.org/issue5207
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/8896d1cc/attachment.html>

From guido at python.org  Tue Mar 11 20:10:51 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Mar 2014 12:10:51 -0700
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
 <531F59EF.8030808@gmail.com>
 <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>
 <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com>
Message-ID: <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com>

Ah, colon, not semicolon.
On Mar 11, 2014 12:08 PM, "Alexander Belopolsky" <
alexander.belopolsky at gmail.com> wrote:

>
> On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum <guido at python.org>wrote:
>
>> Can you show or point us to some examples of the types of dates that
>> cannot be parsed? Do you happen to have the specification for those?
>
>
> This is a known limitation. We don't have an strftime code for TZ in hh:mm
> format.  GNU date uses ':z' for this.
>
> See http://bugs.python.org/issue5207
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/5979d87d/attachment.html>

From random832 at fastmail.us  Tue Mar 11 20:14:22 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 11 Mar 2014 15:14:22 -0400
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <531F59EF.8030808@gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
 <531F59EF.8030808@gmail.com>
Message-ID: <1394565262.23609.93273945.5C970096@webmail.messagingengine.com>

On Tue, Mar 11, 2014, at 14:46, Tin Tvrtkovi? wrote:
> '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time 
> representation, the likes of which can be generated by the datetime 
> module itself (the isoformat method). The %z strftime directive only 
> recognizes offsets without the semicolon. The Python docs also direct 
> users to inspect the platform strftime documentation; on my system the 
> man page clearly states %z is "... The +hhmm or -hhmm numeric timezone", 
> again, no semicolon support.

Maybe Python should switch to platform-independent time
parsing/formatting implementations. These can have a %z that accepts
this format.

I will incidentally note that as of SUSv7 systems are not required to
support %z for strptime at all (nor does it therefore impose any
requirement to reject formats including a colon, or 'Z'.)

From random832 at fastmail.us  Tue Mar 11 20:17:37 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 11 Mar 2014 15:17:37 -0400
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <lfn280$851$1@ger.gmane.org>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
Message-ID: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>

On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
> 11.03.14 14:44, Oscar Benjamin ???????(??):
> > Am I doing this wrong? Is this just a limitation of the old-style %
> > formatting or something that could possibly be improved?
> 
> Yes, this is a limitation of the old-style % formatting. Another example:

Is % formatting not going to be updated to support new features or fix
bugs like this? Is it going to be deprecated? Will a tool be provided to
convert usages (at least with static format strings) to str.format()?

From random832 at fastmail.us  Tue Mar 11 20:22:43 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 11 Mar 2014 15:22:43 -0400
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <lfnb29$sbb$1@ger.gmane.org>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
 <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <lfn34f$k4q$1@ger.gmane.org>
 <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com>
 <lfn98c$4v9$1@ger.gmane.org>
 <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com>
 <lfnb29$sbb$1@ger.gmane.org>
Message-ID: <1394565763.26478.93278337.000AD481@webmail.messagingengine.com>

On Tue, Mar 11, 2014, at 11:45, Antoine Pitrou wrote:
> I don't think that's the problem. What you call "dynamic scoping" is a 
> feature here, and "lexical scoping" wouldn't solve the issue: simply, 
> the parser and compiler run before executing any Python code, so they 
> can't see any changes in decimal precision triggered by a library call.

They could if all such changes were made by keyworded statements or
expressions (which the compiler can understand just fine) rather than
library calls. Which they would necessarily have to be if contexts were
lexically scoped, since they would have to affect the execution of
operators within their scope (probably by passing the context as a third
argument to the operator implementations) without affecting functions
called by them.

From pyideas at rebertia.com  Tue Mar 11 20:28:48 2014
From: pyideas at rebertia.com (Chris Rebert)
Date: Tue, 11 Mar 2014 12:28:48 -0700
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <531F59EF.8030808@gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>
 <531F59EF.8030808@gmail.com>
Message-ID: <CAMZYqRSJpZ=n6RLLk0UREQE_sFAk3MhbnzuaBA4x_AW-2roXdw@mail.gmail.com>

On Tue, Mar 11, 2014 at 11:46 AM, Tin Tvrtkovi? <tinchester at gmail.com> wrote:
> Hello,
>
> it occured to me today that it's not currently possible to parse ISO 8601
> dates using the datetime.datetime.strftime function (I was parsing datetimes
> generated by Postgres). The problem is in the semicolon in the time zone
> offset.
>
> '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time
> representation, the likes of which can be generated by the datetime module
> itself (the isoformat method). The %z strftime directive only recognizes
> offsets without the semicolon. The Python docs also direct users to inspect
> the platform strftime documentation; on my system the man page clearly
> states %z is "... The +hhmm or -hhmm numeric timezone", again, no semicolon
> support.
>
> Googling around, most common suggestions are to use a third party library
> (such as dateutil), so either this functionality really doesn't exist in a
> simple form in the datetime module, or is really undiscoverable. It seems to
> me ISO 8601 is significant enough (since there are even special methods for
> it in the datetime module) for it to be parsable, in a non-complex way, by
> the standard library.

There's already an open bug about this (well, technically RFC 3339,
but close enough):
http://bugs.python.org/issue15873

Cheers,
Chris

From random832 at fastmail.us  Tue Mar 11 20:38:08 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 11 Mar 2014 15:38:08 -0400
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140311184159.GA948@sleipnir.bytereef.org>
References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311184159.GA948@sleipnir.bytereef.org>
Message-ID: <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>

On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote:
>   1) It must be possible to set rounding and traps, so the context
>      is not really fixed.

It _must_ be possible? If there's a way to set rounding for floats, I
can't find it. As for traps, I can't find that either, but a constant
operation that may (will?) trap disables constant-folding.

>>> dis.dis(lambda x: 1.0/0.0)
  1           0 LOAD_CONST               1 (1.0)
              3 LOAD_CONST               2 (0.0)
              6 BINARY_TRUE_DIVIDE
              7 RETURN_VALUE

Maybe what needs to be done is to rigorously define what is a constant
expression (C does this) and explicitly state what context (including
rounding and traps) they take place in, independent of the runtime
context - which, then, an implementation that does _not_ do constant
folding must use to perform operations within a constant expression at
runtime.

Incidentally, did you know there's apparently no limit to how large an
integer ** will constant-fold? I say this because I assume if there were
it would be far below 2**9999999. On the other hand, 2.0**1024 doesn't
constant-fold.

From tinchester at gmail.com  Tue Mar 11 20:39:28 2014
From: tinchester at gmail.com (=?UTF-8?B?VGluIFR2cnRrb3ZpxIc=?=)
Date: Tue, 11 Mar 2014 20:39:28 +0100
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>	<531F59EF.8030808@gmail.com>	<CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>	<CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com>
 <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com>
Message-ID: <531F6670.9010304@gmail.com>

Ugh, I need to get into the habit of checking the issue tracker first. 
Sorry for not doing that this time.

The resolution of that issue is what I'm after, yeah. Also, yes, I meant 
colon instead of semicolon.

Thanks!

On 11.03.2014 20:10, Guido van Rossum wrote:
>
> Ah, colon, not semicolon.
>
> On Mar 11, 2014 12:08 PM, "Alexander Belopolsky" 
> <alexander.belopolsky at gmail.com 
> <mailto:alexander.belopolsky at gmail.com>> wrote:
>
>
>     On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum
>     <guido at python.org <mailto:guido at python.org>> wrote:
>
>         Can you show or point us to some examples of the types of
>         dates that cannot be parsed? Do you happen to have the
>         specification for those? 
>
>
>     This is a known limitation. We don't have an strftime code for TZ
>     in hh:mm format.  GNU date uses ':z' for this.
>
>     See http://bugs.python.org/issue5207
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/59a62b1e/attachment.html>

From stefan at bytereef.org  Tue Mar 11 20:46:25 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 11 Mar 2014 20:46:25 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>
References: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311184159.GA948@sleipnir.bytereef.org>
 <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>
Message-ID: <20140311194625.GA2110@sleipnir.bytereef.org>

random832 at fastmail.us <random832 at fastmail.us> wrote:
> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote:
> >   1) It must be possible to set rounding and traps, so the context
> >      is not really fixed.
> 
> It _must_ be possible? If there's a way to set rounding for floats, I
> can't find it. As for traps, I can't find that either, but a constant
> operation that may (will?) trap disables constant-folding.

If we use IEEE, the default context would not raise anything -- infinities
and NaNs would silently propagate.  This is valuable, but it should be
possible switch it off.  Settting rounding is obviously valuable, too.


Stefan Krah



From breamoreboy at yahoo.co.uk  Tue Mar 11 20:51:17 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 11 Mar 2014 19:51:17 +0000
Subject: [Python-ideas] Parsing ISO 8601 with the standard library
In-Reply-To: <531F6670.9010304@gmail.com>
References: <mailman.40053.1394557800.18129.python-ideas@python.org>	<531F59EF.8030808@gmail.com>	<CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com>	<CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com>
 <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com>
 <531F6670.9010304@gmail.com>
Message-ID: <lfnpfl$lvb$1@ger.gmane.org>

On 11/03/2014 19:39, Tin Tvrtkovi? wrote:
> Ugh, I need to get into the habit of checking the issue tracker first.
> Sorry for not doing that this time.
>
> The resolution of that issue is what I'm after, yeah. Also, yes, I meant
> colon instead of semicolon.
>
> Thanks!
>
> On 11.03.2014 20:10, Guido van Rossum wrote:
>>
>> Ah, colon, not semicolon.
>>
>> On Mar 11, 2014 12:08 PM, "Alexander Belopolsky"
>> <alexander.belopolsky at gmail.com
>> <mailto:alexander.belopolsky at gmail.com>>
>> wrote:
>>
>>
>>     On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum
>>     <guido at python.org
>>     <mailto:guido at python.org>> wrote:
>>
>>         Can you show or point us to some examples of the types of
>>         dates that cannot be parsed? Do you happen to have the
>>         specification for those?
>>
>>
>>     This is a known limitation. We don't have an strftime code for TZ
>>     in hh:mm format.  GNU date uses ':z' for this.
>>
>>     See http://bugs.python.org/issue5207
>>
>

In case you haven't seen it there's also 
http://bugs.python.org/issue15873 - whether both need to remain open 
I've no idea.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From abarnert at yahoo.com  Tue Mar 11 21:10:01 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 11 Mar 2014 13:10:01 -0700
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
Message-ID: <27743BD2-22A8-4194-A177-0AE63D338384@yahoo.com>

On Mar 11, 2014, at 12:17, random832 at fastmail.us wrote:

> On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
>> 11.03.14 14:44, Oscar Benjamin ???????(??):
>>> Am I doing this wrong? Is this just a limitation of the old-style %
>>> formatting or something that could possibly be improved?
>> 
>> Yes, this is a limitation of the old-style % formatting. Another example:
> 
> Is % formatting not going to be updated to support new features or fix
> bugs like this? 

That's an inherent limitation of % formatting: it uses printf-style type codes, whose meaning is baked into the % operator, rather than calling __format__ on its arguments to handle it polymorphically. %f means treat the argument as a float. That limited flexibility is part of what allows it to be simpler, faster, able to share some format strings with C, etc.

It's doing what it was designed to do; the fact that it wasn't designed the same as str.format isn't a bug.

That's not to say that it couldn't be extended with new type codes for decimals. If C1x adds printf type codes (or if gcc or some other compiler creates its own extension), you could make a pretty good argument for % formatting to borrow them.

> Is it going to be deprecated?

It wasn't deprecated in 3.0, and after the issue was raised again, I'm pretty sure the decision was that deprecation is off the table until at least Python 4000. And if you go back and read the arguments, the reason it wasn't deprecated is that for some use cases it's better, in part because it's not extensible, and has the corresponding advantages mentioned above.

> Will a tool be provided to
> convert usages (at least with static format strings) to str.format()?

I'm pretty sure people have already written such tools, back when str.format was first created. I don't think too many people use them. If %-formatting is working for your existing code, or appropriate for some new code, there's no reason to change it. If {}-formatting is appropriate for some new code, it's easier to write it directly than to try to write a different format and then convert it.


From oscar.j.benjamin at gmail.com  Tue Mar 11 21:26:28 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 20:26:28 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140311194625.GA2110@sleipnir.bytereef.org>
References: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311184159.GA948@sleipnir.bytereef.org>
 <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>
 <20140311194625.GA2110@sleipnir.bytereef.org>
Message-ID: <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com>

On 11 March 2014 19:46, Stefan Krah <stefan at bytereef.org> wrote:
> random832 at fastmail.us <random832 at fastmail.us> wrote:
>> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote:
>> >   1) It must be possible to set rounding and traps, so the context
>> >      is not really fixed.
>>
>> It _must_ be possible? If there's a way to set rounding for floats, I
>> can't find it. As for traps, I can't find that either, but a constant
>> operation that may (will?) trap disables constant-folding.
>
> If we use IEEE, the default context would not raise anything -- infinities
> and NaNs would silently propagate.  This is valuable, but it should be
> possible switch it off.  Settting rounding is obviously valuable, too.

I was intending that it would be possible to use the traps and flags
on this context. Perhaps Stefan knows something I don't but I thought
that the constant folder could just use a hyper-sensitive context and
check for exceptions (or otherwise check the flags). I assumed this is
how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not
folded so that they can raise exceptions at the appropriate time.

Rounding is a trickier one. It would be possible to do the same thing
by having the constant folder trap Inexact but it's a little more
limiting since it applies to expressions that are more likely to have
useful values.


Oscar

From ethan at stoneleaf.us  Tue Mar 11 21:25:00 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 11 Mar 2014 13:25:00 -0700
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
Message-ID: <531F711C.1020106@stoneleaf.us>

On 03/11/2014 12:17 PM, random832 at fastmail.us wrote:
> On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
>> 11.03.14 14:44, Oscar Benjamin ???????(??):
>>> Am I doing this wrong? Is this just a limitation of the old-style %
>>> formatting or something that could possibly be improved?
>>
>> Yes, this is a limitation of the old-style % formatting. Another example:
>
> Is % formatting not going to be updated to support new features

No.

> or fix bugs like this?

It's not a bug.  Decimal is not float.

>  Is it going to be deprecated?

No.  %-formatting has its place.

> Will a tool be provided to convert usages (at least with static format strings)
>  to str.format()?

No.  Feel free to write one and share it with the community.  :)

Basically, %-formatting is primitive, .format-formatting is advanced.  If you want advanced behavior, use the advanced 
method.

--
~Ethan~

From stefan at bytereef.org  Tue Mar 11 21:53:31 2014
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 11 Mar 2014 21:53:31 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com>
References: <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311184159.GA948@sleipnir.bytereef.org>
 <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>
 <20140311194625.GA2110@sleipnir.bytereef.org>
 <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com>
Message-ID: <20140311205331.GA2508@sleipnir.bytereef.org>

Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> >> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote:
> >> >   1) It must be possible to set rounding and traps, so the context
> >> >      is not really fixed.
> >>
> >> It _must_ be possible? If there's a way to set rounding for floats, I
> >> can't find it. As for traps, I can't find that either, but a constant
> >> operation that may (will?) trap disables constant-folding.
> >
> > If we use IEEE, the default context would not raise anything -- infinities
> > and NaNs would silently propagate.  This is valuable, but it should be
> > possible switch it off.  Settting rounding is obviously valuable, too.
> 
> I was intending that it would be possible to use the traps and flags
> on this context. Perhaps Stefan knows something I don't but I thought
> that the constant folder could just use a hyper-sensitive context and
> check for exceptions (or otherwise check the flags). I assumed this is
> how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not
> folded so that they can raise exceptions at the appropriate time.

I meant that an external package that modifies e.g. rounding in the global
context would also affect results, so the problem (which was used as an
argument in favor of the fixed width decimal) is not limited to exponents
and precision.


Stefan Krah



From oscar.j.benjamin at gmail.com  Tue Mar 11 21:59:17 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 20:59:17 +0000
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140311205331.GA2508@sleipnir.bytereef.org>
References: <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311184159.GA948@sleipnir.bytereef.org>
 <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com>
 <20140311194625.GA2110@sleipnir.bytereef.org>
 <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com>
 <20140311205331.GA2508@sleipnir.bytereef.org>
Message-ID: <CAHVvXxT6NdpQdUdhyOxBZS+aFCbUpK6mW0-FevBu4XyODW7Eow@mail.gmail.com>

On 11 March 2014 20:53, Stefan Krah <stefan at bytereef.org> wrote:
> Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>
>> I was intending that it would be possible to use the traps and flags
>> on this context. Perhaps Stefan knows something I don't but I thought
>> that the constant folder could just use a hyper-sensitive context and
>> check for exceptions (or otherwise check the flags). I assumed this is
>> how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not
>> folded so that they can raise exceptions at the appropriate time.
>
> I meant that an external package that modifies e.g. rounding in the global
> context would also affect results, so the problem (which was used as an
> argument in favor of the fixed width decimal) is not limited to exponents
> and precision.

Could it not be handled by having the constant folder trap Inexact? I
though that if all calculations can be exact within the fixed
precision limit and exponent range then no rounding occurs so the
result is independent of the rounding mode. Or have I missed
something?


Oscar

From ron3200 at gmail.com  Tue Mar 11 22:44:58 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Tue, 11 Mar 2014 16:44:58 -0500
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <531F0286.8090306@egenix.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>	<CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>	<CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>	<CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>	<CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>	<CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>	<20140310105942.GB12595@ando>	<CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>	<20140310135342.GA10968@sleipnir.bytereef.org>	<CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>	<CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>	<CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzY
 d8WAQ@mail.gmail.com>	<20140311082449.4f0c0fdb@fsol>	<CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>
 <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>
 <531F0286.8090306@egenix.com>
Message-ID: <lfo04c$960$1@ger.gmane.org>



On 03/11/2014 07:33 AM, M.-A. Lemburg wrote:
> On 11.03.2014 12:46, Nick Coghlan wrote:
>> >...[Oscar's proposal being that picking just one decimal floating
>> >     point precision as*the*  floating point precision]...
>> >
>> >That approach buys us a couple of key benefits:
>> >
>> >- they would be true constants, so constant folding, etc, would work
>> >normally, and you could tell from reading the code exactly what it
>> >will do, without needing to worry about the impact of thread local
>> >state
>> >- you could generally use the builtin decimal literals without
>> >learning anything about decimal contexts, because decimal literals
>> >wouldn't be context dependent (although their behaviour would be
>> >formally defined in terms of an IEEE decimal context, for most users
>> >it would just be "this is how Python decimal literals behave")
>> >
>> >The guidance to new users would then be*don't*  use the decimal
>> >module, use decimal literals instead. If you need more configurable
>> >behaviour,*then*  reach for the decimal module. However, the explicit
>> >methods on decimal context objects should probably still be updated to
>> >accept both fixed and variable precision decimals under this model.
> I think you are leaving out one of the most important use cases
> for decimal values: data input and output.
>
> The literals would only appear in programs. However, in most use
> cases, you want to read decimal data from some source, process it
> and then write it back again. This would most likely also require
> using a few decimal literals, but the main source of decimals
> would still be constructors that you choose when writing the
> tools.
>
> Coming back to floats and the decimal constructor:
>
> In investment banking, for example, people usually work with floats
> all the time. You only convert back to decimals at the very end of some
> calculation. The reason here being either that the calculations
> involve complex operations which are not available for decimals,
> trying to keep error intervals small, or a combination of both.
> In general, you try to use as much precision as you can afford
> (in terms of memory and speed), to keep those error intervals
> small.
>
> In accounting, you often use decimals for storing data with
> an (usually contractually or law based) agreed upon precision
> and rounding logic. However, there situations where you have
> to go to floats as well in order to run calculations, e.g.
> for interest, taxes, etc.
>
> In both situations, you want to have the decimal constructor
> take the float values with full precision and only then apply
> the necessary rounding to turn the value into a form which
> complies with the agreed upon rules. It's also not uncommon to
> add correctional bookings to address rounding issues explicitly
> (e.g. when calculating VAT of a large number of individual
> items).
>
> In short: you try to prevent rounding from happening as
> much as possible and when using it, you use it in a
> controlled way.
>
> Based on this, the choice to have the decimal constructor
> use full precision when reading floats is a good one, even
> though it may not feel right for the novice, the casual
> decimal user or as human concept:-)
>
> For decimal literals, I'd argue that if you enter a
> value 1.2500d, you are expecting a decimal with 4 decimal
> places precision, not 64 or 128 bits:-)
>
> The information about the intended precision is implicit in the
> literal. You see this done in exchange rates, stock prices,
> prices at your grocery store, etc.
>
> The decimals can then be transformed into ones with higher
> precision during calculations and then possibly back to lower
> precision, but this is an explicit decision by the system doing
> the calculation.
>
> Anyway, just throwing in some additional entropy into this
> discussion. Probably not all that helpful, since you're already
> converging on two possible solutions:-)

I like django's measurement object approach.

https://docs.djangoproject.com/en/dev/ref/contrib/gis/measure/

It just needs a way to track accuracy (significant digits).  Which can be 
independent of the underlying numeric type.  (I think django has that too.)

Possibly, it could promote it's underlying type (int, float, decimal) if it 
needs to, or raise an exception if the underlying type isn't accurate 
enough to provide the requested precision.

The same concept could be used for currency calculations, or time for that 
matter.

This would do much more than adding a new underlying type to python.  It 
encapsulates expert knowledge into objects that can be used by user who's 
expertise may be in other areas.

The actual underlying numeric type could depend on the amount of accuracy 
needed rather than some preconceived notion of greater accuracy that 
probably isn't needed in most cases.  Ie.. Decimal128 for calculations that 
only require 6 significant digits.

Cheers,
    Ron


From random832 at fastmail.us  Tue Mar 11 22:51:06 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 11 Mar 2014 17:51:06 -0400
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <531F711C.1020106@stoneleaf.us>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
Message-ID: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>

On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
> > Is % formatting not going to be updated to support new features
> 
> No.
>
> > or fix bugs like this?
> 
> It's not a bug.  Decimal is not float.

The bug to which I was referring involves not calling __format__ on
whatever object is passed in as str.format does - Decimal not being
float is no impediment to str.format and Decimal.__format__ accepting
the same format strings that float.__format__ does.

> >  Is it going to be deprecated?
> 
> No.  %-formatting has its place.

What place is that?

    There should be one-- and preferably only one --obvious way to do
    it.

We've got two syntaxes, and two functions, that do _almost_ the same
thing, but with two completely different microlanguages. Why?

From ncoghlan at gmail.com  Tue Mar 11 23:30:40 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 12 Mar 2014 08:30:40 +1000
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
Message-ID: <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com>

On 12 Mar 2014 07:51, <random832 at fastmail.us> wrote:
>
> On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
> > > Is % formatting not going to be updated to support new features
> >
> > No.
> >
> > > or fix bugs like this?
> >
> > It's not a bug.  Decimal is not float.
>
> The bug to which I was referring involves not calling __format__ on
> whatever object is passed in as str.format does - Decimal not being
> float is no impediment to str.format and Decimal.__format__ accepting
> the same format strings that float.__format__ does.
>
> > >  Is it going to be deprecated?
> >
> > No.  %-formatting has its place.
>
> What place is that?
>
>     There should be one-- and preferably only one --obvious way to do
>     it.
>
> We've got two syntaxes, and two functions, that do _almost_ the same
> thing, but with two completely different microlanguages. Why?

Start by reading PEP 3101, which added str.format to address the
limitations of mod-formatting.

For new users, the recommendation is unconditional: use str.format.

mod-formatting is now a power tool for optimisation, consistency with C
implementations, templating format strings without excessive escaping and
(the main reason it still exists), backwards compatibility (both for
existing code and existing programmers).

The original plan was to eventually deprecate mod-formatting entirely, but
not only did we discover that was harder than expected, we also found it
was actually incredibly useful in some cases to have a formatting system
available that relied on coercion rather than polymorphism, since it could
be faster, was more predictable and also more easily mapped to other
languages like C (especially since Python's mod-formatting is based on C's
printf syntax).

That's why the binary interpolation proposal for 3.5 now *only* proposes
restoring mod-formatting for the binary domain, leaving str.format as a
text domain only operation.

Cheers,
Nick.

> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/b19fc92a/attachment.html>

From ethan at stoneleaf.us  Tue Mar 11 22:58:42 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 11 Mar 2014 14:58:42 -0700
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
Message-ID: <531F8712.3010805@stoneleaf.us>

On 03/11/2014 02:51 PM, random832 at fastmail.us wrote:
> On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
>> at some point prior, random832 at fastmail.us wrote:
>>>
>>> Is % formatting not going to be updated to support new features
>>
>> No.
>>
>>> or fix bugs like this?
>>
>> It's not a bug.  Decimal is not float.
>
> The bug to which I was referring involves not calling __format__ on
> whatever object is passed in as str.format does - Decimal not being
> float is no impediment to str.format and Decimal.__format__ accepting
> the same format strings that float.__format__ does.

That is not a bug.  str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.


>>>   Is it going to be deprecated?
>>
>> No.  %-formatting has its place.
>
> What place is that?
>
>      There should be one-- and preferably only one --obvious way to do
>      it.

You'll notice the word 'preferably' in there.  The preferred method is format.  %-formatting has a good place in lazy 
evaluation (in logging, for example), as well as other areas.


> We've got two syntaxes, and two functions, that do _almost_ the same
> thing, but with two completely different microlanguages. Why?

Because % was there first, but had some failings that are addressed by format.

--
~Ethan~

From rosuav at gmail.com  Tue Mar 11 23:47:33 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 12 Mar 2014 09:47:33 +1100
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com>
Message-ID: <CAPTjJmrbz1o7+XyqSHTEOY_jtm6bz0jFYryjYAsELtUDR9fgGg@mail.gmail.com>

On Wed, Mar 12, 2014 at 9:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> mod-formatting is now a power tool for optimisation, consistency with C
> implementations, templating format strings without excessive escaping and
> (the main reason it still exists), backwards compatibility (both for
> existing code and existing programmers).

And consistency with other C-derived languages, too. The only
weirdness of Python's that isn't found in others is the "hey look
isn't it cool" use of the % operator, which results in edge cases
because the operator always takes exactly one argument on the RHS -
which can be solved by turning it into a function:

>>> def sprintf(fmt, *args):
    return fmt%args

>>> sprintf("foo %s bar",(1,2,3))
'foo (1, 2, 3) bar'
>>> sprintf("foo %d bar %d quux",1,2)
'foo 1 bar 2 quux'

ChrisA

From oscar.j.benjamin at gmail.com  Tue Mar 11 23:49:23 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 11 Mar 2014 22:49:23 +0000
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <531F8712.3010805@stoneleaf.us>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <531F8712.3010805@stoneleaf.us>
Message-ID: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>

On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote:
>>
>> The bug to which I was referring involves not calling __format__ on
>> whatever object is passed in as str.format does - Decimal not being
>> float is no impediment to str.format and Decimal.__format__ accepting
>> the same format strings that float.__format__ does.
>
> That is not a bug.  str.__mod__ is not documented as calling an object's
> __format__ method, and indeed it does not.

Instead it calls float on the object:

>>> class A:
...   def __float__(self):
...     print('__float__ called')
...
>>> '%f' % A()
__float__ called
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: nb_float should return float object

There's no mention of that anywhere in the docs though:
http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting

There's also no mention of this in the decimal docs.


Oscar

From ethan at stoneleaf.us  Tue Mar 11 23:46:45 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 11 Mar 2014 15:46:45 -0700
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <531F8712.3010805@stoneleaf.us>
 <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
Message-ID: <531F9255.1090407@stoneleaf.us>

On 03/11/2014 03:49 PM, Oscar Benjamin wrote:
> On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote:
>>>
>>> The bug to which I was referring involves not calling __format__ on
>>> whatever object is passed in as str.format does - Decimal not being
>>> float is no impediment to str.format and Decimal.__format__ accepting
>>> the same format strings that float.__format__ does.
>>
>> That is not a bug.  str.__mod__ is not documented as calling an object's
>> __format__ method, and indeed it does not.
>
> Instead it calls float on the object:
>
>>>> class A:
> ...   def __float__(self):
> ...     print('__float__ called')
> ...
>>>> '%f' % A()
> __float__ called

Well, you did ask for a float, and how else are you going to get one?  ;)

> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: nb_float should return float object
>
> There's no mention of that anywhere in the docs though:
> http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting

We should probably fix that.

--
~Ethan~

From abarnert at yahoo.com  Wed Mar 12 01:45:38 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 11 Mar 2014 17:45:38 -0700
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <531F8712.3010805@stoneleaf.us>
 <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
Message-ID: <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com>


On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

> On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote:
>>> 
>>> The bug to which I was referring involves not calling __format__ on
>>> whatever object is passed in as str.format does - Decimal not being
>>> float is no impediment to str.format and Decimal.__format__ accepting
>>> the same format strings that float.__format__ does.
>> 
>> That is not a bug.  str.__mod__ is not documented as calling an object's
>> __format__ method, and indeed it does not.
> 
> Instead it calls float on the object:
> 
>>>> class A:
> ...   def __float__(self):
> ...     print('__float__ called')
> ...
>>>> '%f' % A()
> __float__ called
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> TypeError: nb_float should return float object
> 
> There's no mention of that anywhere in the docs though:
> http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting

I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't.

It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()".

Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)?

> There's also no mention of this in the decimal docs.

Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.

From harrismh777 at gmail.com  Wed Mar 12 06:33:48 2014
From: harrismh777 at gmail.com (Mark H. Harris)
Date: Tue, 11 Mar 2014 22:33:48 -0700 (PDT)
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <20140311082449.4f0c0fdb@fsol>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>
 <20140309003925.GC28804@ando>
 <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com>
 <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com>
 <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>
 <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>
 <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>
 <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>
 <20140310105942.GB12595@ando>
 <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>
 <20140310135342.GA10968@sleipnir.bytereef.org>
 <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>
 <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>
 <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>
 <20140311082449.4f0c0fdb@fsol>
Message-ID: <35c7108e-2757-4420-80fd-a7f6e6d2a495@googlegroups.com>



On Tuesday, March 11, 2014 2:24:49 AM UTC-5, Antoine Pitrou wrote:
>
> Yuck. Is this some kind of joke? We've gone through the trouble of 
> unifying long and int in Python 3 and now people are proposing two 
> different Decimal types, one with fixed precision and one with 
> arbitrary precision? 


I also agree --in the long term someday-- that unification of number is 
key. Sometime
in the distant future where there is no panic, and life is good, and things
are safe.

At some point the correct approach IMHO is to calculate with decimal 
floating 
point by default. But, this is not that day.

Finding a way (as everyone is talking) to merge the use of binary floating
point, and decimal floating point calcs, that is flexible &efficient, and 
caters to the
most users (both experienced and novice alike) should be the goal... even
if there are caveats and engineering compromises. It just makes sense.

Thanks for all you are doing guys.

kind regards,
m harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/071c57cb/attachment.html>

From oscar.j.benjamin at gmail.com  Wed Mar 12 11:13:23 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 12 Mar 2014 10:13:23 +0000
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <531F8712.3010805@stoneleaf.us>
 <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
 <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com>
Message-ID: <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com>

On 12 March 2014 00:45, Andrew Barnert <abarnert at yahoo.com> wrote:
>
> On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>
>> On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote:
>>> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote:
>>>>
>>>> The bug to which I was referring involves not calling __format__ on
>>>> whatever object is passed in as str.format does - Decimal not being
>>>> float is no impediment to str.format and Decimal.__format__ accepting
>>>> the same format strings that float.__format__ does.
>>>
>>> That is not a bug.  str.__mod__ is not documented as calling an object's
>>> __format__ method, and indeed it does not.
>>
>> Instead it calls float on the object:
>>
>>>>> class A:
>> ...   def __float__(self):
>> ...     print('__float__ called')
>> ...
>>>>> '%f' % A()
>> __float__ called
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>> TypeError: nb_float should return float object
>>
>> There's no mention of that anywhere in the docs though:
>> http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting
>
> I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't.
>
> It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()".
>
> Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)?

Yes that's what I meant. The exact words used there to describe the
'f' format code are 'Floating point decimal format'. There's no
mention that it will first need to coerce to a binary float before
converting that to decimal. (Perhaps those words predate the decimal
module).

>> There's also no mention of this in the decimal docs.
>
> Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.

Because it's a relevant issue to consider when working with Decimals.
In simple cases it will work but in other cases it won't.


Oscar

From eric at trueblade.com  Wed Mar 12 12:36:50 2014
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 12 Mar 2014 07:36:50 -0400
Subject: [Python-ideas] %-formatting with Decimals
In-Reply-To: <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com>
References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com>
 <lfn280$851$1@ger.gmane.org>
 <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com>
 <531F711C.1020106@stoneleaf.us>
 <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com>
 <531F8712.3010805@stoneleaf.us>
 <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com>
 <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com>
 <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com>
Message-ID: <532046D2.6050101@trueblade.com>

On 03/12/2014 06:13 AM, Oscar Benjamin wrote:
> On 12 March 2014 00:45, Andrew Barnert <abarnert at yahoo.com> wrote:
>>
>> On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

>>> There's also no mention of this in the decimal docs.
>>
>> Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.
> 
> Because it's a relevant issue to consider when working with Decimals.
> In simple cases it will work but in other cases it won't.

In the hopefully unlikely event we do get decimal literals, I think it
would be reasonable to extend %f-formatting (and its friends e and g) to
cover decimals as well as floats.

Eric.


From mal at egenix.com  Wed Mar 12 14:11:15 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 12 Mar 2014 14:11:15 +0100
Subject: [Python-ideas] Python Numbers as Human Concept Decimal System
In-Reply-To: <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com>
References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com>	<CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com>	<CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com>	<CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com>	<CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com>	<20140310105942.GB12595@ando>	<CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com>	<20140310135342.GA10968@sleipnir.bytereef.org>	<CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com>	<CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com>	<CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com>	<20140311082449.4f0c0fdb@fsol>	<CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com>	<CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com>	<531F0286.8090306@egenix.com>
 <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com>
Message-ID: <53205CF3.1000100@egenix.com>

On 11.03.2014 14:04, Oscar Benjamin wrote:
> On 11 March 2014 12:33, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>> [...examples from finance...]
>>
>> Anyway, just throwing in some additional entropy into this
>> discussion. Probably not all that helpful, since you're already
>> converging on two possible solutions :-)
> 
> Thanks!

Thanks for the detailed response. It's good to know that things
will work out fine :-)

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 12 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          17 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               28 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From wolfgang.maier at biologie.uni-freiburg.de  Wed Mar 12 16:33:49 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Wed, 12 Mar 2014 15:33:49 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?numerical_type_combining_integer_and_flo?=
	=?utf-8?q?at/decimal=09properties_=5BWas=3A_Re=3A_Python_Numbers_a?=
	=?utf-8?q?s_Human_Concept_Decimal_System=5D?=
References: <loom.20140310T181321-988@post.gmane.org>
Message-ID: <loom.20140312T162210-499@post.gmane.org>

Wolfgang Maier <wolfgang.maier at ...> writes:

> 
> Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah:
> 
> > [My apologies for being terse, I don't have much time to follow this
> discussion right now.]
> 
> > Nick Coghlan <ncoghlan <at> gmail.com> wrote:
> >> I think users of decimal literals will just need to deal with the risk of
> unexpected rounding, as the alternatives are even more problematic.
> 
> > That is why I think we should seriously consider moving to IEEE semantics
> for a decimal literal.  Among other things:
> > - Always round the literal inputs.
> > - Supply IEEE contexts.
> > - Make Decimal64 the default.
> > - Add the missing rounding modes to sqrt() and exp().
> > - Keep the ability to create exact Decimals through the constructor when
> no context is passed.
> > - Make the Decimal constructor use the context for rounding if it is passed.
> > - ...
> 
> While I find this discussion about decimal literals extremely interesting,
> in my opinion, such a literal should have an underlying completely new
> numerical type, if it is really supposed to be for inexperienced users.
> 
> Most of the discussions right now concern rounding issues that occur after
> the decimal point, but I think an at least equally big problem is rounding
> *to the left* of it as in (using current float):
> 
> >>> 1e50 + 1000
> 1e+50
> 
> Importantly, Decimal is no cure here:
> 
> >>> Decimal(10**50) + Decimal(100)
> Decimal('1.000000000000000000000000000E+50')
> 
> (of course, you can debate context settings to make this particular example
> work, but, in general, it happens with big enough numbers.)
> 
> The solution for this example is using ints of course:
> 
> >>> 10**50 + 100
> 100000000000000000000000000000000000000000000000100
> 
> , but obviously this works only for whole numbers, so there currently is no
> built-in way to make this example work correctly:
> 
> >>> 10**50 - 9999999999999999.5
> 1e+50
> 
> (or with Decimal:
> >>> Decimal(10**50) - Decimal('9999999999999999.5')
> Decimal('1.000000000000000000000000000E+50')
> ).
> 
> If we are discussing a new number literal I would like to have it cope with
> this and my suggestion is a new numeric type that's sort of a hybrid between
> int and either Decimal or float, i.e., a type that behaves like int for
> digits left of the decimal point, but may truncate to the right.
> In pure Python, something similar (but not a literal form of course) could
> be implemented as a class that stores the left digits as an int and the
> right digits as a float/Decimal internally. Calculations involving this
> class would be slow due to the constant need of shifting digits between the
> integer?and the float part, and might be too slow for many users even when
> written in C, but its behavior would meet the expectation of inexperienced
> people better than the existing types.
> Going back to Mark Harris' initial proposal of unifying numeric types (which
> I am not trying to support in any way here), such a type would even allow to
> unify int and float since an ints could be considered a subset of the new
> type with a fractional part of zero.
> 

So, I've tried my hands on the pure Python implementation of this (code
below) to make it easier for people to play around with this idea. I found
it easier in the end to represent both the integral and the fractional part
of the new type as Python ints (and restricting the precision for the
fractional component).
In the compound object the integral part has arbitrary precision, the
fractional part a fixed precision of 28 digits by default (and in response
to Guido: this is constant even with leading zeros), but this can be changed
temporarily through a context manager (a very simple one though).

This is a very preliminary version that is incomplete still and could
certainly be optimized in many places. It's also slow, though maybe not as
slow as I thought (it's definitely fun to work with it on simple problems in
interactive mode).

Feedback is welcome,
Wolfgang

Here's the code:

"""
Test of a new numeric class PyNumber.

Usage:

Generate a new PyNumber with a fractional component of 0 (equal to an int):

>>> PyNumber(3)
PyNumber('3.0')

Generate a PyNumber with a fractional component Decimal('0.1'):

>>> PyNumber(3, Decimal('0.1'))
PyNumber('3.1')

Generate the same PyNumber by specifying fractional as int and exponent:

>>> PyNumber(3, 1, exp = 1)
PyNumber('3.1')

Generate the same PyNumber again using a string:

>>> PyNumber('3.1')
PyNumber('3.1')

Control decimal precision through context manager (default precision is 28):

>>> with Prec(3):
    PyNumber('0.333333333333333')
PyNumber('0.333')

>>> with Prec(7):
    PyNumber('1') / PyNumber('3')
PyNumber('0.3333333')

Addition, subtraction, multiplication and division work as expected, but
currently only between PyNumbers.

"""

from decimal import *
from math import log, floor

max_exp = 28
    
class Prec (object):
    def __init__ (self, prec):
        self.prec = prec

    def __enter__ (self):
        global max_exp
        self.old_prec = max_exp
        max_exp = self.prec

    def __exit__ (self, *_):
        global max_exp
        max_exp = self.old_prec
        
class PyNumber (object):
    """Numeric type for intuitive calculations.

    Supports arbitrary precision of integral component, i.e., left of the
    decimal point, and provides fixed precision (module default: 28 digits)
    for the fractional component, i.e. right of the decimal point.

    Behaves well with calculations involving huge and small numbers, for which
    float and Decimal produce rounding errors.

    Modified sum example from statistics module:
    
    >>> sum([1e50, 1.1, -1e50] * 1000) # floating point calculation gives zero.
    0.0

    >>> sum([PyNumber(10**50), PyNumber('1.1'), PyNumber(-10**50)] * 1000,
PyNumber(0))
    PyNumber('1100.0')

    """
    
    def __init__ (self, integral, fractional = 0, exp = 0):
        # PyNumber objects are composed of two integers, one unbounded for
        # representing the integral part of a number, the other restricted to
        # max_exp digits for representing the fractional component.
        
        if isinstance(integral, int):
            if integral >= 0:
                self.integral = integral
                self.sign = 1
            else:
                self.integral = -integral
                self.sign = -1
            if fractional < 0:
                raise TypeError ('negative fraction not allowed')
            if isinstance(fractional, int):
                self.fractional = fractional
                if fractional > 0:
                    if exp == 0:
                        self.exp = floor(log(fractional, 10)+1)
                    else:
                        self.exp = exp
                elif fractional == 0:
                    self.exp = 0
                else:
                    raise TypeError ('Unknown error caused by fractional.')
            elif isinstance(fractional, Decimal):
                if fractional >= 1:
                    raise ValueError ('need a fraction < 1.')
                sign, digits, exp = fractional.as_tuple()
                self.fractional = 0
                for digit in digits:
                    self.fractional = self.fractional*10 + digit
                self.exp = -exp
            else:
                raise TypeError ('Expected int or Decimal for fractional part.')
        elif isinstance(integral, str):
            x = integral.split('.')
            if len(x) > 2:
                raise ValueError('Float format string expected.')
            self.integral = int(x[0])
            if self.integral >= 0:
                self.sign = 1
            else:
                self.integral = -self.integral
                self.sign = -1
            if len(x) == 2:
                self.fractional = int(x[1])
            else:
                self.fractional = 0
            if self.fractional > 0:
                self.exp = len(x[1])
            else:
                self.exp = 0
        else:
            raise TypeError('Expected int or str as first argument.')
        if self.exp > max_exp:
            self.fractional //= 10**(self.exp-max_exp)
            self.exp = max_exp

    def __add__ (self, other):
        if self.sign == -1:
            if other.sign == 1:
                # -1 + 2 = 2 - 1
                return PyNumber(*other._sub_absolute(self))
        if other.sign == -1:
            if self.sign == 1:
                # 1 + (-2) = 1 - 2
                return PyNumber(*self._sub_absolute(other))
        if self.exp >= other.exp:
            integral, fractional = self._add_absolute(other)
        else:
            integral, fractional = other._add_absolute(self)
        return PyNumber(self.sign * integral, fractional, self.exp)

    def _add_absolute(self, other):
        # add integral components.
        integral = self.integral + other.integral
        # adjust and add fractional components
        fractional = self.fractional + other.fractional *
10**(self.exp-other.exp)
        # determine overflow of fractional sum into integral part 
        overflow = fractional // 10**self.exp
        if overflow:
            integral += overflow
            fractional -= overflow * 10**self.exp
        return integral, fractional
    
    def __sub__ (self, other):
        if self.sign == -1 and other.sign == 1:
            # -1 - 2 = -(1 + 2) 
            integral, fractional = self._add_absolute(other)
            return PyNumber(-integral, fractional, self.exp)
        if other.sign == -1 and self.sign == 1:
            # 1 - (-2) = 1 + 2
            return PyNumber(*self._add_absolute(other), exp = self.exp)
        if other.sign == -1 and self.sign == -1:
            # -1 - (-2) = 2 - 1
            return PyNumber(*other._sub_absolute(self))
        return PyNumber(*self._sub_absolute(other))

    def _sub_absolute (self, other):
        sign = 1
        # merge integral and fractional components into big ints.
        a = self.integral * 10**self.exp + self.fractional
        b = other.integral * 10**other.exp + other.fractional
        # adjust the two representations.
        if self.exp > other.exp:
            b *= 10**(self.exp-other.exp)
        elif other.exp > self.exp:
            a *= 10**(other.exp-self.exp)
        # subtract.
        diff = a-b
        if diff < 0:
            diff = -diff
            sign = -1
        # split the big int back into integral and fractional.
        shift = max(self.exp, other.exp)
        integral, fractional = divmod(diff, 10**shift)
        return sign * integral, fractional, shift
    
    def __mul__ (self, other):
        if self.exp < other.exp:
            return other * self
        # multiply the integral components.
        p1 = self.integral * other.integral
        # adjust the fractional parts.
        f1 = self.fractional
        f2 = other.fractional * 10**(self.exp-other.exp)
        # cross-multiply integral and fractional components.
        p2 = (
            self.integral * f2 * 10**self.exp +
            other.integral * f1 * 10**self.exp +
            f1 * f2)
        # determine overflow of cross-multiplication into integral part.
        shift = 2 * self.exp
        overflow = p2 // 10**shift
        if overflow > 0:
            p1 += overflow
            p2 -= overflow * 10**shift
        # truncate trailing zeros.
        while not p2 % 10:
            p2 //= 10
            shift -= 1
        sign = self.sign * other.sign
        return PyNumber(sign*p1, p2, shift)

    def __truediv__ (self, other):
        # merge integral and fractional components into big ints.
        n = self.integral * 10**self.exp + self.fractional
        d = other.integral * 10**other.exp + other.fractional
        # adjust the two representations.
        if self.exp > other.exp:
            d *= 10**(self.exp-other.exp)
        elif other.exp > self.exp:
            n *= 10**(other.exp-self.exp)
        # divide.
        integral, remainder = divmod(n, d)
        # turn remainder into fractional component.
        remainder *= 10
        shift = 1
        while remainder % d and shift < max_exp:
            remainder *= 10
            shift += 1
        fractional = remainder // d
        sign = self.sign * other.sign
        return PyNumber(sign * integral, fractional, shift)

    def __int__ (self):
        return self.integral
    
    def __float__ (self):
        return self.sign * (self.integral + self.fractional / 10**self.exp)

    def __repr__ (self):
        return "PyNumber('{0}.{1}{2}')".format(self.sign * self.integral,
'0'*(self.exp-len(str(self.fractional))), self.fractional)



From guido at python.org  Wed Mar 12 16:37:52 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Mar 2014 08:37:52 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140312T162210-499@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
Message-ID: <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>

This representation makes more sense, it is fixed point. But you can just
use a single integer and keep track of where the point should be.
On Mar 12, 2014 8:34 AM, "Wolfgang Maier" <
wolfgang.maier at biologie.uni-freiburg.de> wrote:

> Wolfgang Maier <wolfgang.maier at ...> writes:
>
> >
> > Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah:
> >
> > > [My apologies for being terse, I don't have much time to follow this
> > discussion right now.]
> >
> > > Nick Coghlan <ncoghlan <at> gmail.com> wrote:
> > >> I think users of decimal literals will just need to deal with the
> risk of
> > unexpected rounding, as the alternatives are even more problematic.
> >
> > > That is why I think we should seriously consider moving to IEEE
> semantics
> > for a decimal literal.  Among other things:
> > > - Always round the literal inputs.
> > > - Supply IEEE contexts.
> > > - Make Decimal64 the default.
> > > - Add the missing rounding modes to sqrt() and exp().
> > > - Keep the ability to create exact Decimals through the constructor
> when
> > no context is passed.
> > > - Make the Decimal constructor use the context for rounding if it is
> passed.
> > > - ...
> >
> > While I find this discussion about decimal literals extremely
> interesting,
> > in my opinion, such a literal should have an underlying completely new
> > numerical type, if it is really supposed to be for inexperienced users.
> >
> > Most of the discussions right now concern rounding issues that occur
> after
> > the decimal point, but I think an at least equally big problem is
> rounding
> > *to the left* of it as in (using current float):
> >
> > >>> 1e50 + 1000
> > 1e+50
> >
> > Importantly, Decimal is no cure here:
> >
> > >>> Decimal(10**50) + Decimal(100)
> > Decimal('1.000000000000000000000000000E+50')
> >
> > (of course, you can debate context settings to make this particular
> example
> > work, but, in general, it happens with big enough numbers.)
> >
> > The solution for this example is using ints of course:
> >
> > >>> 10**50 + 100
> > 100000000000000000000000000000000000000000000000100
> >
> > , but obviously this works only for whole numbers, so there currently is
> no
> > built-in way to make this example work correctly:
> >
> > >>> 10**50 - 9999999999999999.5
> > 1e+50
> >
> > (or with Decimal:
> > >>> Decimal(10**50) - Decimal('9999999999999999.5')
> > Decimal('1.000000000000000000000000000E+50')
> > ).
> >
> > If we are discussing a new number literal I would like to have it cope
> with
> > this and my suggestion is a new numeric type that's sort of a hybrid
> between
> > int and either Decimal or float, i.e., a type that behaves like int for
> > digits left of the decimal point, but may truncate to the right.
> > In pure Python, something similar (but not a literal form of course)
> could
> > be implemented as a class that stores the left digits as an int and the
> > right digits as a float/Decimal internally. Calculations involving this
> > class would be slow due to the constant need of shifting digits between
> the
> > integer?and the float part, and might be too slow for many users even
> when
> > written in C, but its behavior would meet the expectation of
> inexperienced
> > people better than the existing types.
> > Going back to Mark Harris' initial proposal of unifying numeric types
> (which
> > I am not trying to support in any way here), such a type would even
> allow to
> > unify int and float since an ints could be considered a subset of the new
> > type with a fractional part of zero.
> >
>
> So, I've tried my hands on the pure Python implementation of this (code
> below) to make it easier for people to play around with this idea. I found
> it easier in the end to represent both the integral and the fractional part
> of the new type as Python ints (and restricting the precision for the
> fractional component).
> In the compound object the integral part has arbitrary precision, the
> fractional part a fixed precision of 28 digits by default (and in response
> to Guido: this is constant even with leading zeros), but this can be
> changed
> temporarily through a context manager (a very simple one though).
>
> This is a very preliminary version that is incomplete still and could
> certainly be optimized in many places. It's also slow, though maybe not as
> slow as I thought (it's definitely fun to work with it on simple problems
> in
> interactive mode).
>
> Feedback is welcome,
> Wolfgang
>
> Here's the code:
>
> """
> Test of a new numeric class PyNumber.
>
> Usage:
>
> Generate a new PyNumber with a fractional component of 0 (equal to an int):
>
> >>> PyNumber(3)
> PyNumber('3.0')
>
> Generate a PyNumber with a fractional component Decimal('0.1'):
>
> >>> PyNumber(3, Decimal('0.1'))
> PyNumber('3.1')
>
> Generate the same PyNumber by specifying fractional as int and exponent:
>
> >>> PyNumber(3, 1, exp = 1)
> PyNumber('3.1')
>
> Generate the same PyNumber again using a string:
>
> >>> PyNumber('3.1')
> PyNumber('3.1')
>
> Control decimal precision through context manager (default precision is
> 28):
>
> >>> with Prec(3):
>     PyNumber('0.333333333333333')
> PyNumber('0.333')
>
> >>> with Prec(7):
>     PyNumber('1') / PyNumber('3')
> PyNumber('0.3333333')
>
> Addition, subtraction, multiplication and division work as expected, but
> currently only between PyNumbers.
>
> """
>
> from decimal import *
> from math import log, floor
>
> max_exp = 28
>
> class Prec (object):
>     def __init__ (self, prec):
>         self.prec = prec
>
>     def __enter__ (self):
>         global max_exp
>         self.old_prec = max_exp
>         max_exp = self.prec
>
>     def __exit__ (self, *_):
>         global max_exp
>         max_exp = self.old_prec
>
> class PyNumber (object):
>     """Numeric type for intuitive calculations.
>
>     Supports arbitrary precision of integral component, i.e., left of the
>     decimal point, and provides fixed precision (module default: 28 digits)
>     for the fractional component, i.e. right of the decimal point.
>
>     Behaves well with calculations involving huge and small numbers, for
> which
>     float and Decimal produce rounding errors.
>
>     Modified sum example from statistics module:
>
>     >>> sum([1e50, 1.1, -1e50] * 1000) # floating point calculation gives
> zero.
>     0.0
>
>     >>> sum([PyNumber(10**50), PyNumber('1.1'), PyNumber(-10**50)] * 1000,
> PyNumber(0))
>     PyNumber('1100.0')
>
>     """
>
>     def __init__ (self, integral, fractional = 0, exp = 0):
>         # PyNumber objects are composed of two integers, one unbounded for
>         # representing the integral part of a number, the other restricted
> to
>         # max_exp digits for representing the fractional component.
>
>         if isinstance(integral, int):
>             if integral >= 0:
>                 self.integral = integral
>                 self.sign = 1
>             else:
>                 self.integral = -integral
>                 self.sign = -1
>             if fractional < 0:
>                 raise TypeError ('negative fraction not allowed')
>             if isinstance(fractional, int):
>                 self.fractional = fractional
>                 if fractional > 0:
>                     if exp == 0:
>                         self.exp = floor(log(fractional, 10)+1)
>                     else:
>                         self.exp = exp
>                 elif fractional == 0:
>                     self.exp = 0
>                 else:
>                     raise TypeError ('Unknown error caused by fractional.')
>             elif isinstance(fractional, Decimal):
>                 if fractional >= 1:
>                     raise ValueError ('need a fraction < 1.')
>                 sign, digits, exp = fractional.as_tuple()
>                 self.fractional = 0
>                 for digit in digits:
>                     self.fractional = self.fractional*10 + digit
>                 self.exp = -exp
>             else:
>                 raise TypeError ('Expected int or Decimal for fractional
> part.')
>         elif isinstance(integral, str):
>             x = integral.split('.')
>             if len(x) > 2:
>                 raise ValueError('Float format string expected.')
>             self.integral = int(x[0])
>             if self.integral >= 0:
>                 self.sign = 1
>             else:
>                 self.integral = -self.integral
>                 self.sign = -1
>             if len(x) == 2:
>                 self.fractional = int(x[1])
>             else:
>                 self.fractional = 0
>             if self.fractional > 0:
>                 self.exp = len(x[1])
>             else:
>                 self.exp = 0
>         else:
>             raise TypeError('Expected int or str as first argument.')
>         if self.exp > max_exp:
>             self.fractional //= 10**(self.exp-max_exp)
>             self.exp = max_exp
>
>     def __add__ (self, other):
>         if self.sign == -1:
>             if other.sign == 1:
>                 # -1 + 2 = 2 - 1
>                 return PyNumber(*other._sub_absolute(self))
>         if other.sign == -1:
>             if self.sign == 1:
>                 # 1 + (-2) = 1 - 2
>                 return PyNumber(*self._sub_absolute(other))
>         if self.exp >= other.exp:
>             integral, fractional = self._add_absolute(other)
>         else:
>             integral, fractional = other._add_absolute(self)
>         return PyNumber(self.sign * integral, fractional, self.exp)
>
>     def _add_absolute(self, other):
>         # add integral components.
>         integral = self.integral + other.integral
>         # adjust and add fractional components
>         fractional = self.fractional + other.fractional *
> 10**(self.exp-other.exp)
>         # determine overflow of fractional sum into integral part
>         overflow = fractional // 10**self.exp
>         if overflow:
>             integral += overflow
>             fractional -= overflow * 10**self.exp
>         return integral, fractional
>
>     def __sub__ (self, other):
>         if self.sign == -1 and other.sign == 1:
>             # -1 - 2 = -(1 + 2)
>             integral, fractional = self._add_absolute(other)
>             return PyNumber(-integral, fractional, self.exp)
>         if other.sign == -1 and self.sign == 1:
>             # 1 - (-2) = 1 + 2
>             return PyNumber(*self._add_absolute(other), exp = self.exp)
>         if other.sign == -1 and self.sign == -1:
>             # -1 - (-2) = 2 - 1
>             return PyNumber(*other._sub_absolute(self))
>         return PyNumber(*self._sub_absolute(other))
>
>     def _sub_absolute (self, other):
>         sign = 1
>         # merge integral and fractional components into big ints.
>         a = self.integral * 10**self.exp + self.fractional
>         b = other.integral * 10**other.exp + other.fractional
>         # adjust the two representations.
>         if self.exp > other.exp:
>             b *= 10**(self.exp-other.exp)
>         elif other.exp > self.exp:
>             a *= 10**(other.exp-self.exp)
>         # subtract.
>         diff = a-b
>         if diff < 0:
>             diff = -diff
>             sign = -1
>         # split the big int back into integral and fractional.
>         shift = max(self.exp, other.exp)
>         integral, fractional = divmod(diff, 10**shift)
>         return sign * integral, fractional, shift
>
>     def __mul__ (self, other):
>         if self.exp < other.exp:
>             return other * self
>         # multiply the integral components.
>         p1 = self.integral * other.integral
>         # adjust the fractional parts.
>         f1 = self.fractional
>         f2 = other.fractional * 10**(self.exp-other.exp)
>         # cross-multiply integral and fractional components.
>         p2 = (
>             self.integral * f2 * 10**self.exp +
>             other.integral * f1 * 10**self.exp +
>             f1 * f2)
>         # determine overflow of cross-multiplication into integral part.
>         shift = 2 * self.exp
>         overflow = p2 // 10**shift
>         if overflow > 0:
>             p1 += overflow
>             p2 -= overflow * 10**shift
>         # truncate trailing zeros.
>         while not p2 % 10:
>             p2 //= 10
>             shift -= 1
>         sign = self.sign * other.sign
>         return PyNumber(sign*p1, p2, shift)
>
>     def __truediv__ (self, other):
>         # merge integral and fractional components into big ints.
>         n = self.integral * 10**self.exp + self.fractional
>         d = other.integral * 10**other.exp + other.fractional
>         # adjust the two representations.
>         if self.exp > other.exp:
>             d *= 10**(self.exp-other.exp)
>         elif other.exp > self.exp:
>             n *= 10**(other.exp-self.exp)
>         # divide.
>         integral, remainder = divmod(n, d)
>         # turn remainder into fractional component.
>         remainder *= 10
>         shift = 1
>         while remainder % d and shift < max_exp:
>             remainder *= 10
>             shift += 1
>         fractional = remainder // d
>         sign = self.sign * other.sign
>         return PyNumber(sign * integral, fractional, shift)
>
>     def __int__ (self):
>         return self.integral
>
>     def __float__ (self):
>         return self.sign * (self.integral + self.fractional / 10**self.exp)
>
>     def __repr__ (self):
>         return "PyNumber('{0}.{1}{2}')".format(self.sign * self.integral,
> '0'*(self.exp-len(str(self.fractional))), self.fractional)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/b71cbc99/attachment-0001.html>

From wolfgang.maier at biologie.uni-freiburg.de  Wed Mar 12 16:48:39 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Wed, 12 Mar 2014 15:48:39 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and
	float/decimal properties [Was: Re: Python Numbers as Human
	Concept Decimal System]
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
 <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
Message-ID: <loom.20140312T164301-427@post.gmane.org>

Guido van Rossum <guido at ...> writes:

> 
> This representation makes more sense, it is fixed point. But you can just
use a single integer and keep track of where the point should be.

Right, the main reason why I haven't tried that is that I started out with
an int for representing the integral part and a Decimal for the rest, so
from the beginning I always had two separate objects in the code.
On the other hand, I assume that a single int might also slow down the
trimming of the fractional part, which is necessary to keep its precision
fixed ?







From random832 at fastmail.us  Wed Mar 12 16:49:09 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 12 Mar 2014 11:49:09 -0400
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
 <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
Message-ID: <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com>

On Wed, Mar 12, 2014, at 11:37, Guido van Rossum wrote:
> This representation makes more sense, it is fixed point. But you can just
> use a single integer and keep track of where the point should be.

If you're keeping track of it other than implicitly, isn't that floating
point? Just of a NNNNNNN*10^-X variety instead of 1.NNNNNN*2^X.

From guido at python.org  Wed Mar 12 17:13:29 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Mar 2014 09:13:29 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <loom.20140312T164301-427@post.gmane.org>
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
 <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
 <loom.20140312T164301-427@post.gmane.org>
Message-ID: <CAP7+vJ+RS+XpnLaPhcVawLoN2PGhUii91AZwYqvbHfmhPiS5VA@mail.gmail.com>

On Wed, Mar 12, 2014 at 8:48 AM, Wolfgang Maier <
wolfgang.maier at biologie.uni-freiburg.de> wrote:

> Guido van Rossum <guido at ...> writes:
> > This representation makes more sense, it is fixed point. But you can just
> use a single integer and keep track of where the point should be.
>
> Right, the main reason why I haven't tried that is that I started out with
> an int for representing the integral part and a Decimal for the rest, so
> from the beginning I always had two separate objects in the code.
> On the other hand, I assume that a single int might also slow down the
> trimming of the fractional part, which is necessary to keep its precision
> fixed ?
>

You can't be sure without timing it, but my expectation is that unless the
number of digits is really large (like in the 1000s or more) the fewer
number of operations will always be quicker.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/cfbd5403/attachment.html>

From guido at python.org  Wed Mar 12 17:15:15 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Mar 2014 09:15:15 -0700
Subject: [Python-ideas] numerical type combining integer and
 float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal
 System]
In-Reply-To: <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com>
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
 <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
 <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com>
Message-ID: <CAP7+vJLacrRWK8ewVDo9aCMk6XdM+fvMiaexk10TD0MZfmGWjA@mail.gmail.com>

On Wed, Mar 12, 2014 at 8:49 AM, <random832 at fastmail.us> wrote:

> On Wed, Mar 12, 2014, at 11:37, Guido van Rossum wrote:
> > This representation makes more sense, it is fixed point. But you can just
> > use a single integer and keep track of where the point should be.
>
> If you're keeping track of it other than implicitly, isn't that floating
> point? Just of a NNNNNNN*10^-X variety instead of 1.NNNNNN*2^X.
>

No, his rules for how many digits to keep (everything to the left, only 28
to the right) make it fixed-point. Keeping track of the number explicitly
just makes it simpler to create variants where instead of 28 you use
another number of digits.

But the use cases for this kind of representation are pretty limited.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/7702906a/attachment.html>

From ericsnowcurrently at gmail.com  Thu Mar 13 08:43:59 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 13 Mar 2014 01:43:59 -0600
Subject: [Python-ideas] re-organizing the sys module
Message-ID: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>

The sys module is perhaps the most core module to Python and one of
the oldest.  It has grown relatively organically over the years and
its contents could use a little clean up.  Similar proposals have come
up before but without enough interest to keep them rolling.

My motivation for cleaning up sys is a combination of things:

* I'm working on a proposal that may include adding a sub-namespace to
sys (similar to what we did with sys.implementation) and removing-ish
some attributes.
* The namespace is large enough that scanning quickly through it,
particularly alphabetically a la help() or the docs, isn't very
helpful.
* Other outstanding proposals like PEP 432 may restructure underlying
APIs and having the sys module mirror the resulting APIs would help
reduce the cognitive overhead in this space.
* It would be nice to support descriptors on the module.
* The current structure does not communicate the role of the module very well.

Changing the sys module should be done carefully and as little as
possible.  The status quo has something like a +10 modifier when it
comes to sys changes.  The current API is pretty stable and
undoubtedly a lot of code relies on it.  This proposal reflects an
understanding of those considerations and addresses them in an
entirely backward compatible way (including impact on alternate Python
implementations).

At the moment I'm not proposing any big changes to sys nor any changes
at all to its API.  That can come later. <wink>  Instead, this
proposal is about setting the stage for better flexibility in making
changes to sys when the need arises.  Consequently I propose just
2.5000000000000001 relatively simple changes:

1. the existing sys module be renamed to _sys (and the stdlib be
changed accordingly where appropriate);
2. a pure Python sys module be added to the stdlib that wraps the existing API;
2.5. the new module should use the replace-itself-in-sys.modules
technique to provide a module subclass that supports descriptors.

Thoughts?

-eric

From ericsnowcurrently at gmail.com  Thu Mar 13 09:01:03 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 13 Mar 2014 02:01:03 -0600
Subject: [Python-ideas] a standard approach to
	module-replace-itself-in-sys.modules
Message-ID: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com>

Not that long ago we had a discussion about the esoteric, yet totally
supported and not going anywhere, trick of having a module replace
itself in sys.modules.  Presently there isn't any convention
surrounding the technique.  It's simply a matter of sticking something
else into sys.modules[__name__].

The problem is that there are some subtle things that can go wrong in
the import system if you aren't careful and people generally won't
know that.  (Then again the whole technique isn't exactly in the
mainstream!)  To help reduce the risk here, I propose that we provide
a helper in importlib.util and an associated convention.

The helper will be a class decorator and the convention will be to
decorate 1 class at the bottom of the module, where the class will
contain any relevant customizations.  The decorator will do the
following:

1. create a new module subclass;
2. merge the decorated class namespace into the subclass;
3. create an instance of the subclass;
4. merge the original module namespace into the instance;
5. set sys.modules[__name__] to the new instance.

For example:

@importlib.util.replace_module
class CustomModule:
    CHEDDAR = None
    @property
    def limburger (self):
        return None

There's more to it but that captures the gist. I've been exploring API
ideas, but ultimately we don't need much to get the ball rolling.  In
the meantime, I've published an unrefined but mostly working version
of replace_module() online:

https://bitbucket.org/ericsnowcurrently/odds_and_ends/src/default/replace_module.py

-eric

From solipsis at pitrou.net  Thu Mar 13 10:17:47 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 13 Mar 2014 10:17:47 +0100
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
Message-ID: <lfrt3c$62p$1@ger.gmane.org>

Le 13/03/2014 08:43, Eric Snow a ?crit :
> At the moment I'm not proposing any big changes to sys nor any changes
> at all to its API.  That can come later. <wink>  Instead, this
> proposal is about setting the stage for better flexibility in making
> changes to sys when the need arises.  Consequently I propose just
> 2.5000000000000001 relatively simple changes:
>
> 1. the existing sys module be renamed to _sys (and the stdlib be
> changed accordingly where appropriate);
> 2. a pure Python sys module be added to the stdlib that wraps the existing API;

I'm not sure what that brings until the need arises.
However, making it a pure Python sys module means it won't be 
immediately available at startup like sys currently is (i.e. it's a 
built-in module). I don't know if there are any scenarios where it may 
matter.

> 2.5. the new module should use the replace-itself-in-sys.modules
> technique to provide a module subclass that supports descriptors.

I don't think any module should be specific in that regard. Special 
cases generally make the language more complicated to learn. Also, you 
haven't explained the rationale.

Regards

Antoine.



From ncoghlan at gmail.com  Thu Mar 13 10:29:09 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 13 Mar 2014 19:29:09 +1000
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
Message-ID: <CADiSq7eTqkjHe0Q2777bHE66ccgPpMT8VD3S2OgfdWaQ5CcS9A@mail.gmail.com>

On 13 March 2014 17:43, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> At the moment I'm not proposing any big changes to sys nor any changes
> at all to its API.  That can come later. <wink>  Instead, this
> proposal is about setting the stage for better flexibility in making
> changes to sys when the need arises.  Consequently I propose just
> 2.5000000000000001 relatively simple changes:
>
> 1. the existing sys module be renamed to _sys (and the stdlib be
> changed accordingly where appropriate);
> 2. a pure Python sys module be added to the stdlib that wraps the existing API;

As Antoine suggests, we shouldn't do that until we have a concrete use
case (although I'm generally approving of the idea, there are also
benefits to having it as entirely a builtin module).

It may seem counterintuitive, but it's actually easier to sell
non-invasive prep work after already making the case for the
subsequent change :)

> 2.5. the new module should use the replace-itself-in-sys.modules
> technique to provide a module subclass that supports descriptors.

I'm generally against doing that without a clear rationale for why
defining a new object to use isn't more appropriate (we've survived
attribute -> function transitions before, like the one to using the
thread safe sys.exc_info()).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From storchaka at gmail.com  Thu Mar 13 13:04:15 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 13 Mar 2014 14:04:15 +0200
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <lfrt3c$62p$1@ger.gmane.org>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
 <lfrt3c$62p$1@ger.gmane.org>
Message-ID: <lfs6qo$tvh$1@ger.gmane.org>

13.03.14 11:17, Antoine Pitrou ???????(??):
>> 1. the existing sys module be renamed to _sys (and the stdlib be
>> changed accordingly where appropriate);
>> 2. a pure Python sys module be added to the stdlib that wraps the
>> existing API;
>
> I'm not sure what that brings until the need arises.
> However, making it a pure Python sys module means it won't be
> immediately available at startup like sys currently is (i.e. it's a
> built-in module). I don't know if there are any scenarios where it may
> matter.

Many sys attributes (argv, displayhook, excepthook, modules, path, 
path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout, 
tracebacklimit) are used implicit in builtins and in interpreter core. 
Even import system doesn't work without the sys module. So it should be 
initialized very early and finalized very late.



From brett at python.org  Thu Mar 13 13:54:16 2014
From: brett at python.org (Brett Cannon)
Date: Thu, 13 Mar 2014 08:54:16 -0400
Subject: [Python-ideas] a standard approach to
	module-replace-itself-in-sys.modules
In-Reply-To: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com>
References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com>
Message-ID: <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com>

On Thu, Mar 13, 2014 at 4:01 AM, Eric Snow <ericsnowcurrently at gmail.com>wrote:

> Not that long ago we had a discussion about the esoteric, yet totally
> supported and not going anywhere, trick of having a module replace
> itself in sys.modules.  Presently there isn't any convention
> surrounding the technique.  It's simply a matter of sticking something
> else into sys.modules[__name__].
>
> The problem is that there are some subtle things that can go wrong in
> the import system if you aren't careful and people generally won't
> know that.  (Then again the whole technique isn't exactly in the
> mainstream!)  To help reduce the risk here, I propose that we provide
> a helper in importlib.util and an associated convention.
>

Do we actually want to promote this practice by codifying support for it in
the stdlib? If you're doing this to get decorators for module attributes
then you should say this is for that goal and the solution is getting the
module swapping hack to be easier to use (and might call for a better
solution). Otherwise this proposal feels like it's trying to codify
behaviour that is rarely necessary.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/18b8eb42/attachment.html>

From wolfgang.maier at biologie.uni-freiburg.de  Thu Mar 13 18:28:49 2014
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier)
Date: Thu, 13 Mar 2014 17:28:49 +0000 (UTC)
Subject: [Python-ideas] numerical type combining integer and
	float/decimal properties [Was: Re: Python Numbers as Human
	Concept Decimal System]
References: <loom.20140310T181321-988@post.gmane.org>
 <loom.20140312T162210-499@post.gmane.org>
 <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com>
Message-ID: <loom.20140313T175526-176@post.gmane.org>

Guido van Rossum <guido at ...> writes:

> 
> This representation makes more sense, it is fixed point. But you can just
use a single integer and keep track of where the point should be.

One of the reasons I started out with two separately stored parts has to do
with my initial motivation for forking the parallel Python Numbers as Human
Concept Decimal System discussion.

That one is about a new numeric 'd' literal and the discussion now seems to
go in the direction of having this generate a new number type closely
related to the current decimal.Decimal, but somewhat simplified.

I definitely agree with Oscar Benjamin on his point that decimal.Decimal
from the stdlib is too complex (especially with Contexts) to be made
accessible through a literal constructor, but I would even go a step further
than him and say that if such a new number type gets introduced it should
have additional properties that decimal.Decimal doesn't have.

One of the concerns about a new type seems to be that it makes Python's
number system even more complex rather than simplifying it, but my
suggestion would be to make the new literal create a compound number type
behaving like my PyNumber class.

Please regard my use of two integers as an implementation detail here. My
preference for a built-in type would actually be an integer for the integral
part, but a decimal for the fractional part, it was just a bit harder to
code up quickly.
I envision this decimal part to be essentially the simplified decimal that
Oscar Benjamin advocates for, so it would have most of the power of
decimal.Decimal, but it would only be concerned with the fractional part of
numbers.

The user would never directly encounter this new decimal type, but instead
work with the compound "PyNumber" type.
I agree, this still introduces a new type, but, importantly, this new type
could be considered a superset of int. In fact, every int could be thought
of as a "PyNumber" with a fractional part of 0 (just like real numbers could
be thought of as complex with an imaginary part of 0j).
In the long run, it would thus be possible to make, e.g., division of
integers return a "PyNumber" and to replace float in conversions from an
integral number to a fractional one. Then there would be no need anymore to
expose the skeleton of number representations by things like:

>>> float(math.factorial(500))
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    float(math.factorial(500))
OverflowError: long int too large to convert to float

and there would be no more precision loss like this occurring anymore:

>>> a = 26313083693369353016721801216
>>> a + .5
2.631308369336935e+28

So, essentially I would prefer a potential new number type to improve the
user-experience with Python numbers in more ways than currently discussed
with the 'd' literal - and to come back full circle now:

Having a separate representation for the type's integral component could be
advantageous when this is expected to be combined with ints a lot.

Cheers,
Wolfgang



From barry at python.org  Thu Mar 13 21:03:53 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 13 Mar 2014 16:03:53 -0400
Subject: [Python-ideas] re-organizing the sys module
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
 <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org>
Message-ID: <20140313160353.745d9659@anarchist.wooz.org>

On Mar 13, 2014, at 02:04 PM, Serhiy Storchaka wrote:

>Many sys attributes (argv, displayhook, excepthook, modules, path,
>path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout,
>tracebacklimit) are used implicit in builtins and in interpreter core. Even
>import system doesn't work without the sys module. So it should be
>initialized very early and finalized very late.

If the C API PySys_* functions were updated to use _sys instead, hopefully
most stuff in the core would still work, right?

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/c06a142f/attachment.sig>

From barry at python.org  Thu Mar 13 21:06:29 2014
From: barry at python.org (Barry Warsaw)
Date: Thu, 13 Mar 2014 16:06:29 -0400
Subject: [Python-ideas] a standard approach to
	module-replace-itself-in-sys.modules
References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com>
 <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com>
Message-ID: <20140313160629.5a01af85@anarchist.wooz.org>

On Mar 13, 2014, at 08:54 AM, Brett Cannon wrote:

>Do we actually want to promote this practice by codifying support for it in
>the stdlib? If you're doing this to get decorators for module attributes
>then you should say this is for that goal and the solution is getting the
>module swapping hack to be easier to use (and might call for a better
>solution). Otherwise this proposal feels like it's trying to codify
>behaviour that is rarely necessary.

I don't think it necessarily has to be promoted as something we want people to
generally do, but if they're going to do it I do think it would be nice to
have a bless (i.e. stdlib) way.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/09953f01/attachment-0001.sig>

From njs at pobox.com  Fri Mar 14 02:59:14 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 01:59:14 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
Message-ID: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>

Hi all,

In the process of cleaning out old design... peculiarities... in
numpy, I happened to look into the history of attempts to add syntax
for matrix multiplication to Python, since the lack of this is (as
you'll see) at the root of various intractable problems we have. I was
pretty surprised; it turns out that even though numerical folks have
been whinging about missing this operator for ~15 years, the only two
attempts that have been made to add it were:

PEP 211, which instead adds an operator for itertools.product, aka,
"maybe we can sneak matrix multiply past Guido in some sort of...
large, wooden rabbit..."

and

PEP 225, aka "let's add 12 new operators and figure out what to do
with them later"

I'd have rejected these too! So I thought, maybe we should try the
radical tactic of writing down what we actually want, carefully
explaining why we want it, and then asking for it. And at least this
way, if it gets rejected, we'll know that it was rejected for the
right reasons...

You'll notice that this draft is rather more developed than the
average first-round PEP posting, because it's already been the rounds
of all the various numerical package mailing lists to build consensus;
no point in asking for the wrong thing. Don't let that slow you down,
though. I think what we have here is fairly convincing and covers a
lot of the design space (at least it convinced me, which I wasn't sure
of at the start), but I'm still totally open to changing anything here
based on comments and feedback. AFAICT the numerical community would
walk over hot coals if there were an infix matrix multiplication
operator on the other side. (BTW, since this is python-ideas -- have
you considered adding hot coals to python 3? It might do wonders for
uptake.) ...Anyway, the point is, I'm sure I can wrangle them into
accepting any useful suggestions or other changes deemed necessary by
the broader Python community.

-n

--- [begin draft PEP -- monospace font recommended] ---

PEP: XXXX
Title: Dedicated infix operators for matrix multiplication and matrix power
Version: $Revision$
Last-Modified: $Date$
Author: Nathaniel J. Smith <njs at pobox.com>
Status: Draft
Type: Standards Track
Python-Version: 3.5
Content-Type: text/x-rst
Created: 20-Feb-2014
Post-History:

Abstract
========

This PEP proposes two new binary operators dedicated to matrix
multiplication and matrix power, spelled ``@`` and ``@@``
respectively.  (Mnemonic: ``@`` is ``*`` for mATrices.)


Specification
=============

Two new binary operators are added to the Python language, together
with corresponding in-place versions:

=======  ========================= ===============================
 Op      Precedence/associativity     Methods
=======  ========================= ===============================
``@``    Same as ``*``             ``__matmul__``, ``__rmatmul__``
``@@``   Same as ``**``            ``__matpow__``, ``__rmatpow__``
``@=``   n/a                       ``__imatmul__``
``@@=``  n/a                       ``__imatpow__``
=======  ========================= ===============================

No implementations of these methods are added to the builtin or
standard library types.  However, a number of projects have reached
consensus on the recommended semantics for these operations; see
`Intended usage details`_ below.


Motivation
==========

Executive summary
-----------------

In numerical code, there are two important operations which compete
for use of Python's ``*`` operator: elementwise multiplication, and
matrix multiplication.  In the nearly twenty years since the Numeric
library was first proposed, there have been many attempts to resolve
this tension [#hugunin]_; none have been really satisfactory.
Currently, most numerical Python code uses ``*`` for elementwise
multiplication, and function/method syntax for matrix multiplication;
however, this leads to ugly and unreadable code in common
circumstances.  The problem is bad enough that significant amounts of
code continue to use the opposite convention (which has the virtue of
producing ugly and unreadable code in *different* circumstances), and
this API fragmentation across codebases then creates yet more
problems.  There does not seem to be any *good* solution to the
problem of designing a numerical API within current Python syntax --
only a landscape of options that are bad in different ways.  The
minimal change to Python syntax which is sufficient to resolve these
problems is the addition of a single new infix operator for matrix
multiplication.

Matrix multiplication has a singular combination of features which
distinguish it from other binary operations, which together provide a
uniquely compelling case for the addition of a dedicated infix
operator:

* Just as for the existing numerical operators, there exists a vast
  body of prior art supporting the use of infix notation for matrix
  multiplication across all fields of mathematics, science, and
  engineering; ``@`` harmoniously fills a hole in Python's existing
  operator system.

* ``@`` greatly clarifies real-world code.

* ``@`` provides a smoother onramp for less experienced users, who are
  particularly harmed by hard-to-read code and API fragmentation.

* ``@`` benefits a substantial and growing portion of the Python user
  community.

* ``@`` will be used frequently -- in fact, evidence suggests it may
  be used more frequently than ``//`` or the bitwise operators.

* ``@`` allows the Python numerical community to reduce fragmentation,
  and finally standardize on a single consensus duck type for all
  numerical array objects.

And, given the existence of ``@``, it makes more sense than not to
have ``@@``, ``@=``, and ``@@=``, so they are added as well.


Background: What's wrong with the status quo?
---------------------------------------------

When we crunch numbers on a computer, we usually have lots and lots of
numbers to deal with.  Trying to deal with them one at a time is
cumbersome and slow -- especially when using an interpreted language.
Instead, we want the ability to write down simple operations that
apply to large collections of numbers all at once.  The *n-dimensional
array* is the basic object that all popular numeric computing
environments use to make this possible.  Python has several libraries
that provide such arrays, with numpy being at present the most
prominent.

When working with n-dimensional arrays, there are two different ways
we might want to define multiplication.  One is elementwise
multiplication::

  [[1, 2],     [[11, 12],     [[1 * 11, 2 * 12],
   [3, 4]]  x   [13, 14]]  =   [3 * 13, 4 * 14]]

and the other is `matrix multiplication`_:

.. _matrix multiplication: https://en.wikipedia.org/wiki/Matrix_multiplication

::

  [[1, 2],     [[11, 12],     [[1 * 11 + 2 * 13, 1 * 12 + 2 * 14],
   [3, 4]]  x   [13, 14]]  =   [3 * 11 + 4 * 13, 3 * 12 + 4 * 14]]

Elementwise multiplication is useful because it lets us easily and
quickly perform many multiplications on a large collection of values,
without writing a slow and cumbersome ``for`` loop.  And this works as
part of a very general schema: when using the array objects provided
by numpy or other numerical libraries, all Python operators work
elementwise on arrays of all dimensionalities.  The result is that one
can write functions using straightforward code like ``a * b + c / d``,
treating the variables as if they were simple values, but then
immediately use this function to efficiently perform this calculation
on large collections of values, while keeping them organized using
whatever arbitrarily complex array layout works best for the problem
at hand.

Matrix multiplication is more of a special case.  It's only defined on
2d arrays (also known as "matrices"), and multiplication is the only
operation that has a meaningful "matrix" version -- "matrix addition"
is the same as elementwise addition; there is no such thing as "matrix
bitwise-or" or "matrix floordiv"; "matrix division" can be defined but
is not very useful, etc.  However, matrix multiplication is still used
very heavily across all numerical application areas; mathematically,
it's one of the most fundamental operations there is.

Because Python syntax currently allows for only a single
multiplication operator ``*``, libraries providing array-like objects
must decide: either use ``*`` for elementwise multiplication, or use
``*`` for matrix multiplication.  And, unfortunately, it turns out
that when doing general-purpose number crunching, both operations are
used frequently, and there are major advantages to using infix rather
than function call syntax in both cases.  Thus it is not at all clear
which convention is optimal, or even acceptable; often it varies on a
case-by-case basis.

Nonetheless, network effects mean that it is very important that we
pick *just one* convention.  In numpy, for example, it is technically
possible to switch between the conventions, because numpy provides two
different types with different ``__mul__`` methods.  For
``numpy.ndarray`` objects, ``*`` performs elementwise multiplication,
and matrix multiplication must use a function call (``numpy.dot``).
For ``numpy.matrix`` objects, ``*`` performs matrix multiplication,
and elementwise multiplication requires function syntax.  Writing code
using ``numpy.ndarray`` works fine.  Writing code using
``numpy.matrix`` also works fine.  But trouble begins as soon as we
try to integrate these two pieces of code together.  Code that expects
an ``ndarray`` and gets a ``matrix``, or vice-versa, may crash or
return incorrect results.  Keeping track of which functions expect
which types as inputs, and return which types as outputs, and then
converting back and forth all the time, is incredibly cumbersome and
impossible to get right at any scale.  Functions that defensively try
to handle both types as input and DTRT, find themselves floundering
into a swamp of ``isinstance`` and ``if`` statements.

PEP 238 split ``/`` into two operators: ``/`` and ``//``.  Imagine the
chaos that would have resulted if it had instead split ``int`` into
two types: ``classic_int``, whose ``__div__`` implemented floor
division, and ``new_int``, whose ``__div__`` implemented true
division.  This, in a more limited way, is the situation that Python
number-crunchers currently find themselves in.

In practice, the vast majority of projects have settled on the
convention of using ``*`` for elementwise multiplication, and function
call syntax for matrix multiplication (e.g., using ``numpy.ndarray``
instead of ``numpy.matrix``).  This reduces the problems caused by API
fragmentation, but it doesn't eliminate them.  The strong desire to
use infix notation for matrix multiplication has caused a number of
specialized array libraries to continue to use the opposing convention
(e.g., scipy.sparse, pyoperators, pyviennacl) despite the problems
this causes, and ``numpy.matrix`` itself still gets used in
introductory programming courses, often appears in StackOverflow
answers, and so forth.  Well-written libraries thus must continue to
be prepared to deal with both types of objects, and, of course, are
also stuck using unpleasant funcall syntax for matrix multiplication.
After nearly two decades of trying, the numerical community has still
not found any way to resolve these problems within the constraints of
current Python syntax (see `Rejected alternatives to adding a new
operator`_ below).

This PEP proposes the minimum effective change to Python syntax that
will allow us to drain this swamp.  It splits ``*`` into two
operators, just as was done for ``/``: ``*`` for elementwise
multiplication, and ``@`` for matrix multiplication.  (Why not the
reverse?  Because this way is compatible with the existing consensus,
and because it gives us a consistent rule that all the built-in
numeric operators also apply in an elementwise manner to arrays; the
reverse convention would lead to more special cases.)

So that's why matrix multiplication doesn't and can't just use ``*``.
Now, in the the rest of this section, we'll explain why it nonetheless
meets the high bar for adding a new operator.


Why should matrix multiplication be infix?
------------------------------------------

Right now, most numerical code in Python uses syntax like
``numpy.dot(a, b)`` or ``a.dot(b)`` to perform matrix multiplication.
This obviously works, so why do people make such a fuss about it, even
to the point of creating API fragmentation and compatibility swamps?

Matrix multiplication shares two features with ordinary arithmetic
operations like addition and multiplication on numbers: (a) it is used
very heavily in numerical programs -- often multiple times per line of
code -- and (b) it has an ancient and universally adopted tradition of
being written using infix syntax.  This is because, for typical
formulas, this notation is dramatically more readable than any
function call syntax.  Here's an example to demonstrate:

One of the most useful tools for testing a statistical hypothesis is
the linear hypothesis test for OLS regression models.  It doesn't
really matter what all those words I just said mean; if we find
ourselves having to implement this thing, what we'll do is look up
some textbook or paper on it, and encounter many mathematical formulas
that look like:

.. math::

    S = (H \beta - r)^T (H V H^T)^{-1} (H \beta - r)

Here the various variables are all vectors or matrices (details for
the curious: [#lht]_).

Now we need to write code to perform this calculation. In current
numpy, matrix multiplication can be performed using either the
function or method call syntax. Neither provides a particularly
readable translation of the formula::

    import numpy as np
    from numpy.linalg import inv, solve

    # Using dot function:
    S = np.dot((np.dot(H, beta) - r).T,
               np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r))

    # Using dot method:
    S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)

With the ``@`` operator, the direct translation of the above formula
becomes::

    S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)

Notice that there is now a transparent, 1-to-1 mapping between the
symbols in the original formula and the code that implements it.

Of course, an experienced programmer will probably notice that this is
not the best way to compute this expression.  The repeated computation
of :math:`H \beta - r` should perhaps be factored out; and,
expressions of the form ``dot(inv(A), B)`` should almost always be
replaced by the more numerically stable ``solve(A, B)``.  When using
``@``, performing these two refactorings gives us::

    # Version 1 (as above)
    S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)

    # Version 2
    trans_coef = H @ beta - r
    S = trans_coef.T @ inv(H @ V @ H.T) @ trans_coef

    # Version 3
    S = trans_coef.T @ solve(H @ V @ H.T, trans_coef)

Notice that when comparing between each pair of steps, it's very easy
to see exactly what was changed.  If we apply the equivalent
transformations to the code using the .dot method, then the changes
are much harder to read out or verify for correctness::

    # Version 1 (as above)
    S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r)

    # Version 2
    trans_coef = H.dot(beta) - r
    S = trans_coef.T.dot(inv(H.dot(V).dot(H.T))).dot(trans_coef)

    # Version 3
    S = trans_coef.T.dot(solve(H.dot(V).dot(H.T)), trans_coef)

Readability counts!  The statements using ``@`` are shorter, contain
more whitespace, can be directly and easily compared both to each
other and to the textbook formula, and contain only meaningful
parentheses.  This last point is particularly important for
readability: when using function-call syntax, the required parentheses
on every operation create visual clutter that makes it very difficult
to parse out the overall structure of the formula by eye, even for a
relatively simple formula like this one.  Eyes are terrible at parsing
non-regular languages.  I made and caught many errors while trying to
write out the 'dot' formulas above.  I know they still contain at
least one error, maybe more.  (Exercise: find it.  Or them.)  The
``@`` examples, by contrast, are not only correct, they're obviously
correct at a glance.

If we are even more sophisticated programmers, and writing code that
we expect to be reused, then considerations of speed or numerical
accuracy might lead us to prefer some particular order of evaluation.
Because ``@`` makes it possible to omit irrelevant parentheses, we can
be certain that if we *do* write something like ``(H @ V) @ H.T``,
then our readers will know that the parentheses must have been added
intentionally to accomplish some meaningful purpose.  In the ``dot``
examples, it's impossible to know which nesting decisions are
important, and which are arbitrary.

Infix ``@`` dramatically improves matrix code usability at all stages
of programmer interaction.


Transparent syntax is especially crucial for non-expert programmers
-------------------------------------------------------------------

A large proportion of scientific code is written by people who are
experts in their domain, but are not experts in programming.  And
there are many university courses run each year with titles like "Data
analysis for social scientists" which assume no programming
background, and teach some combination of mathematical techniques,
introduction to programming, and the use of programming to implement
these mathematical techniques, all within a 10-15 week period.  These
courses are more and more often being taught in Python rather than
special-purpose languages like R or Matlab.

For these kinds of users, whose programming knowledge is fragile, the
existence of a transparent mapping between formulas and code often
means the difference between succeeding and failing to write that code
at all.  This is so important that such classes often use the
``numpy.matrix`` type which defines ``*`` to mean matrix
multiplication, even though this type is buggy and heavily
disrecommended by the rest of the numpy community for the
fragmentation that it causes.  This pedagogical use case is, in fact,
the *only* reason ``numpy.matrix`` remains a supported part of numpy.
Adding ``@`` will benefit both beginning and advanced users with
better syntax; and furthermore, it will allow both groups to
standardize on the same notation from the start, providing a smoother
on-ramp to expertise.


But isn't matrix multiplication a pretty niche requirement?
-----------------------------------------------------------

The world is full of continuous data, and computers are increasingly
called upon to work with it in sophisticated ways.  Arrays are the
lingua franca of finance, machine learning, 3d graphics, computer
vision, robotics, operations research, econometrics, meteorology,
computational linguistics, recommendation systems, neuroscience,
astronomy, bioinformatics (including genetics, cancer research, drug
discovery, etc.), physics engines, quantum mechanics, geophysics,
network analysis, and many other application areas.  In most or all of
these areas, Python is rapidly becoming a dominant player, in large
part because of its ability to elegantly mix traditional discrete data
structures (hash tables, strings, etc.) on an equal footing with
modern numerical data types and algorithms.

We all live in our own little sub-communities, so some Python users
may be surprised to realize the sheer extent to which Python is used
for number crunching -- especially since much of this particular
sub-community's activity occurs outside of traditional Python/FOSS
channels.  So, to give some rough idea of just how many numerical
Python programmers are actually out there, here are two numbers: In
2013, there were 7 international conferences organized specifically on
numerical Python [#scipy-conf]_ [#pydata-conf]_.  At PyCon 2014, ~20%
of the tutorials appear to involve the use of matrices
[#pycon-tutorials]_.

To quantify this further, we used Github's "search" function to look
at what modules are actually imported across a wide range of
real-world code (i.e., all the code on Github).  We checked for
imports of several popular stdlib modules, a variety of numerically
oriented modules, and various other extremely high-profile modules
like django and lxml (the latter of which is the #1 most downloaded
package on PyPI).  Starred lines indicate packages which export array-
or matrix-like objects which will adopt ``@`` if this PEP is
approved::

    Count of Python source files on Github matching given search terms
                     (as of 2014-04-10, ~21:00 UTC)
    ================ ==========  ===============  =======  ===========
    module           "import X"  "from X import"    total  total/numpy
    ================ ==========  ===============  =======  ===========
    sys                 2374638            63301  2437939         5.85
    os                  1971515            37571  2009086         4.82
    re                  1294651             8358  1303009         3.12
    numpy ************** 337916 ********** 79065 * 416981 ******* 1.00
    warnings             298195            73150   371345         0.89
    subprocess           281290            63644   344934         0.83
    django                62795           219302   282097         0.68
    math                 200084            81903   281987         0.68
    threading            212302            45423   257725         0.62
    pickle+cPickle       215349            22672   238021         0.57
    matplotlib           119054            27859   146913         0.35
    sqlalchemy            29842            82850   112692         0.27
    pylab *************** 36754 ********** 41063 ** 77817 ******* 0.19
    scipy *************** 40829 ********** 28263 ** 69092 ******* 0.17
    lxml                  19026            38061    57087         0.14
    zlib                  40486             6623    47109         0.11
    multiprocessing       25247            19850    45097         0.11
    requests              30896              560    31456         0.08
    jinja2                 8057            24047    32104         0.08
    twisted               13858             6404    20262         0.05
    gevent                11309             8529    19838         0.05
    pandas ************** 14923 *********** 4005 ** 18928 ******* 0.05
    sympy                  2779             9537    12316         0.03
    theano *************** 3654 *********** 1828 *** 5482 ******* 0.01
    ================ ==========  ===============  =======  ===========

These numbers should be taken with several grains of salt (see
footnote for discussion: [#github-details]_), but, to the extent they
can be trusted, they suggest that ``numpy`` might be the single
most-imported non-stdlib module in the entire Pythonverse; it's even
more-imported than such stdlib stalwarts as ``subprocess``, ``math``,
``pickle``, and ``threading``.  And numpy users represent only a
subset of the broader numerical community that will benefit from the
``@`` operator.  Matrices may once have been a niche data type
restricted to Fortran programs running in university labs and military
clusters, but those days are long gone.  Number crunching is a
mainstream part of modern Python usage.

In addition, there is some precedence for adding an infix operator to
handle a more-specialized arithmetic operation: the floor division
operator ``//``, like the bitwise operators, is very useful under
certain circumstances when performing exact calculations on discrete
values.  But it seems likely that there are many Python programmers
who have never had reason to use ``//`` (or, for that matter, the
bitwise operators).  ``@`` is no more niche than ``//``.


So ``@`` is good for matrix formulas, but how common are those really?
----------------------------------------------------------------------

We've seen that ``@`` makes matrix formulas dramatically easier to
work with for both experts and non-experts, that matrix formulas
appear in many important applications, and that numerical libraries
like numpy are used by a substantial proportion of Python's user base.
But numerical libraries aren't just about matrix formulas, and being
important doesn't necessarily mean taking up a lot of code: if matrix
formulas only occured in one or two places in the average
numerically-oriented project, then it still wouldn't be worth adding a
new operator.  So how common is matrix multiplication, really?

When the going gets tough, the tough get empirical.  To get a rough
estimate of how useful the ``@`` operator will be, the table below
shows the rate at which different Python operators are actually used
in the stdlib, and also in two high-profile numerical packages -- the
scikit-learn machine learning library, and the nipy neuroimaging
library -- normalized by source lines of code (SLOC).  Rows are sorted
by the 'combined' column, which pools all three code bases together.
The combined column is thus strongly weighted towards the stdlib,
which is much larger than both projects put together (stdlib: 411575
SLOC, scikit-learn: 50924 SLOC, nipy: 37078 SLOC). [#sloc-details]_

The ``dot`` row (marked ``******``) counts how common matrix multiply
operations are in each codebase.

::

    ====  ======  ============  ====  ========
      op  stdlib  scikit-learn  nipy  combined
    ====  ======  ============  ====  ========
       =    2969          5536  4932      3376 / 10,000 SLOC
       -     218           444   496       261
       +     224           201   348       231
      ==     177           248   334       196
       *     156           284   465       192
       %     121           114   107       119
      **      59           111   118        68
      !=      40            56    74        44
       /      18           121   183        41
       >      29            70   110        39
      +=      34            61    67        39
       <      32            62    76        38
      >=      19            17    17        18
      <=      18            27    12        18
     dot ***** 0 ********** 99 ** 74 ****** 16
       |      18             1     2        15
       &      14             0     6        12
      <<      10             1     1         8
      //       9             9     1         8
      -=       5            21    14         8
      *=       2            19    22         5
      /=       0            23    16         4
      >>       4             0     0         3
       ^       3             0     0         3
       ~       2             4     5         2
      |=       3             0     0         2
      &=       1             0     0         1
     //=       1             0     0         1
      ^=       1             0     0         0
     **=       0             2     0         0
      %=       0             0     0         0
     <<=       0             0     0         0
     >>=       0             0     0         0
    ====  ======  ============  ====  ========

These two numerical packages alone contain ~780 uses of matrix
multiplication.  Within these packages, matrix multiplication is used
more heavily than most comparison operators (``<`` ``!=`` ``<=``
``>=``).  Even when we dilute these counts by including the stdlib
into our comparisons, matrix multiplication is still used more often
in total than any of the bitwise operators, and 2x as often as ``//``.
This is true even though the stdlib, which contains a fair amount of
integer arithmetic and no matrix operations, makes up more than 80% of
the combined code base.

By coincidence, the numeric libraries make up approximately the same
proportion of the 'combined' codebase as numeric tutorials make up of
PyCon 2014's tutorial schedule, which suggests that the 'combined'
column may not be *wildly* unrepresentative of new Python code in
general.  While it's impossible to know for certain, from this data it
seems entirely possible that across all Python code currently being
written, matrix multiplication is already used more often than ``//``
and the bitwise operations.


But isn't it weird to add an operator with no stdlib uses?
----------------------------------------------------------

It's certainly unusual (though ``Ellipsis`` was also added without any
stdlib uses).  But the important thing is whether a change will
benefit users, not where the software is being downloaded from.  It's
clear from the above that ``@`` will be used, and used heavily.  And
this PEP provides the critical piece that will allow the Python
numerical community to finally reach consensus on a standard duck type
for all array-like objects, which is a necessary precondition to ever
adding a numerical array type to the stdlib.


Matrix power and in-place operators
-----------------------------------

The primary motivation for this PEP is ``@``; the other proposed
operators don't have nearly as much impact.  The matrix power operator
``@@`` is useful and well-defined, but not really necessary.  It is
still included, though, for consistency: if we have an ``@`` that is
analogous to ``*``, then it would be weird and surprising to *not*
have an ``@@`` that is analogous to ``**``.  Similarly, the in-place
operators ``@=`` and ``@@=`` provide limited value -- it's more common
to write ``a = (b @ a)`` than it is to write ``a = (a @ b)``, and
in-place matrix operations still generally have to allocate
substantial temporary storage -- but they are included for
completeness and symmetry.


Compatibility considerations
============================

Currently, the only legal use of the ``@`` token in Python code is at
statement beginning in decorators.  The new operators are all infix;
the one place they can never occur is at statement beginning.
Therefore, no existing code will be broken by the addition of these
operators, and there is no possible parsing ambiguity between
decorator-@ and the new operators.

Another important kind of compatibility is the mental cost paid by
users to update their understanding of the Python language after this
change, particularly for users who do not work with matrices and thus
do not benefit.  Here again, ``@`` has minimal impact: even
comprehensive tutorials and references will only need to add a
sentence or two to fully document this PEP's changes for a
non-numerical audience.


Intended usage details
======================

This section is informative, rather than normative -- it documents the
consensus of a number of libraries that provide array- or matrix-like
objects on how the ``@`` and ``@@`` operators will be implemented.

This section uses the numpy terminology for describing arbitrary
multidimensional arrays of data, because it is a superset of all other
commonly used models.  In this model, the *shape* of any array is
represented by a tuple of integers.  Because matrices are
two-dimensional, they have len(shape) == 2, while 1d vectors have
len(shape) == 1, and scalars have shape == (), i.e., they are "0
dimensional".  Any array contains prod(shape) total entries.  Notice
that `prod(()) == 1`_ (for the same reason that sum(()) == 0); scalars
are just an ordinary kind of array, not a special case.  Notice also
that we distinguish between a single scalar value (shape == (),
analogous to ``1``), a vector containing only a single entry (shape ==
(1,), analogous to ``[1]``), a matrix containing only a single entry
(shape == (1, 1), analogous to ``[[1]]``), etc., so the dimensionality
of any array is always well-defined.  Other libraries with more
restricted representations (e.g., those that support 2d arrays only)
might implement only a subset of the functionality described here.

.. _prod(()) == 1: https://en.wikipedia.org/wiki/Empty_product

Semantics
---------

The recommended semantics for ``@`` for different inputs are:

* 2d inputs are conventional matrices, and so the semantics are
  obvious: we apply conventional matrix multiplication.  If we write
  ``arr(2, 3)`` to represent an arbitrary 2x3 array, then ``arr(3, 4)
  @ arr(4, 5)`` returns an array with shape (3, 5).

* 1d vector inputs are promoted to 2d by prepending or appending a '1'
  to the shape, the operation is performed, and then the added
  dimension is removed from the output.  The 1 is always added on the
  "outside" of the shape: prepended for left arguments, and appended
  for right arguments.  The result is that matrix @ vector and vector
  @ matrix are both legal (assuming compatible shapes), and both
  return 1d vectors; vector @ vector returns a scalar.  This is
  clearer with examples.

  * ``arr(2, 3) @ arr(3, 1)`` is a regular matrix product, and returns
    an array with shape (2, 1), i.e., a column vector.

  * ``arr(2, 3) @ arr(3)`` performs the same computation as the
    previous (i.e., treats the 1d vector as a matrix containing a
    single *column*, shape = (3, 1)), but returns the result with
    shape (2,), i.e., a 1d vector.

  * ``arr(1, 3) @ arr(3, 2)`` is a regular matrix product, and returns
    an array with shape (1, 2), i.e., a row vector.

  * ``arr(3) @ arr(3, 2)`` performs the same computation as the
    previous (i.e., treats the 1d vector as a matrix containing a
    single *row*, shape = (1, 3)), but returns the result with shape
    (2,), i.e., a 1d vector.

  * ``arr(1, 3) @ arr(3, 1)`` is a regular matrix product, and returns
    an array with shape (1, 1), i.e., a single value in matrix form.

  * ``arr(3) @ arr(3)`` performs the same computation as the
    previous, but returns the result with shape (), i.e., a single
    scalar value, not in matrix form.  So this is the standard inner
    product on vectors.

  An infelicity of this definition for 1d vectors is that it makes
  ``@`` non-associative in some cases (``(Mat1 @ vec) @ Mat2`` !=
  ``Mat1 @ (vec @ Mat2)``).  But this seems to be a case where
  practicality beats purity: non-associativity only arises for strange
  expressions that would never be written in practice; if they are
  written anyway then there is a consistent rule for understanding
  what will happen (``Mat1 @ vec @ Mat2`` is parsed as ``(Mat1 @ vec)
  @ Mat2``, just like ``a - b - c``); and, not supporting 1d vectors
  would rule out many important use cases that do arise very commonly
  in practice.  No-one wants to explain to new users why to solve the
  simplest linear system in the obvious way, they have to type
  ``(inv(A) @ b[:, np.newaxis]).flatten()`` instead of ``inv(A) @ b``,
  or perform an ordinary least-squares regression by typing
  ``solve(X.T @ X, X @ y[:, np.newaxis]).flatten()`` instead of
  ``solve(X.T @ X, X @ y)``.  No-one wants to type ``(a[np.newaxis, :]
  @ b[:, np.newaxis])[0, 0]`` instead of ``a @ b`` every time they
  compute an inner product, or ``(a[np.newaxis, :] @ Mat @ b[:,
  np.newaxis])[0, 0]`` for general quadratic forms instead of ``a @
  Mat @ b``.  In addition, sage and sympy (see below) use these
  non-associative semantics with an infix matrix multiplication
  operator (they use ``*``), and they report that they haven't
  experienced any problems caused by it.

* For inputs with more than 2 dimensions, we treat the last two
  dimensions as being the dimensions of the matrices to multiply, and
  'broadcast' across the other dimensions.  This provides a convenient
  way to quickly compute many matrix products in a single operation.
  For example, ``arr(10, 2, 3) @ arr(10, 3, 4)`` performs 10 separate
  matrix multiplies, each of which multiplies a 2x3 and a 3x4 matrix
  to produce a 2x4 matrix, and then returns the 10 resulting matrices
  together in an array with shape (10, 2, 4).  The intuition here is
  that we treat these 3d arrays of numbers as if they were 1d arrays
  *of matrices*, and then apply matrix multiplication in an
  elementwise manner, where now each 'element' is a whole matrix.
  Note that broadcasting is not limited to perfectly aligned arrays;
  in more complicated cases, it allows several simple but powerful
  tricks for controlling how arrays are aligned with each other; see
  [#broadcasting]_ for details.  (In particular, it turns out that
  when broadcasting is taken into account, the standard scalar *
  matrix product is a special case of the elementwise multiplication
  operator ``*``.)

  If one operand is >2d, and another operand is 1d, then the above
  rules apply unchanged, with 1d->2d promotion performed before
  broadcasting.  E.g., ``arr(10, 2, 3) @ arr(3)`` first promotes to
  ``arr(10, 2, 3) @ arr(3, 1)``, then broadcasts the right argument to
  create the aligned operation ``arr(10, 2, 3) @ arr(10, 3, 1)``,
  multiplies to get an array with shape (10, 2, 1), and finally
  removes the added dimension, returning an array with shape (10, 2).
  Similarly, ``arr(2) @ arr(10, 2, 3)`` produces an intermediate array
  with shape (10, 1, 3), and a final array with shape (10, 3).

* 0d (scalar) inputs raise an error.  Scalar * matrix multiplication
  is a mathematically and algorithmically distinct operation from
  matrix @ matrix multiplication, and is already covered by the
  elementwise ``*`` operator.  Allowing scalar @ matrix would thus
  both require an unnecessary special case, and violate TOOWTDI.

The recommended semantics for ``@@`` are::

    def __matpow__(self, n):
        if not isinstance(n, numbers.Integral):
            raise TypeError("@@ not implemented for fractional powers")
        if n == 0:
            return identity_matrix_with_shape(self.shape)
        elif n < 0:
            return inverse(self) @ (self @@ (n + 1))
        else:
            return self @ (self @@ (n - 1))

(Of course we expect that much more efficient implementations will be
used in practice.)  Notice that if given an appropriate definition of
``identity_matrix_with_shape``, then this definition will
automatically handle >2d arrays appropriately.  Notice also that with
this definition, ``vector @@ 2`` gives the squared Euclidean length of
the vector, a commonly used value.  Also, while it is rarely useful to
explicitly compute inverses or other negative powers in standard
immediate-mode dense matrix code, these computations are natural when
doing symbolic or deferred-mode computations (as in e.g. sympy,
theano, numba, numexpr); therefore, negative powers are fully
supported.  Fractional powers, though, bring in variety of
`mathematical complications`_, so we leave it to individual projects
to decide whether they want to try to define some reasonable semantics
for fractional inputs.

.. _`mathematical complications`:
https://en.wikipedia.org/wiki/Square_root_of_a_matrix


Adoption
--------

We group existing Python projects which provide array- or matrix-like
types based on what API they currently use for elementwise and matrix
multiplication.

**Projects which currently use * for *elementwise* multiplication, and
function/method calls for *matrix* multiplication:**

The developers of the following projects have expressed an intention
to implement ``@`` and ``@@`` on their array-like types using the
above semantics:

* numpy
* pandas
* blaze
* theano

The following projects have been alerted to the existence of the PEP,
but it's not yet known what they plan to do if it's accepted.  We
don't anticipate that they'll have any objections, though, since
everything proposed here is consistent with how they already do
things:

* pycuda
* panda3d

**Projects which currently use * for *matrix* multiplication, and
function/method calls for *elementwise* multiplication:**

The following projects have expressed an intention, if this PEP is
accepted, to migrate from their current API to the elementwise-``*``,
matmul-``@`` convention (i.e., this is a list of projects whose API
fragmentation will probably be eliminated if this PEP is accepted):

* numpy (``numpy.matrix``)
* scipy.sparse
* pyoperators
* pyviennacl

The following projects have been alerted to the existence of the PEP,
but it's not known what they plan to do if it's accepted (i.e., this
is a list of projects whose API fragmentation may or may not be
eliminated if this PEP is accepted):

* cvxopt

**Projects which currently use * for *matrix* multiplication, and
which do not implement elementwise multiplication at all:**

There are several projects which implement matrix types, but from a
very different perspective than the numerical libraries discussed
above.  These projects focus on computational methods for analyzing
matrices in the sense of abstract mathematical objects (i.e., linear
maps over free modules over rings), rather than as big bags full of
numbers that need crunching.  And it turns out that from the abstract
math point of view, there isn't much use for elementwise operations in
the first place; as discussed in the Background section above,
elementwise operations are motivated by the bag-of-numbers approach.
The different goals of these projects mean that they don't encounter
the basic problem that this PEP exists to address, making it mostly
irrelevant to them; while they appear superficially similar, they're
actually doing something quite different.  They use ``*`` for matrix
multiplication (and for group actions, and so forth), and if this PEP
is accepted, their expressed intention is to continue doing so, while
perhaps adding ``@`` and ``@@`` on matrices as aliases for ``*`` and
``**``:

* sympy
* sage

If you know of any actively maintained Python libraries which provide
an interface for working with numerical arrays or matrices, and which
are not listed above, then please let the PEP author know:
njs at pobox.com


Rationale for specification details
===================================

Choice of operator
------------------

Why ``@`` instead of some other punctuation symbol? It doesn't matter
much, and there isn't any consensus across other programming languages
about how this operator should be named [#matmul-other-langs]_, but
``@`` has a few advantages:

* ``@`` is a friendly character that Pythoneers are already used to
  typing in decorators, and its use in email addresses means it is
  more likely to be easily accessible across keyboard layouts than
  some other characters (e.g. ``$`` or non-ASCII characters).

* The mATrices mnemonic is cute.

* It's round like ``*`` and :math:`\cdot`.

* The use of a single-character token makes ``@@`` possible, which is
  a nice bonus.

* The swirly shape is reminiscent of the simultaneous sweeps over rows
  and columns that define matrix multiplication; its asymmetry is
  evocative of its non-commutative nature.


(Non)-Definitions for built-in types
------------------------------------

No ``__matmul__`` or ``__matpow__`` are defined for builtin numeric
types (``float``, ``int``, etc.) or for the ``numbers.Number``
hierarchy, because these types represent scalars, and the consensus
semantics for ``@`` are that it should raise an error on scalars.

We do not -- for now -- define a ``__matmul__`` method on the standard
``memoryview`` or ``array.array`` objects, for several reasons.
First, there is currently no way to create multidimensional memoryview
objects using only the stdlib, and array objects cannot represent
multidimensional data at all, which makes ``__matmul__`` much less
useful.  Second, providing a quality implementation of matrix
multiplication is highly non-trivial.  Naive nested loop
implementations are very slow and providing one in CPython would just
create a trap for users.  But the alternative -- providing a modern,
competitive matrix multiply -- would require that CPython link to a
BLAS library, which brings a set of new complications.  In particular,
several popular BLAS libraries (including the one that ships by
default on OS X) currently break the use of ``multiprocessing``
[#blas-fork]_.  And finally, we'd have to add quite a bit beyond
``__matmul__`` before ``memoryview`` or ``array.array`` would be
useful for numeric work -- like elementwise versions of the other
arithmetic operators, just to start.  Put together, these
considerations mean that the cost/benefit of adding ``__matmul__`` to
these types just isn't there, so for now we'll continue to delegate
these problems to numpy and friends, and defer a more systematic
solution to a future proposal.

There are also non-numeric Python builtins which define ``__mul__``
(``str``, ``list``, ...).  We do not define ``__matmul__`` for these
types either, because why would we even do that.


Unresolved issues
-----------------

Associativity of ``@``
''''''''''''''''''''''

It's been suggested that ``@`` should be right-associative, on the
grounds that for expressions like ``Mat @ Mat @ vec``, the two
different evaluation orders produce the same result, but the
right-associative order ``Mat @ (Mat @ vec)`` will be faster and use
less memory than the left-associative order ``(Mat @ Mat) @ vec``.
(Matrix-vector multiplication is much cheaper than matrix-matrix
multiplication).  It would be a shame if users found themselves
required to use an overabundance of parentheses to achieve acceptable
speed/memory usage in common situations, but, it's not currently clear
whether such cases actually are common enough to override Python's
general rule of left-associativity, or even whether they're more
common than the symmetric cases where left-associativity would be
faster (though this does seem intuitively plausible).  The only way to
answer this is probably to do an audit of some real-world uses and
check how often the associativity matters in practice; if this PEP is
accepted in principle, then we should probably do this check before
finalizing it.


Rejected alternatives to adding a new operator
==============================================

Over the past few decades, the Python numeric community has explored a
variety of ways to resolve the tension between matrix and elementwise
multiplication operations.  PEP 211 and PEP 225, both proposed in 2000
and last seriously discussed in 2008 [#threads-2008]_, were early
attempts to add new operators to solve this problem, but suffered from
serious flaws; in particular, at that time the Python numerical
community had not yet reached consensus on the proper API for array
objects, or on what operators might be needed or useful (e.g., PEP 225
proposes 6 new operators with unspecified semantics).  Experience
since then has now led to consensus that the best solution, for both
numeric Python and core Python, is to add a single infix operator for
matrix multiply (together with the other new operators this implies
like ``@=``).

We review some of the rejected alternatives here.

**Use a second type that defines __mul__ as matrix multiplication:**
As discussed above (`Background: What's wrong with the status quo?`_),
this has been tried this for many years via the ``numpy.matrix`` type
(and its predecessors in Numeric and numarray).  The result is a
strong consensus among both numpy developers and developers of
downstream packages that ``numpy.matrix`` should essentially never be
used, because of the problems caused by having conflicting duck types
for arrays.  (Of course one could then argue we should *only* define
``__mul__`` to be matrix multiplication, but then we'd have the same
problem with elementwise multiplication.)  There have been several
pushes to remove ``numpy.matrix`` entirely; the only counter-arguments
have come from educators who find that its problems are outweighed by
the need to provide a simple and clear mapping between mathematical
notation and code for novices (see `Transparent syntax is especially
crucial for non-expert programmers`_).  But, of course, starting out
newbies with a dispreferred syntax and then expecting them to
transition later causes its own problems.  The two-type solution is
worse than the disease.

**Add lots of new operators, or add a new generic syntax for defining
infix operators:** In addition to being generally un-Pythonic and
repeatedly rejected by BDFL fiat, this would be using a sledgehammer
to smash a fly.  The scientific python community has consensus that
adding one operator for matrix multiplication is enough to fix the one
otherwise unfixable pain point. (In retrospect, we all think PEP 225
was a bad idea too -- or at least far more complex than it needed to
be.)

**Add a new @ (or whatever) operator that has some other meaning in
general Python, and then overload it in numeric code:** This was the
approach taken by PEP 211, which proposed defining ``@`` to be the
equivalent of ``itertools.product``.  The problem with this is that
when taken on its own terms, adding an infix operator for
``itertools.product`` is just silly.  (During discussions of this PEP,
a similar suggestion was made to define ``@`` as a general purpose
function composition operator, and this suffers from the same problem;
``functools.compose`` isn't even useful enough to exist.)  Matrix
multiplication has a uniquely strong rationale for inclusion as an
infix operator.  There almost certainly don't exist any other binary
operations that will ever justify adding any other infix operators to
Python.

**Add a .dot method to array types so as to allow "pseudo-infix"
A.dot(B) syntax:** This has been in numpy for some years, and in many
cases it's better than dot(A, B).  But it's still much less readable
than real infix notation, and in particular still suffers from an
extreme overabundance of parentheses.  See `Why should matrix
multiplication be infix?`_ above.

**Use a 'with' block to toggle the meaning of * within a single code
block**: E.g., numpy could define a special context object so that
we'd have::

    c = a * b   # element-wise multiplication
    with numpy.mul_as_dot:
        c = a * b  # matrix multiplication

However, this has two serious problems: first, it requires that every
array-like type's ``__mul__`` method know how to check some global
state (``numpy.mul_is_currently_dot`` or whatever).  This is fine if
``a`` and ``b`` are numpy objects, but the world contains many
non-numpy array-like objects.  So this either requires non-local
coupling -- every numpy competitor library has to import numpy and
then check ``numpy.mul_is_currently_dot`` on every operation -- or
else it breaks duck-typing, with the above code doing radically
different things depending on whether ``a`` and ``b`` are numpy
objects or some other sort of object.  Second, and worse, ``with``
blocks are dynamically scoped, not lexically scoped; i.e., any
function that gets called inside the ``with`` block will suddenly find
itself executing inside the mul_as_dot world, and crash and burn
horribly -- if you're lucky.  So this is a construct that could only
be used safely in rather limited cases (no function calls), and which
would make it very easy to shoot yourself in the foot without warning.

**Use a language preprocessor that adds extra numerically-oriented
operators and perhaps other syntax:** (As per recent BDFL suggestion:
[#preprocessor]_) This suggestion seems based on the idea that
numerical code needs a wide variety of syntax additions.  In fact,
given ``@``, most numerical users don't need any other operators or
syntax; it solves the one really painful problem that cannot be solved
by other means, and that causes painful reverberations through the
larger ecosystem.  Defining a new language (presumably with its own
parser which would have to be kept in sync with Python's, etc.), just
to support a single binary operator, is neither practical nor
desireable.  In the numerical context, Python's competition is
special-purpose numerical languages (Matlab, R, IDL, etc.).  Compared
to these, Python's killer feature is exactly that one can mix
specialized numerical code with code for XML parsing, web page
generation, database access, network programming, GUI libraries, and
so forth, and we also gain major benefits from the huge variety of
tutorials, reference material, introductory classes, etc., which use
Python.  Fragmenting "numerical Python" from "real Python" would be a
major source of confusion.  A major motivation for this PEP is to
*reduce* fragmentation.  Having to set up a preprocessor would be an
especially prohibitive complication for unsophisticated users.  And we
use Python because we like Python!  We don't want
almost-but-not-quite-Python.

**Use overloading hacks to define a "new infix operator" like *dot*,
as in a well-known Python recipe:** (See: [#infix-hack]_) Beautiful is
better than ugly.  This is... not beautiful.  And not Pythonic.  And
especially unfriendly to beginners, who are just trying to wrap their
heads around the idea that there's a coherent underlying system behind
these magic incantations that they're learning, when along comes an
evil hack like this that violates that system, creates bizarre error
messages when accidentally misused, and whose underlying mechanisms
can't be understood without deep knowledge of how object oriented
systems work.  We've considered promoting this as a general solution,
and perhaps if the PEP is rejected we'll revisit this option, but so
far the numeric community has mostly elected to leave this one on the
shelf.


References
==========

.. [#preprocessor] From a comment by GvR on a G+ post by GvR; the
   comment itself does not seem to be directly linkable:
https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u
.. [#infix-hack] http://code.activestate.com/recipes/384122-infix-operators/
   http://www.sagemath.org/doc/reference/misc/sage/misc/decorators.html#sage.misc.decorators.infix_operator
.. [#scipy-conf] http://conference.scipy.org/past.html
.. [#pydata-conf] http://pydata.org/events/
.. [#lht] In this formula, :math:`\beta` is a vector or matrix of
   regression coefficients, :math:`V` is the estimated
   variance/covariance matrix for these coefficients, and we want to
   test the null hypothesis that :math:`H\beta = r`; a large :math:`S`
   then indicates that this hypothesis is unlikely to be true. For
   example, in an analysis of human height, the vector :math:`\beta`
   might contain one value which was the the average height of the
   measured men, and another value which was the average height of the
   measured women, and then setting :math:`H = [1, -1], r = 0` would
   let us test whether men and women are the same height on
   average. Compare to eq. 2.139 in
   http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode17.html

   Example code is adapted from
https://github.com/rerpy/rerpy/blob/0d274f85e14c3b1625acb22aed1efa85d122ecb7/rerpy/incremental_ls.py#L202

.. [#pycon-tutorials] Out of the 36 tutorials scheduled for PyCon 2014
   (https://us.pycon.org/2014/schedule/tutorials/), we guess that the
   8 below will almost certainly deal with matrices:

   * Dynamics and control with Python

   * Exploring machine learning with Scikit-learn

   * How to formulate a (science) problem and analyze it using Python
     code

   * Diving deeper into Machine Learning with Scikit-learn

   * Data Wrangling for Kaggle Data Science Competitions ? An etude

   * Hands-on with Pydata: how to build a minimal recommendation
     engine.

   * Python for Social Scientists

   * Bayesian statistics made simple

   In addition, the following tutorials could easily involve matrices:

   * Introduction to game programming

   * mrjob: Snakes on a Hadoop *("We'll introduce some data science
     concepts, such as user-user similarity, and show how to calculate
     these metrics...")*

   * Mining Social Web APIs with IPython Notebook

   * Beyond Defaults: Creating Polished Visualizations Using Matplotlib

   This gives an estimated range of 8 to 12 / 36 = 22% to 33% of
   tutorials dealing with matrices; saying ~20% then gives us some
   wiggle room in case our estimates are high.

.. [#sloc-details] SLOCs were defined as physical lines which contain
   at least one token that is not a COMMENT, NEWLINE, ENCODING,
   INDENT, or DEDENT.  Counts were made by using ``tokenize`` module
   from Python 3.2.3 to examine the tokens in all files ending ``.py``
   underneath some directory.  Only tokens which occur at least once
   in the source trees are included in the table.  The counting script
   will be available as an auxiliary file once this PEP is submitted;
   until then, it can be found here:
   https://gist.github.com/njsmith/9157645

   Matrix multiply counts were estimated by counting how often certain
   tokens which are used as matrix multiply function names occurred in
   each package.  In principle this could create false positives, but
   as far as I know the counts are exact; it's unlikely that anyone is
   using ``dot`` as a variable name when it's also the name of one of
   the most widely-used numpy functions.

   All counts were made using the latest development version of each
   project as of 21 Feb 2014.

   'stdlib' is the contents of the Lib/ directory in commit
   d6aa3fa646e2 to the cpython hg repository, and treats the following
   tokens as indicating matrix multiply: n/a.

   'scikit-learn' is the contents of the sklearn/ directory in commit
   69b71623273ccfc1181ea83d8fb9e05ae96f57c7 to the scikit-learn
   repository (https://github.com/scikit-learn/scikit-learn), and
   treats the following tokens as indicating matrix multiply: ``dot``,
   ``fast_dot``, ``safe_sparse_dot``.

   'nipy' is the contents of the nipy/ directory in commit
   5419911e99546401b5a13bd8ccc3ad97f0d31037 to the nipy repository
   (https://github.com/nipy/nipy/), and treats the following tokens as
   indicating matrix multiply: ``dot``.

.. [#blas-fork] BLAS libraries have a habit of secretly spawning
   threads, even when used from single-threaded programs.  And threads
   play very poorly with ``fork()``; the usual symptom is that
   attempting to perform linear algebra in a child process causes an
   immediate deadlock.

.. [#threads-2008] http://fperez.org/py4science/numpy-pep225/numpy-pep225.html

.. [#broadcasting] http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html

.. [#matmul-other-langs]
http://mail.scipy.org/pipermail/scipy-user/2014-February/035499.html

.. [#github-details] Counts were produced by manually entering the
   string ``"import foo"`` or ``"from foo import"`` (with quotes) into
   the Github code search page, e.g.:
   https://github.com/search?q=%22import+numpy%22&ref=simplesearch&type=Code
   on 2014-04-10 at ~21:00 UTC.  The reported values are the numbers
   given in the "Languages" box on the lower-left corner, next to
   "Python".  This also causes some undercounting (e.g., leaving out
   Cython code, and possibly one should also count HTML docs and so
   forth), but these effects are negligible (e.g., only ~1% of numpy
   usage appears to occur in Cython code, and probably even less for
   the other modules listed).  The use of this box is crucial,
   however, because these counts appear to be stable, while the
   "overall" counts listed at the top of the page ("We've found ___
   code results") are highly variable even for a single search --
   simply reloading the page can cause this number to vary by a factor
   of 2 (!!).  (They do seem to settle down if one reloads the page
   repeatedly, but nonetheless this is spooky enough that it seemed
   better to avoid these numbers.)

   These numbers should of course be taken with multiple grains of
   salt; it's not clear how representative Github is of Python code in
   general, and limitations of the search tool make it impossible to
   get precise counts.  AFAIK this is the best data set currently
   available, but it'd be nice if it were better.  In particular:

   * Lines like ``import sys, os`` will only be counted in the ``sys``
     row.

   * A file containing both ``import X`` and ``from X import`` will be
     counted twice

   * Imports of the form ``from X.foo import ...`` are missed.  We
     could catch these by instead searching for "from X", but this is
     a common phrase in English prose, so we'd end up with false
     positives from comments, strings, etc.  For many of the modules
     considered this shouldn't matter too much -- for example, the
     stdlib modules have flat namespaces -- but it might especially
     lead to undercounting of django, scipy, and twisted.

   Also, it's possible there exist other non-stdlib modules we didn't
   think to test that are even more-imported than numpy -- though we
   tried quite a few of the obvious suspects.  If you find one, let us
   know!  The modules tested here were chosen based on a combination
   of intuition and the top-100 list at pypi-ranking.info.

   Fortunately, it doesn't really matter if it turns out that numpy
   is, say, merely the *third* most-imported non-stdlib module, since
   the point is just that numeric programming is a common and
   mainstream activity.

   Finally, we should point out the obvious: whether a package is
   import**ed** is rather different from whether it's import**ant**.
   No-one's claiming numpy is "the most important package" or anything
   like that.  Certainly more packages depend on distutils, e.g., then
   depend on numpy -- and far fewer source files import distutils than
   import numpy.  But this is fine for our present purposes.  Most
   source files don't import distutils because most source files don't
   care how they're distributed, so long as they are; these source
   files thus don't care about details of how distutils' API works.
   This PEP is in some sense about changing how numpy's and related
   packages' APIs work, so the relevant metric is to look at source
   files that are choosing to directly interact with that API, which
   is sort of like what we get by looking at import statements.

.. [#hugunin] The first such proposal occurs in Jim Hugunin's very
   first email to the matrix SIG in 1995, which lays out the first
   draft of what became Numeric. He suggests using ``*`` for
   elementwise multiplication, and ``%`` for matrix multiplication:
   https://mail.python.org/pipermail/matrix-sig/1995-August/000002.html


Copyright
=========

This document has been placed in the public domain.

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From njs at pobox.com  Fri Mar 14 03:08:17 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 02:08:17 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>

On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote:
> [...]

Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really,
I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't
too distracting!

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From ericsnowcurrently at gmail.com  Fri Mar 14 03:08:35 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 13 Mar 2014 20:08:35 -0600
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <lfrt3c$62p$1@ger.gmane.org>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
 <lfrt3c$62p$1@ger.gmane.org>
Message-ID: <CALFfu7A16Fo9KeVTc=RBqM2OhA9tSj9Hx3ngviVtids5=npFNg@mail.gmail.com>

On Mar 13, 2014 3:18 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> Le 13/03/2014 08:43, Eric Snow a ?crit :
>
>> 1. the existing sys module be renamed to _sys (and the stdlib be
>> changed accordingly where appropriate);
>> 2. a pure Python sys module be added to the stdlib that wraps the
existing API;
>
>
> I'm not sure what that brings until the need arises.

It means no one needs to change 'import sys' to 'import _sys' in their code.

> However, making it a pure Python sys module means it won't be immediately
available at startup like sys currently is (i.e. it's a built-in module). I
don't know if there are any scenarios where it may matter.

As I noted, we would need to update some spots in the stdlib to import _sys
rather than sys.

>
>
>> 2.5. the new module should use the replace-itself-in-sys.modules
>> technique to provide a module subclass that supports descriptors.
>
>
> I don't think any module should be specific in that regard. Special cases
generally make the language more complicated to learn. Also, you haven't
explained the rationale.

As others have noted, there isn't much point to making any of these changes
until we have some need that they satisfy.  I anticipate having that need
in the mid-term and wanted to see if the proposed approach is palatable.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/db60724e/attachment.html>

From ericsnowcurrently at gmail.com  Fri Mar 14 03:53:52 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Thu, 13 Mar 2014 20:53:52 -0600
Subject: [Python-ideas] a standard approach to
	module-replace-itself-in-sys.modules
In-Reply-To: <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com>
References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com>
 <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com>
Message-ID: <CALFfu7DSO9cQ=1k0yXpEjnnd3aq-1HjSDRovpXuV9w+AT1uBqA@mail.gmail.com>

On Mar 13, 2014 6:54 AM, "Brett Cannon" <brett at python.org> wrote:
> Do we actually want to promote this practice by codifying support for it
in the stdlib?

I guess I was expecting that question from you. :)  I figure if the
practice is officially supported then we should help folks doing it to not
shoot themselves in the foot.  As to how much such a tool will promote the
practice, I'm not sure that it will, particularly if the helper is located
out of the way.  However I'm sure you've watched similar additions unfold
and have a better sense of that than I do.

> If you're doing this to get decorators for module attributes then you
should say this is for that goal and the solution is getting the module
swapping hack to be easier to use (and might call for a better solution).
Otherwise this proposal feels like it's trying to codify behaviour that is
rarely necessary.

Agreed.  Furthermore I doubt I'll push on this idea without a strong need
first.  More than anything I wanted to share what seemed like an elegant
solution to the "problem", even if we don't need to solve it right away.
If I get more serious about it I'll hit up folks that are using the current
hack (Django?) and see if it improves anything for them.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/0fcb4c2b/attachment.html>

From abarnert at yahoo.com  Fri Mar 14 05:03:57 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 13 Mar 2014 21:03:57 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>
Message-ID: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>

This (or various related ideas) actually has come up somewhat regularly. I think the reason it hasn't gone anywhere is that there was never enough input from the numerical community; someone says, "I'll bet this would be useful for numerics," and someone else says "Then why haven't the numarray/numeric/numpy guys asked for it since 2000?", and nobody has an answer to that. This obviously is the answer to that.

One thing that always comes up is a suggestion for using Unicode. There are obvious downsides--the Unicode multiplication character isn't easy to type; even if Python and major code editors are fully Unicode friendly, code often has to go through channels that may not be; etc. But it's worth asking whether the numeric community has considered this and rejected it for these reasons, or for other reasons, or if they'd be happy with it but just didn't think Python was ready for it, or whatever.

Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least.

On Mar 13, 2014, at 19:08, Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> [...]
> 
> Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really,
> I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't
> too distracting!
> 
> -n
> 
> -- 
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From alexander.belopolsky at gmail.com  Fri Mar 14 05:18:48 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 14 Mar 2014 00:18:48 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>
 <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>
Message-ID: <CAP7h-xYNFCVCMfmZoCtUneV8XWTyuaNDE7K+oEgOf-EV+N4+rQ@mail.gmail.com>

On Fri, Mar 14, 2014 at 12:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote:

> One thing that always comes up is a suggestion for using Unicode. There
> are obvious downsides--the Unicode multiplication character isn't easy to
> type; even if Python and major code editors are fully Unicode friendly,
> code often has to go through channels that may not be; etc. But it's worth
> asking whether the numeric community has considered this and rejected it
> for these reasons, or for other reasons, or if they'd be happy with it but
> just didn't think Python was ready for it, or whatever.


It did.  Some 60 years ago [1].  Mostly rejected by now [2].

[1] http://en.wikipedia.org/wiki/APL_(programming_language)
[2] http://www.jsoftware.com/start.htm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/9f3cd458/attachment-0001.html>

From tjreedy at udel.edu  Fri Mar 14 05:33:35 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 14 Mar 2014 00:33:35 -0400
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <lfu0r5$s4i$1@ger.gmane.org>

On 3/13/2014 9:59 PM, Nathaniel Smith wrote:

> I'd have rejected these too! So I thought, maybe we should try the
> radical tactic of writing down what we actually want, carefully
> explaining why we want it, and then asking for it.

Great idea ;-). Until someone comes up with reasons to reject this, I am +1.

-- 
Terry Jan Reedy


From njs at pobox.com  Fri Mar 14 06:05:42 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 05:05:42 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>
 <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>
Message-ID: <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com>

On Fri, Mar 14, 2014 at 4:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
> This (or various related ideas) actually has come up somewhat regularly. I think the reason it hasn't gone anywhere is that there was never enough input from the numerical community; someone says, "I'll bet this would be useful for numerics," and someone else says "Then why haven't the numarray/numeric/numpy guys asked for it since 2000?", and nobody has an answer to that. This obviously is the answer to that.

The numeric community has many talents, but crossing the cultural
divide with upstream is not really a speciality...

> One thing that always comes up is a suggestion for using Unicode. There are obvious downsides--the Unicode multiplication character isn't easy to type; even if Python and major code editors are fully Unicode friendly, code often has to go through channels that may not be; etc. But it's worth asking whether the numeric community has considered this and rejected it for these reasons, or for other reasons, or if they'd be happy with it but just didn't think Python was ready for it, or whatever.

We don't really have a strong opinion on which character is used. It
is nice if it's easy to type, though -- esp. because in many ways it's
already beginners who suffer the brunt of the elmul/matmul issues, and
beginners are exactly the people for whom figuring out how to type
some stupid character is a prohibitive speed bump. (Scientific python
has a very large ongoing stream of newbie programmers.) And I don't
know that any of the Unicode characters are actually better. In real
math the two operations are distinguished by context, so there's no
existing character with the right meaning that we can steal. In some
ways having an unusual character for this is better, because it
indicates a special-case operation. If the two multiplication
characters are * and ?, then how do you remember which is which? But
if they're * and @, then it's easy to remember that * is the
general-use one, and @ is the special non-commutative matrix
multiplication one.

It's not like new operators are being added to Python every week and
we need to start scraping the bottom of the barrel for new characters.
ASCII gets the job done, has some minor upsides, and no real
downsides.

> Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least.

I'm not an expert on Haskell and C++ matrix libraries -- perhaps
someone else will speak up -- but Haskell's extensible infix system
and static typing do seem like they put them in a pretty different
design space. And I'm skeptical that there exist Haskell libraries
with anything like numpy's maturity, though I'll be delighted to be
proven wrong.

Eigen is maybe the most popular C++ matrix/array library right now,
and AFAICT from their docs they use a system where there's one "array"
type that *only* supports elementwise multiplication, and one "matrix"
type that *only* supports matrix multiplication, and if you want both
operations you have to cast back and forth. This seems a bit annoying
to me, but not as deadly as it is in Python -- in C++ the static type
system means they can at least fob off the work of keeping track of
which variables are which type, and doing conversions at function
boundaries, onto the compiler. In Python these conversions have to
happen by hand, and the two-type solution is just not workable.

In general, I doubt there's huge amounts of experience to steal from
other languages, because Python is in a unique place: AFAICT it's the
only general purpose language that has ever made serious inroads
against the specialized numeric languages (Matlab, R, GAUSS, IDL,
etc.) on their home territory. So we're breaking new ground here. (All
those languages do have separate infix operators for elementwise and
matrix multiplication; they all involve horrible names like ".*" or
"%*%" and all kinds of weird inconsistencies.)

-n

> On Mar 13, 2014, at 19:08, Nathaniel Smith <njs at pobox.com> wrote:
>
>> On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>> [...]
>>
>> Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really,
>> I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't
>> too distracting!
>>
>> -n
>>
>> --
>> Nathaniel J. Smith
>> Postdoctoral researcher - Informatics - University of Edinburgh
>> http://vorpus.org
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From taleinat at gmail.com  Fri Mar 14 07:42:03 2014
From: taleinat at gmail.com (Tal Einat)
Date: Fri, 14 Mar 2014 08:42:03 +0200
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <CALWZvp4x2sQgU=eLuX5T8gMsGeisOUygbwa0pxWW7nnOr7JwTQ@mail.gmail.com>

On Fri, Mar 14, 2014 at 3:59 AM, Nathaniel Smith <njs at pobox.com> wrote:
>
> PEP: XXXX
> Title: Dedicated infix operators for matrix multiplication and matrix power

A wholehearted +1!

- Tal Einat

From storchaka at gmail.com  Fri Mar 14 07:55:04 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 14 Mar 2014 08:55:04 +0200
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <20140313160353.745d9659@anarchist.wooz.org>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
 <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org>
 <20140313160353.745d9659@anarchist.wooz.org>
Message-ID: <lfu92v$8pb$1@ger.gmane.org>

13.03.14 22:03, Barry Warsaw ???????(??):
> On Mar 13, 2014, at 02:04 PM, Serhiy Storchaka wrote:
>> Many sys attributes (argv, displayhook, excepthook, modules, path,
>> path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout,
>> tracebacklimit) are used implicit in builtins and in interpreter core. Even
>> import system doesn't work without the sys module. So it should be
>> initialized very early and finalized very late.
>
> If the C API PySys_* functions were updated to use _sys instead, hopefully
> most stuff in the core would still work, right?

Perhaps. If sys attributes will be writable properties which updates _sys.


From abarnert at yahoo.com  Fri Mar 14 09:33:09 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 14 Mar 2014 01:33:09 -0700 (PDT)
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com>
 <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com>
 <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com>
Message-ID: <1394785989.3468.YahooMailNeo@web181002.mail.ne1.yahoo.com>

From: Nathaniel Smith <njs at pobox.com>
Sent: Thursday, March 13, 2014 10:05 PM


>On Fri, Mar 14, 2014 at 4:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote:


[snip, here and elsewhere]

>The numeric community has many talents, but crossing the cultural
>divide with upstream is not really a speciality...

Well, you've fixed that here.

>> One thing that always comes up is a suggestion for using Unicode.
>
>We don't really have a strong opinion on which character is used. It
>is nice if it's easy to type, though -- esp. because in many ways it's
>already beginners who suffer the brunt of the elmul/matmul issues, and
>beginners are exactly the people for whom figuring out how to type
>some stupid character is a prohibitive speed bump. (Scientific python
>has a very large ongoing stream of newbie programmers.) And I don't
>know that any of the Unicode characters are actually better. In real
>math the two operations are distinguished by context, so there's no
>existing character with the right meaning that we can steal. In some
>ways having an unusual character for this is better, because it
>indicates a special-case operation. If the two multiplication
>characters are * and ?, then how do you remember which is which? But
>if they're * and @, then it's easy to remember that * is the
>general-use one, and @ is the special non-commutative matrix
>multiplication one.

Honestly, that sounds like a strong negative opinion, with good motivations, not no opinion,

The argument for Unicode is generally "readability is more important than writability," all things being equal. The traditional counter to that is that all things aren't equal, as it's still way too hard for novices to write Unicode characters. But adding the additional rationale that there is no obvious Unicode character for this operator certainly strengthens the case for @.

>It's not like new operators are being added to Python every week and
>we need to start scraping the bottom of the barrel for new characters.
>ASCII gets the job done, has some minor upsides, and no real
>downsides.

Well, we have just about two ASCII characters left, @ and ?,?so we really are scraping the bottom of the barrel. But you made a good argument why this is worthy of one of those two characters.

>> Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least.
>
>I'm not an expert on Haskell and C++ matrix libraries -- perhaps
>someone else will speak up -- but Haskell's extensible infix system
>and static typing do seem like they put them in a pretty different
>design space. And I'm skeptical that there exist Haskell libraries
>with anything like numpy's maturity, though I'll be delighted to be
>proven wrong.

From what I can tell, it's more common to implement matrix libraries in Haskell than to use them. Probably because Haskell is more of a research and/or teaching language than a language you'd expect, say, physicists to use. But I was hoping someone would actually know that, rather than just guess?

>Eigen is maybe the most popular C++ matrix/array library right now,
>and AFAICT from their docs they use a system where there's one "array"
>type that *only* supports elementwise multiplication, and one "matrix"
>type that *only* supports matrix multiplication, and if you want both
>operations you have to cast back and forth. This seems a bit annoying
>to me, but not as deadly as it is in Python -- in C++ the static type
>system means they can at least fob off the work of keeping track of
>which variables are which type, and doing conversions at function
>boundaries, onto the compiler. In Python these conversions have to
>happen by hand, and the two-type solution is just not workable.


Yes, that's why I suspected that C++ experience would not be applicable to Python. Assigning a matrix value to an array variable, or passing it to an array function, can be an implicit (but still obvious) conversion. And the way they'd solve the same problems Python solves with duck typing would be would generic functions, which is pretty different from duck-typed functions. And I'd expect that most other static languages are similar. But it might be worth adding to the PEP if you actually know the answer rather than just suspecting.

>In general, I doubt there's huge amounts of experience to steal from
>other languages, because Python is in a unique place: AFAICT it's the
>only general purpose language that has ever made serious inroads
>against the specialized numeric languages (Matlab, R, GAUSS, IDL,
>etc.) on their home territory. So we're breaking new ground here. (All
>those languages do have separate infix operators for elementwise and
>matrix multiplication; they all involve horrible names like ".*" or
>"%*%" and all kinds of weird inconsistencies.)

You could be right. Most of the core low-level libraries are in C, and C++ programmers have a habit of just using C libraries with their C APIs rather than wrapping them in generic/OO C++ APIs? (Honestly, I think automatic C compatibility is one of the biggest problems with C++, not its biggest strength. In Python, it's easy to wrap a C library, and you have to do it, so you do it and end up with a great API; in C++, it's easy to wrap a C library, but you don't have to, so you end up using it C-style and losing all the benefits of C++.)

From mal at egenix.com  Fri Mar 14 11:16:34 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 14 Mar 2014 11:16:34 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <5322D702.6040907@egenix.com>

On 14.03.2014 02:59, Nathaniel Smith wrote:
> PEP: XXXX
> Title: Dedicated infix operators for matrix multiplication and matrix power
>
> ...
>
> Specification
> =============
> 
> Two new binary operators are added to the Python language, together
> with corresponding in-place versions:
> 
> =======  ========================= ===============================
>  Op      Precedence/associativity     Methods
> =======  ========================= ===============================
> ``@``    Same as ``*``             ``__matmul__``, ``__rmatmul__``
> ``@@``   Same as ``**``            ``__matpow__``, ``__rmatpow__``
> ``@=``   n/a                       ``__imatmul__``
> ``@@=``  n/a                       ``__imatpow__``
> =======  ========================= ===============================
>
> ...
>
> When working with n-dimensional arrays, there are two different ways
> we might want to define multiplication.  One is elementwise
> multiplication::
> 
>   [[1, 2],     [[11, 12],     [[1 * 11, 2 * 12],
>    [3, 4]]  x   [13, 14]]  =   [3 * 13, 4 * 14]]
> 
> and the other is `matrix multiplication`_:
> 
> .. _matrix multiplication: https://en.wikipedia.org/wiki/Matrix_multiplication

I have some questions:

1. Since in math, the operator is usually spelt "?" (the center dot,
   or "." but that's already reserved for methods and attributes in
   Python), why not try to use that instead of "@" (which in Python
   already identifies decorators) ?

2. The PEP should include a section on how other programming languages
   solve this, i.e. what syntax they use for matrix multiplications.

3. Since matrix multiplication is only one type of product you find
   in math, albeit a very frequently used one, how would those other
   products fit into the picture ? Would then have to use methods
   again ? E.g. think of cross product, inner product, outer/tensor
   product.

4. Another very common operation needed in vector/matrix calculation
   is transposition. This is usually written as superscript "T" or "t"
   ("?" in Unicode). Wouldn't this operator be needed as well, to
   make the picture complete ? OTOH, we currently don't have postfix
   operators in Python, so I guess writing this as A.transpose()
   comes close enough ;-)

Now since this is all about syntactic sugar, we also need to look at
some code examples:


I == A @@ -1 @ A
vs.
I == A ?? -1 ? A
vs.
I == A.inverse().dot(A)


(A @ B).transpose() == A.transpose() @ B.transpose()
vs.
(A ? B).transpose() == A.transpose() ? B.transpose()
vs.
A.dot(B).transpose() == A.transpose().dot(B.transpose())


c = A @ v
vs.
c = A ? v
vs.
c = A.dot(v)


Hmm, even though I'd love to see matrix operators in Python,
I don't think they really add clarity to the syntax of matrix
calculations -  a bit disappointing, I must say :-(

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 14 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          15 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               26 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Fri Mar 14 11:24:26 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 11:24:26 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <20140314112426.7854e9f6@fsol>


Hi,

On Fri, 14 Mar 2014 01:59:14 +0000
Nathaniel Smith <njs at pobox.com> wrote:
> 
> PEP 211, which instead adds an operator for itertools.product, aka,
> "maybe we can sneak matrix multiply past Guido in some sort of...
> large, wooden rabbit..."

A chocolate rabbit might be tastier for some core developers ;-)

Ok, so first I'm gonna state upfront that I don't have any personal
interest in this proposal (i.e. I don't do matrix multiplications often
enough for it to matter to me). That said:

> This PEP proposes two new binary operators dedicated to matrix
> multiplication and matrix power, spelled ``@`` and ``@@``
> respectively.  (Mnemonic: ``@`` is ``*`` for mATrices.)

That sounds like a strange choice to me. There's nothing in "@" or "@@"
that suggests "matrices" or "multiplication". Also, are there
precedents in other languages?

AFAIK, other languages sometimes use ".*" for matrix multiplication (as
opposed to element-wise multiplication). Why not re-use that
convention? Would that lead to parsing difficulties?

That said, I'm rather sympathetic to the general idea, and if "@" is
the least contentious option, then I'm ok with it too ("@@" doesn't
sound like a good idea at all, though).

Thanks for writing this PEP!

Regards

Antoine.



From solipsis at pitrou.net  Fri Mar 14 11:29:31 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 11:29:31 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com>
Message-ID: <20140314112931.4c38de75@fsol>

On Fri, 14 Mar 2014 11:16:34 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> 
> 4. Another very common operation needed in vector/matrix calculation
>    is transposition. This is usually written as superscript "T" or "t"
>    ("?" in Unicode). Wouldn't this operator be needed as well, to
>    make the picture complete ? OTOH, we currently don't have postfix
>    operators in Python, so I guess writing this as A.transpose()
>    comes close enough ;-)

Or simply implement __invert__, so you can write it "~ A".
(__invert__ doesn't really invert a number, it takes the bitwise
complement :-))

> Hmm, even though I'd love to see matrix operators in Python,
> I don't think they really add clarity to the syntax of matrix
> calculations -  a bit disappointing, I must say :-(

I think they do when you have several of these operations combined in a
single formula, with parentheses and the like.

Regards

Antoine.



From robert.kern at gmail.com  Fri Mar 14 12:25:27 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 11:25:27 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <5322D702.6040907@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com>
Message-ID: <lfuouo$v25$1@ger.gmane.org>

On 2014-03-14 10:16, M.-A. Lemburg wrote:

> I have some questions:
>
> 1. Since in math, the operator is usually spelt "?" (the center dot,
>     or "." but that's already reserved for methods and attributes in
>     Python), why not try to use that instead of "@" (which in Python
>     already identifies decorators) ?

I think the current feeling of the Python core team is against including 
non-ASCII characters in the language's keywords or operators. Even if that were 
not so, I would still recommend against it because it would be quite difficult 
to type. I don't know off-hand the key combination to do it on my native system, 
and it would change from system to system.

> 2. The PEP should include a section on how other programming languages
>     solve this, i.e. what syntax they use for matrix multiplications.
>
> 3. Since matrix multiplication is only one type of product you find
>     in math, albeit a very frequently used one, how would those other
>     products fit into the picture ? Would then have to use methods
>     again ? E.g. think of cross product, inner product, outer/tensor
>     product.

Our experience is that these have come up much less regularly than matrix 
multiplication. The two products in common use in our code are the Hadamard 
product (elementwise multiplication, currently assigned to * in numpy) and 
matrix multiplication (currently done with the function numpy.dot()).

> 4. Another very common operation needed in vector/matrix calculation
>     is transposition. This is usually written as superscript "T" or "t"
>     ("?" in Unicode). Wouldn't this operator be needed as well, to
>     make the picture complete ? OTOH, we currently don't have postfix
>     operators in Python, so I guess writing this as A.transpose()
>     comes close enough ;-)

Indeed. Numpy already uses a .T property for this.

> Now since this is all about syntactic sugar, we also need to look at
> some code examples:
>
>
> I == A @@ -1 @ A
> vs.
> I == A ?? -1 ? A
> vs.
> I == A.inverse().dot(A)
>
>
> (A @ B).transpose() == A.transpose() @ B.transpose()
> vs.
> (A ? B).transpose() == A.transpose() ? B.transpose()
> vs.
> A.dot(B).transpose() == A.transpose().dot(B.transpose())

(A @ B).T == B.T @ A.T
(A ? B).T == B.T ? A.T
A.dot(B).T == B.T.dot(A.T)

(FWIW, I didn't notice the math error until I wrote out the @ version.)

> c = A @ v
> vs.
> c = A ? v
> vs.
> c = A.dot(v)
>
>
> Hmm, even though I'd love to see matrix operators in Python,
> I don't think they really add clarity to the syntax of matrix
> calculations -  a bit disappointing, I must say :-(

Some more from real code:

RSR = R.dot(var_beta.dot(R.T))
RSR = R @ var_beta @ R.T

xx_inv.dot(xeps.dot(xx_inv))
xx_inv @ xeps @ xx_inv

dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 
4/period*F2lower.dot(F2lower.T)
dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ 
F2lower.T)

dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + 
FX.dot(Gi.dot(dFX2_dper.T))
(dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T)

torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
(((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From mal at egenix.com  Fri Mar 14 12:48:13 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 14 Mar 2014 12:48:13 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfuouo$v25$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org>
Message-ID: <5322EC7D.1020908@egenix.com>

On 14.03.2014 12:25, Robert Kern wrote:
> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>> 4. Another very common operation needed in vector/matrix calculation
>>     is transposition. This is usually written as superscript "T" or "t"
>>     ("?" in Unicode). Wouldn't this operator be needed as well, to
>>     make the picture complete ? OTOH, we currently don't have postfix
>>     operators in Python, so I guess writing this as A.transpose()
>>     comes close enough ;-)
> 
> Indeed. Numpy already uses a .T property for this.

Ah, good trick :-)

>> Now since this is all about syntactic sugar, we also need to look at
>> some code examples:
>>
>>
>> I == A @@ -1 @ A
>> vs.
>> I == A ?? -1 ? A
>> vs.
>> I == A.inverse().dot(A)
>>
>>
>> (A @ B).transpose() == B.transpose() @ A.transpose()
>> vs.
>> (A ? B).transpose() == B.transpose() ? A.transpose()
>> vs.
>> A.dot(B).transpose() == B.transpose().dot(A.transpose())
> 
> (A @ B).T == B.T @ A.T
> (A ? B).T == B.T ? A.T
> A.dot(B).T == B.T.dot(A.T)
> 
> (FWIW, I didn't notice the math error until I wrote out the @ version.)

Thanks; I should have proofread the email before hitting the send button.

I've correct the quoted version above to have the comparisons
return True for all A and B instead of just for a select few :-)

>> c = A @ v
>> vs.
>> c = A ? v
>> vs.
>> c = A.dot(v)
>>
>>
>> Hmm, even though I'd love to see matrix operators in Python,
>> I don't think they really add clarity to the syntax of matrix
>> calculations -  a bit disappointing, I must say :-(
> 
> Some more from real code:
> 
> RSR = R.dot(var_beta.dot(R.T))
> RSR = R @ var_beta @ R.T
> 
> xx_inv.dot(xeps.dot(xx_inv))
> xx_inv @ xeps @ xx_inv
> 
> dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T)
> dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T)
> 
> dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T))
> (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T)
> 
> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']

This doesn't look very readable to me - the operator saves you
a few parens in some situations, but as in the last example, it can
also require adding new ones.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 14 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          15 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               26 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Fri Mar 14 12:54:47 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 12:54:47 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com>
Message-ID: <20140314125447.3486c23a@fsol>

On Fri, 14 Mar 2014 12:48:13 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> > 
> > Some more from real code:
> > 
> > RSR = R.dot(var_beta.dot(R.T))
> > RSR = R @ var_beta @ R.T
> > 
> > xx_inv.dot(xeps.dot(xx_inv))
> > xx_inv @ xeps @ xx_inv
> > 
> > dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T)
> > dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T)
> > 
> > dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T))
> > (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T)
> > 
> > torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
> > (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']
> 
> This doesn't look very readable to me - the operator saves you
> a few parens in some situations, but as in the last example, it can
> also require adding new ones.

The parentheses mirror those necessary in the equivalent mathematical
formula, though, so they are "natural" in a sense.

I do find the "@" examples much more readable myself - except that I
don't understand what they are about, of course :-)

Regards

Antoine.



From steve at pearwood.info  Fri Mar 14 13:32:39 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Mar 2014 23:32:39 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <20140314112426.7854e9f6@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol>
Message-ID: <20140314123239.GP12595@ando>

On Fri, Mar 14, 2014 at 11:24:26AM +0100, Antoine Pitrou wrote:

> > This PEP proposes two new binary operators dedicated to matrix
> > multiplication and matrix power, spelled ``@`` and ``@@``
> > respectively.  (Mnemonic: ``@`` is ``*`` for mATrices.)
> 
> That sounds like a strange choice to me. There's nothing in "@" or "@@"
> that suggests "matrices" or "multiplication".

The PEP even gives a mnemonic for it: @ is for mATrix multiplication.

There's nothing in @ that suggests "decorator", nor is there anything 
about * that suggests scalar multiplication.

Out of the common symbols used in Python, & is possibly the only 
operator symbol with a non-arbitrary connection to its meaning. & was 
originally a ligature of "et", the Latin word for "and". Everything else 
is pretty much arbitrary, and it's only habit and long use that makes 
them seem normal.


> AFAIK, other languages sometimes use ".*" for matrix multiplication (as
> opposed to element-wise multiplication). Why not re-use that
> convention? Would that lead to parsing difficulties?

"Syntax should not look like grit on Tim Peters' monitor."

.* is, in my opinion, pretty ugly, and leads to even ugly 
extensions:

a .* b  # matrix multiplication
a .*= b  # in place matrix multiplication
a .** b  # matrix exponentiation
a .**= b  # in-place matrix exponentiation

And it suffers from exactly the same lack of connection to matrix 
multiplication as @ does: there is nothing about . that spells "matrix".

It also has the problem that it could cause confusion with vector dot 
product. Read out A .* B aloud and you get something that sounds like it 
might mean dot product, "A dot times B". 

 
> That said, I'm rather sympathetic to the general idea, and if "@" is
> the least contentious option, then I'm ok with it too ("@@" doesn't
> sound like a good idea at all, though).

I don't see why. Since Python uses * and ** for scalar multiplication 
and exponentiation, and given that @ is accepted as matrix 
multiplication, then it is completely logical to follow the same 
pattern, which gives us @@ for matrix exponentiation. That's no sillier 
than using ** for exponentiation, instead of ^ as the Gods intended :-)





-- 
Steven

From solipsis at pitrou.net  Fri Mar 14 13:57:40 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 13:57:40 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
Message-ID: <20140314135740.050bcd7f@fsol>

On Fri, 14 Mar 2014 23:32:39 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Mar 14, 2014 at 11:24:26AM +0100, Antoine Pitrou wrote:
> 
> > > This PEP proposes two new binary operators dedicated to matrix
> > > multiplication and matrix power, spelled ``@`` and ``@@``
> > > respectively.  (Mnemonic: ``@`` is ``*`` for mATrices.)
> > 
> > That sounds like a strange choice to me. There's nothing in "@" or "@@"
> > that suggests "matrices" or "multiplication".
> 
> The PEP even gives a mnemonic for it: @ is for mATrix multiplication.

Well, "@" is not pronounced "at" in every country, so your mnemonic
will only work for a subset of Python users.

> There's nothing in @ that suggests "decorator", nor is there anything 
> about * that suggests scalar multiplication.

Apart from the fact that "*" is commonly used for multiplication in
most programming languages, you mean?

("@" either spells "decorator" or "in reply to" or "e-mail address" to
me, depending on the context)

> It also has the problem that it could cause confusion with vector dot 
> product. Read out A .* B aloud and you get something that sounds like it 
> might mean dot product, "A dot times B". 

That is true, and I realize I might have got the convention reversed :-)

But, again, "dot product" is an English term which has no direct
equivalent in other languages. In French, "dot product" doesn't exist,
we only say "scalar product" and therefore the confusion doesn't exist.

> > That said, I'm rather sympathetic to the general idea, and if "@" is
> > the least contentious option, then I'm ok with it too ("@@" doesn't
> > sound like a good idea at all, though).
> 
> I don't see why. Since Python uses * and ** for scalar multiplication 
> and exponentiation, and given that @ is accepted as matrix 
> multiplication, then it is completely logical to follow the same 
> pattern, which gives us @@ for matrix exponentiation.

It's logical, but it's much less useful, and it's also starting to look
really weird: since the operator space is scarce, it makes sense to
only provide those that have common use.

Regards

Antoine.



From donald at stufft.io  Fri Mar 14 14:01:38 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 14 Mar 2014 09:01:38 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <20140314135740.050bcd7f@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
 <20140314135740.050bcd7f@fsol>
Message-ID: <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io>

The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. 

> On Mar 14, 2014, at 8:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> ("@" either spells "decorator" or "in reply to" or "e-mail address" to
> me, depending on the context)

From solipsis at pitrou.net  Fri Mar 14 14:19:56 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 14:19:56 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
 <20140314135740.050bcd7f@fsol>
 <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io>
Message-ID: <20140314141956.25a5b920@fsol>

On Fri, 14 Mar 2014 09:01:38 -0400
Donald Stufft <donald at stufft.io> wrote:
> The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. 

It's not about "handling it", it's about finding the best candidate.

Regards

Antoine.



From mal at egenix.com  Fri Mar 14 14:20:33 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 14 Mar 2014 14:20:33 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfuouo$v25$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org>
Message-ID: <53230221.1030501@egenix.com>

On 14.03.2014 12:25, Robert Kern wrote:
> On 2014-03-14 10:16, M.-A. Lemburg wrote:
> 
>> I have some questions:
>>
>> 1. Since in math, the operator is usually spelt "?" (the center dot,
>>     or "." but that's already reserved for methods and attributes in
>>     Python), why not try to use that instead of "@" (which in Python
>>     already identifies decorators) ?
> 
> I think the current feeling of the Python core team is against including non-ASCII characters in the
> language's keywords or operators. Even if that were not so, I would still recommend against it
> because it would be quite difficult to type. I don't know off-hand the key combination to do it on
> my native system, and it would change from system to system.

That's a fair argument. How about using the degree symbol instead: "?" ?

(A ? B).T == B.T ? A.T

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 14 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          15 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               26 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From robert.kern at gmail.com  Fri Mar 14 14:33:26 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 13:33:26 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53230221.1030501@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com>
Message-ID: <lfv0em$r68$1@ger.gmane.org>

On 2014-03-14 13:20, M.-A. Lemburg wrote:
> On 14.03.2014 12:25, Robert Kern wrote:
>> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>>
>>> I have some questions:
>>>
>>> 1. Since in math, the operator is usually spelt "?" (the center dot,
>>>      or "." but that's already reserved for methods and attributes in
>>>      Python), why not try to use that instead of "@" (which in Python
>>>      already identifies decorators) ?
>>
>> I think the current feeling of the Python core team is against including non-ASCII characters in the
>> language's keywords or operators. Even if that were not so, I would still recommend against it
>> because it would be quite difficult to type. I don't know off-hand the key combination to do it on
>> my native system, and it would change from system to system.
>
> That's a fair argument. How about using the degree symbol instead: "?" ?
>
> (A ? B).T == B.T ? A.T

That's still not ASCII, and I still don't know how to type it off the top of my 
head (US keyboard, OS X), though experimentation shows that Alt-0 does it easily 
enough. I don't know if it's universally that easy.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From masklinn at masklinn.net  Fri Mar 14 14:35:15 2014
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 14 Mar 2014 14:35:15 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
 <20140314135740.050bcd7f@fsol>
 <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io>
Message-ID: <599A7E12-670C-47E9-9DAD-F6DD3E0B1263@masklinn.net>

On 2014-03-14, at 14:01 , Donald Stufft <donald at stufft.io> wrote:
> The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. 

People don't "handle" anything, they learn the semantics of the @ symbol
in-context, whatever name they give it. That does not mean
non-english-speakers associate it with the sound "at" (they don't, in
my experience), even ignoring the dodginess of using "at" as a mnemonic
for "matrix", and @ as a shortcut for that (doubly so in Python where
@ already means "decorator")

From ethan at stoneleaf.us  Fri Mar 14 15:11:26 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 14 Mar 2014 07:11:26 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfuouo$v25$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
Message-ID: <53230E0E.9030707@stoneleaf.us>

On 03/14/2014 04:25 AM, Robert Kern wrote:
>
> Some more from real code:
>
> RSR = R.dot(var_beta.dot(R.T))
> RSR = R @ var_beta @ R.T
>
> xx_inv.dot(xeps.dot(xx_inv))
> xx_inv @ xeps @ xx_inv
>
> dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T)
> dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T)
>
> dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T))
> (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T)
>
> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']

Arrgggghhhhhhh!! Uncle! Uncle!

+1!

--
~Ethan~

From ethan at stoneleaf.us  Fri Mar 14 15:13:27 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 14 Mar 2014 07:13:27 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <5322EC7D.1020908@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com>
Message-ID: <53230E87.7070503@stoneleaf.us>

On 03/14/2014 04:48 AM, M.-A. Lemburg wrote:
> On 14.03.2014 12:25, Robert Kern wrote:
>>
>> RSR = R.dot(var_beta.dot(R.T))
>> RSR = R @ var_beta @ R.T
>>
>> xx_inv.dot(xeps.dot(xx_inv))
>> xx_inv @ xeps @ xx_inv
>>
>> dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T)
>> dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T)
>>
>> dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T))
>> (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T)
>>
>> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
>> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']
>
> This doesn't look very readable to me - the operator saves you
> a few parens in some situations, but as in the last example, it can
> also require adding new ones.

The difference being that grouping parens are easier to read and are less visual clutter than calling parens.

--
~Ethan~

From ericsnowcurrently at gmail.com  Fri Mar 14 15:42:39 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 14 Mar 2014 08:42:39 -0600
Subject: [Python-ideas] re-organizing the sys module
In-Reply-To: <lfu92v$8pb$1@ger.gmane.org>
References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com>
 <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org>
 <20140313160353.745d9659@anarchist.wooz.org>
 <lfu92v$8pb$1@ger.gmane.org>
Message-ID: <CALFfu7DYonXKqUdJww59g-taADS7xNB15-M48jjMcs1eiGtHmw@mail.gmail.com>

On Mar 14, 2014 12:55 AM, "Serhiy Storchaka" <storchaka at gmail.com> wrote:
> Perhaps. If sys attributes will be writable properties which updates _sys.

Precisely!

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/24f2aca0/attachment.html>

From robert.kern at gmail.com  Fri Mar 14 15:44:02 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 14:44:02 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53230221.1030501@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com>
Message-ID: <lfv4j2$hcd$1@ger.gmane.org>

On 2014-03-14 13:20, M.-A. Lemburg wrote:
> On 14.03.2014 12:25, Robert Kern wrote:
>> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>>
>>> I have some questions:
>>>
>>> 1. Since in math, the operator is usually spelt "?" (the center dot,
>>>      or "." but that's already reserved for methods and attributes in
>>>      Python), why not try to use that instead of "@" (which in Python
>>>      already identifies decorators) ?
>>
>> I think the current feeling of the Python core team is against including non-ASCII characters in the
>> language's keywords or operators. Even if that were not so, I would still recommend against it
>> because it would be quite difficult to type. I don't know off-hand the key combination to do it on
>> my native system, and it would change from system to system.
>
> That's a fair argument. How about using the degree symbol instead: "?" ?
>
> (A ? B).T == B.T ? A.T

Your point is taken, though. I do find these smaller symbols more readable and 
similar to standard mathematical notation than an @ sign, which is as big or 
bigger than most uppercase characters. Unfortunately, ASCII leaves us few 
single-character options.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From guido at python.org  Fri Mar 14 15:49:40 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Mar 2014 07:49:40 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfv4j2$hcd$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
Message-ID: <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>

Where on earth is the degree sign on my keyboard? (Don't answer. It's a
rhetorical question.)


On Fri, Mar 14, 2014 at 7:44 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On 2014-03-14 13:20, M.-A. Lemburg wrote:
>
>> On 14.03.2014 12:25, Robert Kern wrote:
>>
>>> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>>>
>>>  I have some questions:
>>>>
>>>> 1. Since in math, the operator is usually spelt "?" (the center dot,
>>>>      or "." but that's already reserved for methods and attributes in
>>>>      Python), why not try to use that instead of "@" (which in Python
>>>>      already identifies decorators) ?
>>>>
>>>
>>> I think the current feeling of the Python core team is against including
>>> non-ASCII characters in the
>>> language's keywords or operators. Even if that were not so, I would
>>> still recommend against it
>>> because it would be quite difficult to type. I don't know off-hand the
>>> key combination to do it on
>>> my native system, and it would change from system to system.
>>>
>>
>> That's a fair argument. How about using the degree symbol instead: "?" ?
>>
>> (A ? B).T == B.T ? A.T
>>
>
> Your point is taken, though. I do find these smaller symbols more readable
> and similar to standard mathematical notation than an @ sign, which is as
> big or bigger than most uppercase characters. Unfortunately, ASCII leaves
> us few single-character options.
>
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>   -- Umberto Eco
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/0971552f/attachment.html>

From p.f.moore at gmail.com  Fri Mar 14 15:53:08 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 14 Mar 2014 14:53:08 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53230E87.7070503@stoneleaf.us>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
Message-ID: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>

On 14 March 2014 14:13, Ethan Furman <ethan at stoneleaf.us> wrote:
>>>
>>> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
>>> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']
>>
>>
>> This doesn't look very readable to me - the operator saves you
>> a few parens in some situations, but as in the last example, it can
>> also require adding new ones.
>
>
> The difference being that grouping parens are easier to read and are less
> visual clutter than calling parens.

Personally, my biggest problem with all of these is that the @ sign is
a bit too big and bulky, so it's visually jarring.

But:

1. As the PEP mentions, there aren't many options available.
2. I trust the scientific community to choose something that they are
comfortable with (I'm only an interested outsider).
3. Ultimately, this is just bikeshedding.

One genuine question though - when the PEP was developed, were
multi-character operators like .* or <*> considered? A "rejected
alternative operator symbols" would be a useful addition to the PEP
(although it'd rob all us non-experts of the opportunity to bikeshed
:-))

On a related note, the @@ operator is visually dreadful (far too
heavy). While I see the */** analogy, and I appreciate that there's
few good options, I'd definitely like to see some evidence that it's
"the best of a bad lot" in the PEP.

Paul.

From njs at pobox.com  Fri Mar 14 15:55:06 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 14:55:06 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53230221.1030501@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com>
Message-ID: <CAPJVwBnSPYstfU+NhJAiCvNSXBK8HjxZ2zYV7=e0qDz3EY73=A@mail.gmail.com>

On Fri, Mar 14, 2014 at 1:20 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 14.03.2014 12:25, Robert Kern wrote:
>> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>>
>>> I have some questions:
>>>
>>> 1. Since in math, the operator is usually spelt "?" (the center dot,
>>>     or "." but that's already reserved for methods and attributes in
>>>     Python), why not try to use that instead of "@" (which in Python
>>>     already identifies decorators) ?
>>
>> I think the current feeling of the Python core team is against including non-ASCII characters in the
>> language's keywords or operators. Even if that were not so, I would still recommend against it
>> because it would be quite difficult to type. I don't know off-hand the key combination to do it on
>> my native system, and it would change from system to system.
>
> That's a fair argument. How about using the degree symbol instead: "?" ?
>
> (A ? B).T == B.T ? A.T

Well, obviously we can bikeshed this all day :-). For reasons the
draft PEP goes into in more detail, what we need is a symbol that
means "matrix rather than scalar/elementwise multiplication", and
there is no existing conventional symbol (either in math or
programming) that has that meaning -- \cdot, \degree, etc. don't have
that meaning any more than @ does. So we have to make something up.

My feeling is that @ is the least-bad option, and -- as the person
who's maybe been staring at these code samples the longest -- I found
that I got used to it pretty quickly. But I don't think debating is
going to lead to any "obviously best" answer, at some point we'll just
have to pick something. So maybe it's more useful to focus more on the
other parts of the proposal? :-)

-n

From robert.kern at gmail.com  Fri Mar 14 15:56:31 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 14:56:31 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
Message-ID: <lfv5af$roe$1@ger.gmane.org>

On 2014-03-14 14:49, Guido van Rossum wrote:
> Where on earth is the degree sign on my keyboard? (Don't answer. It's a
> rhetorical question.)

To be fair to Marc-Andre, there is a key for this right on his keyboard. It is 
we benighted Americans who lack it.

   http://en.wikipedia.org/wiki/German_keyboard_layout

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From joseph.martinot-lagarde at m4x.org  Fri Mar 14 15:54:02 2014
From: joseph.martinot-lagarde at m4x.org (Joseph Martinot-Lagarde)
Date: Fri, 14 Mar 2014 14:54:02 +0000 (UTC)
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com>
 <lfv4j2$hcd$1@ger.gmane.org>
Message-ID: <loom.20140314T154850-185@post.gmane.org>

Robert Kern <robert.kern at ...> writes:

> 
> On 2014-03-14 13:20, M.-A. Lemburg wrote:
> > On 14.03.2014 12:25, Robert Kern wrote:
> >> On 2014-03-14 10:16, M.-A. Lemburg wrote:
> >>
> >>> I have some questions:
> >>>
> >>> 1. Since in math, the operator is usually spelt "?" (the center dot,
> >>>      or "." but that's already reserved for methods and attributes in
> >>>      Python), why not try to use that instead of " <at> " (which in Python
> >>>      already identifies decorators) ?
> >>
> >> I think the current feeling of the Python core team is against
including non-ASCII characters in the
> >> language's keywords or operators. Even if that were not so, I would
still recommend against it
> >> because it would be quite difficult to type. I don't know off-hand the
key combination to do it on
> >> my native system, and it would change from system to system.
> >
> > That's a fair argument. How about using the degree symbol instead: "?" ?
> >
> > (A ? B).T == B.T ? A.T
> 
> Your point is taken, though. I do find these smaller symbols more readable
and 
> similar to standard mathematical notation than an  <at>  sign, which is as
big or 
> bigger than most uppercase characters. Unfortunately, ASCII leaves us few 
> single-character options.
> 

Putting aside tha ascii problem, ? is easily written using a AZERTY
keyboard. It is smaller and less convoluted than @ and looks like the
mathematical notation for function composition, which is similar to matrix
multiplication.
Still, not ascii and not displayed on every keyboard...



From njs at pobox.com  Fri Mar 14 16:00:02 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 15:00:02 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfuouo$v25$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
Message-ID: <CAPJVwBmRhWp_4KSuXL==cVAqp1-tQb1fw8u7GFjRtez3kuSP2w@mail.gmail.com>

On Fri, Mar 14, 2014 at 11:25 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2014-03-14 10:16, M.-A. Lemburg wrote:
>> Now since this is all about syntactic sugar, we also need to look at
>> some code examples:
[...]
>> c = A @ v
>> vs.
>> c = A ? v
>> vs.
>> c = A.dot(v)
>>
>> Hmm, even though I'd love to see matrix operators in Python,
>> I don't think they really add clarity to the syntax of matrix
>> calculations -  a bit disappointing, I must say :-(
>
> Some more from real code:
>
> RSR = R.dot(var_beta.dot(R.T))
> RSR = R @ var_beta @ R.T
>
> xx_inv.dot(xeps.dot(xx_inv))
> xx_inv @ xeps @ xx_inv
>
> dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) -
> 4/period*F2lower.dot(F2lower.T)
> dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @
> F2lower.T)
>
> dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) +
> FX.dot(Gi.dot(dFX2_dper.T))
> (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @
> dFX2_dper.T)
>
> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key']))
> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key']

For those skimming along, note that there's also a more detailed
example in the draft PEP, looking at a few different aspects of code
usability:
  https://github.com/numpy/numpy/pull/4351/files#diff-dd521035cec59cd5fb2e040f0703742bR230

-n

From mal at egenix.com  Fri Mar 14 16:16:11 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 14 Mar 2014 16:16:11 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfv5af$roe$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<5322D702.6040907@egenix.com>
 <lfuouo$v25$1@ger.gmane.org>	<53230221.1030501@egenix.com>
 <lfv4j2$hcd$1@ger.gmane.org>	<CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
 <lfv5af$roe$1@ger.gmane.org>
Message-ID: <53231D3B.6070908@egenix.com>

On 14.03.2014 15:56, Robert Kern wrote:
> On 2014-03-14 14:49, Guido van Rossum wrote:
>> Where on earth is the degree sign on my keyboard? (Don't answer. It's a
>> rhetorical question.)
> 
> To be fair to Marc-Andre, there is a key for this right on his keyboard. It is we benighted
> Americans who lack it.
> 
>   http://en.wikipedia.org/wiki/German_keyboard_layout

The keyboard is indeed where I got the idea from :-)

I didn't know that it's not available on US keyboards; only German
and French keyboard appear to have a key for it:

http://en.wikipedia.org/wiki/Degree_symbol#Keyboard_entry

Ok, enough bike shedding for today :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 14 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          15 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               26 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From flying-sheep at web.de  Fri Mar 14 16:29:02 2014
From: flying-sheep at web.de (Philipp A.)
Date: Fri, 14 Mar 2014 16:29:02 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53231D3B.6070908@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
 <lfv5af$roe$1@ger.gmane.org> <53231D3B.6070908@egenix.com>
Message-ID: <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com>

well, on linux, everyone has ???.

on american layouts, it?s AltGr+Shift+0, I think.

also i think it?s one reason why you americans love your funny imperial
units so much: you can?t even *type* ?23?C? if you?re not on linux.


2014-03-14 16:16 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:

> On 14.03.2014 15:56, Robert Kern wrote:
> > On 2014-03-14 14:49, Guido van Rossum wrote:
> >> Where on earth is the degree sign on my keyboard? (Don't answer. It's a
> >> rhetorical question.)
> >
> > To be fair to Marc-Andre, there is a key for this right on his keyboard.
> It is we benighted
> > Americans who lack it.
> >
> >   http://en.wikipedia.org/wiki/German_keyboard_layout
>
> The keyboard is indeed where I got the idea from :-)
>
> I didn't know that it's not available on US keyboards; only German
> and French keyboard appear to have a key for it:
>
> http://en.wikipedia.org/wiki/Degree_symbol#Keyboard_entry
>
> Ok, enough bike shedding for today :-)
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Mar 14 2014)
> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-03-29: PythonCamp 2014, Cologne, Germany ...          15 days to go
> 2014-04-09: PyCon 2014, Montreal, Canada ...               26 days to go
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/eae3fb81/attachment-0001.html>

From phd at phdru.name  Fri Mar 14 16:36:43 2014
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 14 Mar 2014 16:36:43 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
 <lfv5af$roe$1@ger.gmane.org> <53231D3B.6070908@egenix.com>
 <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com>
Message-ID: <20140314153643.GA14506@phdru.name>

On Fri, Mar 14, 2014 at 04:29:02PM +0100, "Philipp A." <flying-sheep at web.de> wrote:
> well, on linux, everyone has ???????.

   Well, not on my linux.

PS. I am -10**6 on any non-ascii character.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From ron3200 at gmail.com  Fri Mar 14 16:41:18 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 14 Mar 2014 10:41:18 -0500
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140314123239.GP12595@ando>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
Message-ID: <lfv7ug$tuv$1@ger.gmane.org>



On 03/14/2014 07:32 AM, Steven D'Aprano wrote:
>> >AFAIK, other languages sometimes use ".*" for matrix multiplication (as
>> >opposed to element-wise multiplication). Why not re-use that
>> >convention? Would that lead to parsing difficulties?
> "Syntax should not look like grit on Tim Peters' monitor."
>
> .* is, in my opinion, pretty ugly, and leads to even ugly
> extensions:
>
> a .* b  # matrix multiplication
> a .*= b  # in place matrix multiplication
> a .** b  # matrix exponentiation
> a .**= b  # in-place matrix exponentiation
>
> And it suffers from exactly the same lack of connection to matrix
> multiplication as @ does: there is nothing about . that spells "matrix".
>
> It also has the problem that it could cause confusion with vector dot
> product. Read out A .* B aloud and you get something that sounds like it
> might mean dot product, "A dot times B".

The best I can come up with is to use '^' instead of dot.

    a ^* b  # matrix multiplication
    a ^^ b  # matrix exponentiation

Also two ^ together looks like a M.  It's not a strong association, but it 
may help some people remember what it's for.   Or the second one could be 
spelled... ^**, but I think the ^^ is cleaner.


Question?  Is it possible for python to tell these apart at compile time?

 >>> a = 6
 >>> b = 4
 >>> c = 8
 >>> a^b
2
 >>> a ^b c
   File "<stdin>", line 1
     a ^b c
          ^
SyntaxError: invalid syntax

So that this later example does...

      a.__use__(self, "b", c)


The ^b is unrelated to the name b in the second case.  Or is this just not 
possible?


If it could be done, then ...

      a ^* b    -->     a.__use__(self, "*", b)
      a ^^ b    -->     a.__use__(self, "^", b)


Where the __use__ method is defined on a module level object.  (and not 
defined on builtin's as a general rule.)

     def __use__(self, s, other):
        if s == "*":
           ...           # do matrix multiply
           return result
        elif s == "^":
           ...           # do matrix exponentiation
           return result
        raise TypeError



Anyway to make this work nicely?

Cheers,
    Ron











From zachary.ware+pyideas at gmail.com  Fri Mar 14 16:46:25 2014
From: zachary.ware+pyideas at gmail.com (Zachary Ware)
Date: Fri, 14 Mar 2014 10:46:25 -0500
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
Message-ID: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>

Disclaimer: I am not a numpy (or similar) user and have only skimmed
this thread.  If I've simply missed something big, I can be safely
ignored :)

On Fri, Mar 14, 2014 at 9:53 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> Personally, my biggest problem with all of these is that the @ sign is
> a bit too big and bulky, so it's visually jarring.
>
> But:
>
> 1. As the PEP mentions, there aren't many options available.
> 2. I trust the scientific community to choose something that they are
> comfortable with (I'm only an interested outsider).
> 3. Ultimately, this is just bikeshedding.
>
> One genuine question though - when the PEP was developed, were
> multi-character operators like .* or <*> considered? A "rejected
> alternative operator symbols" would be a useful addition to the PEP
> (although it'd rob all us non-experts of the opportunity to bikeshed
> :-))
>
> On a related note, the @@ operator is visually dreadful (far too
> heavy). While I see the */** analogy, and I appreciate that there's
> few good options, I'd definitely like to see some evidence that it's
> "the best of a bad lot" in the PEP.

I agree with Paul, @/@@ just look scary as operators.  Here's a few
multi-character options I've come up with that I would like to see
shot down in flames before @/@@ are added to Python:

>< (for multiplication, not sure about exponentiation.  I only like it because it's the shortest thing I've come up with that looks somewhat like multiplication)
[*] / [**] ([] make me think 'matrix', but this might be confusing to
the parser)
|*| / |**| (pretty close to [], shouldn't confuse the parser)

The downside is that the inplace version of matrix exponentiation
would be a 5 character operator ([**]=), which I will freely admit is
not appealing, but it *looks* quite a lot nicer to me than "@@=" does.

-- 
Zach

From njs at pobox.com  Fri Mar 14 17:20:38 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 16:20:38 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
Message-ID: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com>

On Fri, Mar 14, 2014 at 3:46 PM, Zachary Ware
<zachary.ware+pyideas at gmail.com> wrote:
> I agree with Paul, @/@@ just look scary as operators.  Here's a few
> multi-character options I've come up with that I would like to see
> shot down in flames before @/@@ are added to Python:
>
>>< (for multiplication, not sure about exponentiation.  I only like it because it's the shortest thing I've come up with that looks somewhat like multiplication)
> [*] / [**] ([] make me think 'matrix', but this might be confusing to
> the parser)
> |*| / |**| (pretty close to [], shouldn't confuse the parser)

I really liked the [*] or |*| "footnote operator" idea, and got all
excited, until I tried it :-). Using the PEP's example of a linear
hypothesis test, compare:

    S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)

versus

    S = (H [*] beta - r).T [*] inv(H [*] V [*] H.T) [*] (H [*] beta - r)

    S = (H |*| beta - r).T |*| inv(H |*| V |*| H.T) |*| (H |*| beta - r)

A big part of the motivation for wanting an infix operator for this
kind of expression is that when using function syntax, the visual
clutter from all the required parentheses makes expressions hard to
parse by eye. [*] has this same problem of introducing visual nesting
and making it hard to pick out which vertical shapes are actually
grouping parens, and which are the operators. E.g., try counting how
many sub-expressions there are in each of these expressions -- I find
it much easier to pick out the 3 groups in the top example than in the
bottom two. Sort of a variant of leaning toothpick syndrome...

-n

From p.f.moore at gmail.com  Fri Mar 14 17:16:12 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 14 Mar 2014 16:16:12 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
Message-ID: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>

On 14 March 2014 15:46, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote:
> I agree with Paul, @/@@ just look scary as operators.  Here's a few
> multi-character options I've come up with that I would like to see
> shot down in flames before @/@@ are added to Python:

Just as a contrasting point, I've been reading this thread on gmail
with a proportional font. I went and looked at the PEP in a fixed
width font earlier, and the @ sign doesn't look anywhere near as bad
there.

Paul

From nathan at cmu.edu  Fri Mar 14 17:39:21 2014
From: nathan at cmu.edu (Nathan Schneider)
Date: Fri, 14 Mar 2014 12:39:21 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
Message-ID: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>

On Fri, Mar 14, 2014 at 12:16 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 14 March 2014 15:46, Zachary Ware <zachary.ware+pyideas at gmail.com>
> wrote:
> > I agree with Paul, @/@@ just look scary as operators.  Here's a few
> > multi-character options I've come up with that I would like to see
> > shot down in flames before @/@@ are added to Python:
>
> Just as a contrasting point, I've been reading this thread on gmail
> with a proportional font. I went and looked at the PEP in a fixed
> width font earlier, and the @ sign doesn't look anywhere near as bad
> there.
>
>
In Courier New:

    S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)

Still looks kind of bulky to me, because @ is the height and width of a
capital letter. How about prefixing * with an innocuous backtick?

    S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r)

That way no part of the operator extends to the baseline, so identifiers
and parentheses/brackets are visually well-separated from this as they are
with most other binary operators.

Nathan


> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/88d86d95/attachment-0001.html>

From alexander.belopolsky at gmail.com  Fri Mar 14 17:40:15 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 14 Mar 2014 12:40:15 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com>
Message-ID: <CAP7h-xaTjbTQ+gctt=pj87id7EeTUdYptWTb9_NN-5jn383z6g@mail.gmail.com>

On Fri, Mar 14, 2014 at 12:20 PM, Nathaniel Smith <njs at pobox.com> wrote:

> >>< (for multiplication, not sure about exponentiation.  I only like it
> because it's the shortest thing I've come up with that looks somewhat like
> multiplication)
> > [*] / [**] ([] make me think 'matrix', but this might be confusing to
> > the parser)
> > |*| / |**| (pretty close to [], shouldn't confuse the parser)
>
> I really liked the [*] or |*| "footnote operator" idea, and got all
> excited, until I tried it :-).


+1

Consider this:

A[*](B+C)

vs.

A[0]*(B+C)

the former looks like that later fat-fingered.

Given that in numpy it is no uncommon to have ":", ":,:", "...", or even
"()" to go inside [], [*] is more likely to be interpreted by those who
first see it as some new form of fancy indexing rather than matrix
multiplication.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/fecf6b1d/attachment.html>

From njs at pobox.com  Fri Mar 14 17:41:36 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 16:41:36 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
Message-ID: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>

On Fri, Mar 14, 2014 at 2:53 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> One genuine question though - when the PEP was developed, were
> multi-character operators like .* or <*> considered? A "rejected
> alternative operator symbols" would be a useful addition to the PEP
> (although it'd rob all us non-experts of the opportunity to bikeshed
> :-))
>
> On a related note, the @@ operator is visually dreadful (far too
> heavy). While I see the */** analogy, and I appreciate that there's
> few good options, I'd definitely like to see some evidence that it's
> "the best of a bad lot" in the PEP.

It's worth noting that @@ will probably see marginal use -- matrix
power and matrix inversion are not common operations in number
crunching code. Matrix inversion is very common in math formulas, but
on a computer you almost always want to use a fused invert+multiply
operation instead of inversion itself. My original draft didn't even
allow '@@ -1'; I added it at the request of the symbolic math guys. In
practice the main use case for @@ will probably be as a crutch for
beginners before they learn about better tools like numpy.linalg.solve
(which implements fused invert+multiply).

Anyway, I wrote some more text for the PEP, see what you think:

Rationale for specification details
===================================

Choice of operator
------------------

Why ``@`` instead of some other spelling?  There isn't any consensus
across other programming languages about how this operator should be
named [#matmul-other-langs]_; here we discuss the various options.

Restricting ourselves only to symbols present on US English keyboards,
the punctuation characters that don't already have a meaning in Python
expression context are: ``@``, backtick, ``$``, ``!``, and ``?``.  Of
these options, ``@`` is clearly the best; ``!`` and ``?`` are already
heavily freighted with inapplicable meanings in the programming
context, backtick has been banned from Python by BDFL pronouncement
(see PEP 3099), and ``$`` is uglier, even more dissimilar to ``*`` and
:math:`\cdot`, and has Perl/PHP baggage.  ``$`` is probably the
second-best option of these, though.

Symbols which are not present on US English keyboards start at a
significant disadvantage (having to spend 5 minutes at the beginning
of every numeric Python tutorial just going over keyboard layouts is
not a hassle anyone really wants).  Plus, even if we somehow overcame
the typing problem, it's not clear there are any that are actually
better than ``@``.  Some options that have been suggested include:

* U+00D7 MULTIPLICATION SIGN: ``A ? B``
* U+22C5 DOT OPERATOR: ``A ? B``
* U+2297 CIRCLED TIMES: ``A ? B``
* U+00B0 DEGREE: ``A ? B``

What we need, though, is an operator that means "matrix
multiplication, as opposed to scalar/elementwise multiplication".
There is no conventional symbol for these in mathematics or
programming, where these operations are usually distinguished by
context.  (And U+2297 CIRCLED TIMES is actually used conventionally to
mean exactly the opposite: elementwise multiplication -- the "Hadamard
product" -- as opposed to matrix multiplication).  ``@`` at least has
the virtue that it *looks* like a funny non-commutative operator; a
naive user who knows maths but not programming couldn't look at ``A *
B`` versus ``A ? B``, or ``A * B`` versus ``A ? B``, or ``A * B``
versus ``A ? B`` and guess which one is the usual multiplication, and
which one is the special case.

Finally, there is the option of using multi-character tokens.  Some
options:

* Matlab uses a ``.*`` operator.  Aside from being visually confusable
  with ``*``, this would be a terrible choice for us because in
  Matlab, ``*`` means matrix multiplication and ``.*`` means
  elementwise multiplication, so using ``.*`` for matrix
  multiplication would make us exactly backwards from what Matlab
  users expect.

* APL apparently used ``+.?``, which by combining a multi-character
  token, confusing attribute-access-like . syntax, and a unicode
  character, ranks somewhere below U+2603 SNOWMAN on our candidate
  list.  If we like the idea of combining addition and multiplication
  operators as being evocative of how matrix multiplication actually
  works, then something like ``+*`` could be used -- though this may
  be too easy to confuse with ``*+``, which is just multiplication
  combined with the unary ``+`` operator.

* PEP 211 suggested ``~*`` and ``~**``.  This has the downside that it
  sort of suggests that there is a unary ``*`` operator that is being
  combined with unary ``~``, but it could work.

* R uses ``%*%`` for matrix multiplication.  In R this forms part of a
  general extensible infix system in which all tokens of the form
  ``%foo%`` are user-defined binary operators.  We could steal the
  token without stealing the system.

* Some other plausible candidates that have been suggested: ``><`` (=
  ascii drawing of the multiplication sign ?); the footnote operators
  ``[*]`` and ``[**]`` or ``|*|`` and ``|**|`` (but when used in
  context, the use of vertical grouping symbols tends to recreate the
  nested parentheses visual clutter that was noted as one of the major
  downsides of the function syntax we're trying to get away from);
  ``^*`` and ``^^``.

So, it doesn't matter much, but ``@`` seems as good or better than any
of the alternatives:

* It's a friendly character that Pythoneers are already used to typing
  in decorators, but the decorator usage and the math expression
  usage are sufficiently dissimilar that it would be hard to confuse
  them in practice.

* It's widely accessible across keyboard layouts (and thanks to its
  use in email addresses, this is true even of weird keyboards like
  those in phones).

* It's round like ``*`` and :math:`\cdot`.

* The mATrices mnemonic is cute.

* The use of a single-character token reduces the line-noise effect,
  and makes ``@@`` possible, which is a nice bonus.

* The swirly shape is reminiscent of the simultaneous sweeps over rows
  and columns that define matrix multiplication

* Its asymmetry is evocative of its non-commutative nature.



-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From robert.kern at gmail.com  Fri Mar 14 17:46:27 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 16:46:27 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
Message-ID: <lfvboj$eab$1@ger.gmane.org>

On 2014-03-14 16:39, Nathan Schneider wrote:

> In Courier New:
>
>      S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
>
> Still looks kind of bulky to me, because @ is the height and width of a capital
> letter. How about prefixing * with an innocuous backtick?
>
>      S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r)
>
> That way no part of the operator extends to the baseline, so identifiers and
> parentheses/brackets are visually well-separated from this as they are with most
> other binary operators.

Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From p.f.moore at gmail.com  Fri Mar 14 17:47:39 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 14 Mar 2014 16:47:39 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
Message-ID: <CACac1F_hDzrp6vOi3RMqrVtR=sKGqBmTZGNqd5WsU=6Zead-Nw@mail.gmail.com>

On 14 March 2014 16:41, Nathaniel Smith <njs at pobox.com> wrote:
> Anyway, I wrote some more text for the PEP, see what you think:

>From my POV, this pretty much covers it. Thanks :-)

Paul

From guido at python.org  Fri Mar 14 17:47:52 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Mar 2014 09:47:52 -0700
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>

On Thu, Mar 13, 2014 at 6:59 PM, Nathaniel Smith <njs at pobox.com> wrote:
[...]

> PEP: XXXX
> Title: Dedicated infix operators for matrix multiplication and matrix power
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nathaniel J. Smith <njs at pobox.com>
> Status: Draft
> Type: Standards Track
> Python-Version: 3.5
> Content-Type: text/x-rst
> Created: 20-Feb-2014
> Post-History:
>

I still have to read this but I've assigned a PEP number. Henceforth this
will be known as PEP 465. So far I really like what I've read.

On the bikeshedding, if someone wants to start introducing Unicode
(non-ASCII) characters as operators they would have to propose a separate
PEP arguing the benefits of Unicode -- I am heavily opposed to doing that
ATM, for reasons that have already been brought up in the bikeshed.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/8e02f4ae/attachment-0001.html>

From alexander.belopolsky at gmail.com  Fri Mar 14 17:49:54 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 14 Mar 2014 12:49:54 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
Message-ID: <CAP7h-xbmavvO0BATr8prT0UtFYoeAoHGea0wvFy5Y0zonaAq9w@mail.gmail.com>

On Fri, Mar 14, 2014 at 12:41 PM, Nathaniel Smith <njs at pobox.com> wrote:

> (And U+2297 CIRCLED TIMES is actually used conventionally to
> mean exactly the opposite: elementwise multiplication -- the "Hadamard
> product" -- as opposed to matrix multiplication).
>

It's actually worse: CIRCLED TIMES is commonly used for tensor or outer
product that is opposite to the proposed inner product meaning of vec @ vec.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/ee23b8ca/attachment.html>

From robert.kern at gmail.com  Fri Mar 14 17:54:55 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 16:54:55 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com>
Message-ID: <lfvc8f$luc$1@ger.gmane.org>

On 2014-03-14 16:41, Nathaniel Smith wrote:

> * The mATrices mnemonic is cute.

I recommend dropping this mnemonic from the PEP. It has been pointed out that 
the mnemonic does not cross languages[1], and even to my Anglo ears is not 
especially evocative.

[1] Python, like most programming languages, is unabashedly Anglocentric in its 
keywords, standard library and documentation, so some amount of Anglocentrism in 
new language features is unavoidable. My point is just that an Anglocentric 
mnemonic isn't really helping the argument for the PEP. If the PEP needs a 
mnemonic to survive, this is not going to cut it. If the PEP doesn't need a 
mnemonic, as I believe, it's best left out.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From zachary.ware+pyideas at gmail.com  Fri Mar 14 18:04:24 2014
From: zachary.ware+pyideas at gmail.com (Zachary Ware)
Date: Fri, 14 Mar 2014 12:04:24 -0500
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com>
Message-ID: <CAKJDb-M-pWbsHBka=3NTgC7u7S2_L6+v4p-e+fNXBAO-hReGbA@mail.gmail.com>

On Fri, Mar 14, 2014 at 11:20 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Fri, Mar 14, 2014 at 3:46 PM, Zachary Ware
> <zachary.ware+pyideas at gmail.com> wrote:
>> Here's a few
>> multi-character options I've come up with that I would like to see
>> shot down in flames before @/@@ are added to Python:

<snip>

> A big part of the motivation for wanting an infix operator for this
> kind of expression is that when using function syntax, the visual
> clutter from all the required parentheses makes expressions hard to
> parse by eye. [*] has this same problem of introducing visual nesting
> and making it hard to pick out which vertical shapes are actually
> grouping parens, and which are the operators. E.g., try counting how
> many sub-expressions there are in each of these expressions -- I find
> it much easier to pick out the 3 groups in the top example than in the
> bottom two. Sort of a variant of leaning toothpick syndrome...

This, plus added text for the PEP in a later message, is enough of a
fire for me :).  Thank you.

I'll also admit that, again like Paul, I was reading in a proportional
font.  @/@@ is significantly less bad in fixed width, but I still
don't love it.  I certainly won't stand in the way if it's determined
to be the least bad option, though.

-- 
Zach

From ron3200 at gmail.com  Fri Mar 14 18:05:43 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 14 Mar 2014 12:05:43 -0500
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfv7ug$tuv$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
 <lfv7ug$tuv$1@ger.gmane.org>
Message-ID: <lfvcsp$t4s$1@ger.gmane.org>



On 03/14/2014 10:41 AM, Ron Adam wrote:
>
>
> On 03/14/2014 07:32 AM, Steven D'Aprano wrote:
>>> >AFAIK, other languages sometimes use ".*" for matrix multiplication (as
>>> >opposed to element-wise multiplication). Why not re-use that
>>> >convention? Would that lead to parsing difficulties?
>> "Syntax should not look like grit on Tim Peters' monitor."
>>
>> .* is, in my opinion, pretty ugly, and leads to even ugly
>> extensions:
>>
>> a .* b  # matrix multiplication
>> a .*= b  # in place matrix multiplication
>> a .** b  # matrix exponentiation
>> a .**= b  # in-place matrix exponentiation
>>
>> And it suffers from exactly the same lack of connection to matrix
>> multiplication as @ does: there is nothing about . that spells "matrix".
>>
>> It also has the problem that it could cause confusion with vector dot
>> product. Read out A .* B aloud and you get something that sounds like it
>> might mean dot product, "A dot times B".
>
> The best I can come up with is to use '^' instead of dot.
>
>     a ^* b  # matrix multiplication
>     a ^^ b  # matrix exponentiation
>
> Also two ^ together looks like a M.  It's not a strong association, but it
> may help some people remember what it's for.   Or the second one could be
> spelled... ^**, but I think the ^^ is cleaner.
>
>
> Question?  Is it possible for python to tell these apart at compile time?
>
>  >>> a = 6
>  >>> b = 4
>  >>> c = 8
>  >>> a^b
> 2
>  >>> a ^b c
>    File "<stdin>", line 1
>      a ^b c
>           ^
> SyntaxError: invalid syntax
>
> So that this later example does...
>
>       a.__use__(self, "b", c)
>
>
> The ^b is unrelated to the name b in the second case.  Or is this just not
> possible?

One more comment, if this idea is limited to only allow symbols that can't 
be seen in identifiers, then the conflict goes away also.  That's probably 
not an unreasonable restriction.


> If it could be done, then ...
>
>       a ^* b    -->     a.__use__(self, "*", b)
>       a ^^ b    -->     a.__use__(self, "^", b)
>
>
> Where the __use__ method is defined on a module level object.  (and not
> defined on builtin's as a general rule.)
>
>      def __use__(self, s, other):
>         if s == "*":
>            ...           # do matrix multiply
>            return result
>         elif s == "^":
>            ...           # do matrix exponentiation
>            return result
>         raise TypeError
>
>
>
> Anyway to make this work nicely?

I would of liked it to be more general, but this could work for just 
symbols.  It could also be extended to allow ^'string', at some later date.

Cheers,
     Ron


From solipsis at pitrou.net  Fri Mar 14 18:08:11 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 18:08:11 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <lfvboj$eab$1@ger.gmane.org>
Message-ID: <20140314180811.35d7cdf6@fsol>

On Fri, 14 Mar 2014 16:46:27 +0000
Robert Kern <robert.kern at gmail.com> wrote:
> On 2014-03-14 16:39, Nathan Schneider wrote:
> 
> > In Courier New:
> >
> >      S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
> >
> > Still looks kind of bulky to me, because @ is the height and width of a capital
> > letter. How about prefixing * with an innocuous backtick?
> >
> >      S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r)
> >
> > That way no part of the operator extends to the baseline, so identifiers and
> > parentheses/brackets are visually well-separated from this as they are with most
> > other binary operators.
> 
> Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test.

Not only grit, but the problem with the backtick is that it can look
very close to a straight apostrophe.

I am personally not fond of @, but I find it ok in that it is
distinctive enough without being terribly ugly.

Regards

Antoine.



From njs at pobox.com  Fri Mar 14 18:12:12 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 17:12:12 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140314180811.35d7cdf6@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <lfvboj$eab$1@ger.gmane.org> <20140314180811.35d7cdf6@fsol>
Message-ID: <CAPJVwBmQOALxWgsGiEdZJ7W3BoK4+K9mrHt5e0ZTq7Ln5aRU+Q@mail.gmail.com>

On Fri, Mar 14, 2014 at 5:08 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 14 Mar 2014 16:46:27 +0000
> Robert Kern <robert.kern at gmail.com> wrote:
>> On 2014-03-14 16:39, Nathan Schneider wrote:
>>
>> > In Courier New:
>> >
>> >      S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
>> >
>> > Still looks kind of bulky to me, because @ is the height and width of a capital
>> > letter. How about prefixing * with an innocuous backtick?
>> >
>> >      S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r)
>> >
>> > That way no part of the operator extends to the baseline, so identifiers and
>> > parentheses/brackets are visually well-separated from this as they are with most
>> > other binary operators.
>>
>> Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test.
>
> Not only grit, but the problem with the backtick is that it can look
> very close to a straight apostrophe.

Backtick has in fact been formally banned from py3 by BDFL
pronouncement (for this reason):
   https://mail.python.org/pipermail/python-ideas/2007-January/000054.html
   http://legacy.python.org/dev/peps/pep-3099/

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From mertz at gnosis.cx  Fri Mar 14 18:08:57 2014
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 14 Mar 2014 10:08:57 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <loom.20140314T154850-185@post.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <loom.20140314T154850-185@post.gmane.org>
Message-ID: <CAEbHw4YbhnV+8roPKsO7zhBtv3GyYnP1Zjwe4TDM=Qg1y0LMTA@mail.gmail.com>

I think the fundamental issue with the degree sign is quite simply that it
is *not-ASCII*.  If we are willing to expand the syntax of Python to
include other characters, then we should just use Unicode Character 'DOT
OPERATOR' (U+22C5), which is actually the *exact* correct thing, not just
"something that looks a bit similar."

If we are worried about "stuff that's easy to enter on the keyboard",
there's no reason AZERTY is necessarily more relevant than Dubeolsik or
JCUKEN.  And if we aim for "something that looks similar" we probably have
lots of options on some keyboard layout in the world.




On Fri, Mar 14, 2014 at 7:54 AM, Joseph Martinot-Lagarde <
joseph.martinot-lagarde at m4x.org> wrote:

> Robert Kern <robert.kern at ...> writes:
>
> >
> > On 2014-03-14 13:20, M.-A. Lemburg wrote:
> > > On 14.03.2014 12:25, Robert Kern wrote:
> > >> On 2014-03-14 10:16, M.-A. Lemburg wrote:
> > >>
> > >>> I have some questions:
> > >>>
> > >>> 1. Since in math, the operator is usually spelt "?" (the center dot,
> > >>>      or "." but that's already reserved for methods and attributes in
> > >>>      Python), why not try to use that instead of " <at> " (which in
> Python
> > >>>      already identifies decorators) ?
> > >>
> > >> I think the current feeling of the Python core team is against
> including non-ASCII characters in the
> > >> language's keywords or operators. Even if that were not so, I would
> still recommend against it
> > >> because it would be quite difficult to type. I don't know off-hand the
> key combination to do it on
> > >> my native system, and it would change from system to system.
> > >
> > > That's a fair argument. How about using the degree symbol instead: "?"
> ?
> > >
> > > (A ? B).T == B.T ? A.T
> >
> > Your point is taken, though. I do find these smaller symbols more
> readable
> and
> > similar to standard mathematical notation than an  <at>  sign, which is
> as
> big or
> > bigger than most uppercase characters. Unfortunately, ASCII leaves us few
> > single-character options.
> >
>
> Putting aside tha ascii problem, ? is easily written using a AZERTY
> keyboard. It is smaller and less convoluted than @ and looks like the
> mathematical notation for function composition, which is similar to matrix
> multiplication.
> Still, not ascii and not displayed on every keyboard...
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/b0793869/attachment.html>

From solipsis at pitrou.net  Fri Mar 14 18:10:31 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 18:10:31 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com>
 <lfv5af$roe$1@ger.gmane.org>
Message-ID: <20140314181031.507fb158@fsol>

On Fri, 14 Mar 2014 14:56:31 +0000
Robert Kern <robert.kern at gmail.com> wrote:
> On 2014-03-14 14:49, Guido van Rossum wrote:
> > Where on earth is the degree sign on my keyboard? (Don't answer. It's a
> > rhetorical question.)
> 
> To be fair to Marc-Andre, there is a key for this right on his keyboard. It is 
> we benighted Americans who lack it.
> 
>    http://en.wikipedia.org/wiki/German_keyboard_layout

Just for the record, on a French keyboard the combo is Shift+).

Regards

Antoine.






From solipsis at pitrou.net  Fri Mar 14 18:11:33 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 14 Mar 2014 18:11:33 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org>
 <loom.20140314T154850-185@post.gmane.org>
Message-ID: <20140314181133.6be48954@fsol>

On Fri, 14 Mar 2014 14:54:02 +0000 (UTC)
Joseph Martinot-Lagarde
<joseph.martinot-lagarde at m4x.org> wrote:
> 
> Putting aside tha ascii problem, ? is easily written using a AZERTY
> keyboard. It is smaller and less convoluted than @ and looks like the
> mathematical notation for function composition, which is similar to matrix
> multiplication.

Perhaps we should keep it for the temperature literals that will appear
in Python 4, though :-)

Regards

Antoine.



From guido at python.org  Fri Mar 14 18:53:01 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Mar 2014 10:53:01 -0700
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
Message-ID: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>

I have now read the PEP, and I think it's good. I think it's a waste of
time to keep bikeshedding on the choice of operator -- @ is the best
compromise. I do have a few specific notes:

- Right associativity is not unheard of in Python. E.g. **. If you
  think that for other reasons @ should be right associative, don't
  let Python's tradition stop you. But then you need to decide which
  of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or
  (a*b)@c? And if you choose the latter, it follows that a at b*c means
  a@(b*c) -- is that okay? (And similar examples exist for the other
  choice.)

- Did you consider a duck-typing (is that the word?) attribute?
  E.g. a*b is elementwise multiplication; a.M*b must be used for
  matrix multiplication.  (Your use of .T as "transpose" made me think
  of this.)  Of course the question is, can you get those packages
  that currently use * for matrix multiply to comply? (I don't consider
  this a serious counter-proposal. But you list a bunch of rejected
  alternatives; this could be in that list.

- Is @@ really necessary?  It seems you are adding it mostly because
  it's cute and because of the parallel with **, not because it is
  actually important enough to add new syntax.  And then later you use
  it as an argument for @, which seems a bit circular.  Also, if we
  were to make @ right-associative, the parallel with ** is already
  imperfect.

- For better counts of usages, perhaps Sourcegraph.com might help?  It
  is a source code query engine that has a Python parser and (limited)
  type inference built in (also separately available as pysonar on
  github IIRC). To be clear, I don't need more numbers to be convinced.

Once we've decided on associativity and @@, I'm ready to accept.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c888ea8b/attachment-0001.html>

From ronan.lamy at gmail.com  Fri Mar 14 19:40:49 2014
From: ronan.lamy at gmail.com (Ronan Lamy)
Date: Fri, 14 Mar 2014 18:40:49 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <53234D31.20300@gmail.com>

I have a few remarks, in random order:

* The PEP is not just about the multiplication of matrices. The 
matrix-vector and vector-vector semantics are required in order to get 
nice expressions. This should probably be mentioned more prominently.

* The complicated semantics for higher-dimensional arrays are 
effectively incompatible with the matrix-vector and vector-vector cases: 
for instance, writing the product of a list of matrices by a list of 
vectors requires something like "mats @ vecs[..., np.newaxis]". In other 
words, A @ B is AFAICT nearly equivalent to numpy.dot(A, B), therefore 
people who cannot use numpy.dot() today will not be able to use "@" either.

*  A big part of the problem with np.matrix is that it subclasses 
np.ndarray but has a completely different __mul__(). IMHO, the problems 
with it are a good argument for the importance of the Liskov 
substitution principle, but say rather little about the viability of a 
correctly implemented matrix type.

From antony.lee at berkeley.edu  Fri Mar 14 19:41:11 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Fri, 14 Mar 2014 11:41:11 -0700
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
Message-ID: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>

A simple suggestion: what about defining the special methods as __at__,
__rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)?  This make
the operator more "neutral" -- it make it sound less "wrong" to overload it
for something else that matrix multiplication (which is certainly an
important topic, but even though I use numpy quite a lot, I don't actually
use it at all for linear algebra (in fact I use elementwise multiplications
much more often) -- so counting imports of numpy is a somewhat biaised
metric for counting users of matrix multiplication).

Antony


2014-03-14 10:53 GMT-07:00 Guido van Rossum <guido at python.org>:

> I have now read the PEP, and I think it's good. I think it's a waste of
> time to keep bikeshedding on the choice of operator -- @ is the best
> compromise. I do have a few specific notes:
>
> - Right associativity is not unheard of in Python. E.g. **. If you
>   think that for other reasons @ should be right associative, don't
>   let Python's tradition stop you. But then you need to decide which
>   of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or
>   (a*b)@c? And if you choose the latter, it follows that a at b*c means
>   a@(b*c) -- is that okay? (And similar examples exist for the other
>   choice.)
>
> - Did you consider a duck-typing (is that the word?) attribute?
>   E.g. a*b is elementwise multiplication; a.M*b must be used for
>   matrix multiplication.  (Your use of .T as "transpose" made me think
>   of this.)  Of course the question is, can you get those packages
>   that currently use * for matrix multiply to comply? (I don't consider
>   this a serious counter-proposal. But you list a bunch of rejected
>   alternatives; this could be in that list.
>
> - Is @@ really necessary?  It seems you are adding it mostly because
>   it's cute and because of the parallel with **, not because it is
>   actually important enough to add new syntax.  And then later you use
>   it as an argument for @, which seems a bit circular.  Also, if we
>   were to make @ right-associative, the parallel with ** is already
>   imperfect.
>
> - For better counts of usages, perhaps Sourcegraph.com might help?  It
>   is a source code query engine that has a Python parser and (limited)
>   type inference built in (also separately available as pysonar on
>   github IIRC). To be clear, I don't need more numbers to be convinced.
>
> Once we've decided on associativity and @@, I'm ready to accept.
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/ea723753/attachment.html>

From guido at python.org  Fri Mar 14 19:53:47 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Mar 2014 11:53:47 -0700
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
Message-ID: <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com>

On Fri, Mar 14, 2014 at 11:41 AM, Antony Lee <antony.lee at berkeley.edu>wrote:

> A simple suggestion: what about defining the special methods as __at__,
> __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)?  This make
> the operator more "neutral" -- it make it sound less "wrong" to overload it
> for something else that matrix multiplication (which is certainly an
> important topic, but even though I use numpy quite a lot, I don't actually
> use it at all for linear algebra (in fact I use elementwise multiplications
> much more often) -- so counting imports of numpy is a somewhat biaised
> metric for counting users of matrix multiplication).
>

Ratatatatatat! :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/7ceda45b/attachment.html>

From phd at phdru.name  Fri Mar 14 20:00:09 2014
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 14 Mar 2014 20:00:09 +0100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com>
Message-ID: <20140314190009.GA19829@phdru.name>

On Fri, Mar 14, 2014 at 11:53:47AM -0700, Guido van Rossum <guido at python.org> wrote:
> On Fri, Mar 14, 2014 at 11:41 AM, Antony Lee <antony.lee at berkeley.edu>wrote:
> > A simple suggestion: what about defining the special methods as __at__,
> > __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)?  This make
> > the operator more "neutral" -- it make it sound less "wrong" to overload it
> > for something else that matrix multiplication (which is certainly an
> > important topic, but even though I use numpy quite a lot, I don't actually
> > use it at all for linear algebra (in fact I use elementwise multiplications
> > much more often) -- so counting imports of numpy is a somewhat biaised
> > metric for counting users of matrix multiplication).
> 
> Ratatatatatat! :-)

   Is this the sound made by matrices being multiplied? ;-)

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From robert.kern at gmail.com  Fri Mar 14 20:41:10 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Mar 2014 19:41:10 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
Message-ID: <lfvm08$bjg$1@ger.gmane.org>

On 2014-03-14 17:53, Guido van Rossum wrote:
> I have now read the PEP, and I think it's good. I think it's a waste of time to
> keep bikeshedding on the choice of operator -- @ is the best compromise. I do
> have a few specific notes:
>
> - Right associativity is not unheard of in Python. E.g. **. If you
>    think that for other reasons @ should be right associative, don't
>    let Python's tradition stop you. But then you need to decide which
>    of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or
>    (a*b)@c? And if you choose the latter, it follows that a at b*c means
>    a@(b*c) -- is that okay? (And similar examples exist for the other
>    choice.)

I *think* either works out fine in practice, but I have a preference for @ 
binding tighter than *. `scalar * matrix @ vector` does fewer flops that way, 
and most current expressions are written with this binding anyways:
`scalar * np.dot(matrix, vector)`. It just feels right to me.

> - Did you consider a duck-typing (is that the word?) attribute?

Facade?

>    E.g. a*b is elementwise multiplication; a.M*b must be used for
>    matrix multiplication.  (Your use of .T as "transpose" made me think
>    of this.)  Of course the question is, can you get those packages
>    that currently use * for matrix multiply to comply? (I don't consider
>    this a serious counter-proposal. But you list a bunch of rejected
>    alternatives; this could be in that list.

It seems to me that the left-associativity of * makes this less useful than a 
dedicated operator if we consider chains of matrix multiplications.

   A.M * B.M * C == (A.M * B.M) * C

We probably could make the rule that if the right-operand is one of these 
matmul-facades, then the result should also be a matmul-facade, but otherwise 
would be a plain numpy array. That would take care of this case, but it's not 
obvious to me that it's unambiguously the right thing to do in all cases.

> - Is @@ really necessary?  It seems you are adding it mostly because
>    it's cute and because of the parallel with **, not because it is
>    actually important enough to add new syntax.  And then later you use
>    it as an argument for @, which seems a bit circular.  Also, if we
>    were to make @ right-associative, the parallel with ** is already
>    imperfect.

In my personal experience, I would use a matrix power operator much less than a 
matrix multiplication operator. I, at least, am content to continue to use a 
function call for that.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From skip at pobox.com  Fri Mar 14 20:42:26 2014
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 14 Mar 2014 14:42:26 -0500
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>

On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote:
> You'll notice that this draft is rather more developed than the
> average first-round PEP posting

Has this acquired a PEP number yet? It seems far enough along that it
really ought to be more broadly visible.

Skip

From phd at phdru.name  Fri Mar 14 20:53:41 2014
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 14 Mar 2014 20:53:41 +0100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>
Message-ID: <20140314195341.GB19829@phdru.name>

On Fri, Mar 14, 2014 at 02:42:26PM -0500, Skip Montanaro <skip at pobox.com> wrote:
> On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote:
> > You'll notice that this draft is rather more developed than the
> > average first-round PEP posting
> 
> Has this acquired a PEP number yet? It seems far enough along that it
> really ought to be more broadly visible.

   https://mail.python.org/pipermail/python-ideas/2014-March/027099.html

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From zachary.ware+pyideas at gmail.com  Fri Mar 14 20:54:11 2014
From: zachary.ware+pyideas at gmail.com (Zachary Ware)
Date: Fri, 14 Mar 2014 14:54:11 -0500
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>
Message-ID: <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com>

On Fri, Mar 14, 2014 at 2:42 PM, Skip Montanaro <skip at pobox.com> wrote:
> On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> You'll notice that this draft is rather more developed than the
>> average first-round PEP posting
>
> Has this acquired a PEP number yet? It seems far enough along that it
> really ought to be more broadly visible.

Guido committed it to peps earlier:
http://hg.python.org/peps/file/tip/pep-0465.txt

It should be available at http://www.python.org/dev/peps/pep-0465/ soon.

--
Zach

From skip at pobox.com  Fri Mar 14 21:04:11 2014
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 14 Mar 2014 15:04:11 -0500
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com>
 <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com>
Message-ID: <CANc-5Uy4GvtsduvYxWmniHiqRueLHpz8esn1quzHJYveH9JKWA@mail.gmail.com>

On Fri, Mar 14, 2014 at 2:54 PM, Zachary Ware
<zachary.ware+pyideas at gmail.com> wrote:
> Guido committed it to peps earlier:
> http://hg.python.org/peps/file/tip/pep-0465.txt
>
> It should be available at http://www.python.org/dev/peps/pep-0465/ soon.

Thanks. I was just searching the PEP 0 page and didn't see it.

Skip

From ncoghlan at gmail.com  Fri Mar 14 21:15:51 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Mar 2014 06:15:51 +1000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
Message-ID: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>

On 15 Mar 2014 04:42, "Antony Lee" <antony.lee at berkeley.edu> wrote:
>
> A simple suggestion: what about defining the special methods as __at__,
__rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)?  This make
the operator more "neutral" -- it make it sound less "wrong" to overload it
for something else that matrix multiplication (which is certainly an
important topic, but even though I use numpy quite a lot, I don't actually
use it at all for linear algebra (in fact I use elementwise multiplications
much more often) -- so counting imports of numpy is a somewhat biaised
metric for counting users of matrix multiplication).

The method name for "*" is "__mul__" rather than "__star__", and so on for
the other types. Naming the magic methods for their intended semantics
rather than their syntax is an established pattern.

A few other miscellaneous comments:

- nice work on the PEP Nathaniel!
- as with others, "@" as the operator doesn't thrill me, but I also think
it crosses the threshold of "good enough given the constraints"
- the PEP should probably recommend adding an "operator.matmul" function, a
"PyObject_MatrixMultiply" C API and consider whether or not the new special
method should be given a C level type slot.

Cheers,
Nick.

>
> Antony
>
>
> 2014-03-14 10:53 GMT-07:00 Guido van Rossum <guido at python.org>:
>>
>> I have now read the PEP, and I think it's good. I think it's a waste of
time to keep bikeshedding on the choice of operator -- @ is the best
compromise. I do have a few specific notes:
>>
>> - Right associativity is not unheard of in Python. E.g. **. If you
>>   think that for other reasons @ should be right associative, don't
>>   let Python's tradition stop you. But then you need to decide which
>>   of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or
>>   (a*b)@c? And if you choose the latter, it follows that a at b*c means
>>   a@(b*c) -- is that okay? (And similar examples exist for the other
>>   choice.)
>>
>> - Did you consider a duck-typing (is that the word?) attribute?
>>   E.g. a*b is elementwise multiplication; a.M*b must be used for
>>   matrix multiplication.  (Your use of .T as "transpose" made me think
>>   of this.)  Of course the question is, can you get those packages
>>   that currently use * for matrix multiply to comply? (I don't consider
>>   this a serious counter-proposal. But you list a bunch of rejected
>>   alternatives; this could be in that list.
>>
>> - Is @@ really necessary?  It seems you are adding it mostly because
>>   it's cute and because of the parallel with **, not because it is
>>   actually important enough to add new syntax.  And then later you use
>>   it as an argument for @, which seems a bit circular.  Also, if we
>>   were to make @ right-associative, the parallel with ** is already
>>   imperfect.
>>
>> - For better counts of usages, perhaps Sourcegraph.com might help?  It
>>   is a source code query engine that has a Python parser and (limited)
>>   type inference built in (also separately available as pysonar on
>>   github IIRC). To be clear, I don't need more numbers to be convinced.
>>
>> Once we've decided on associativity and @@, I'm ready to accept.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/317338b9/attachment-0001.html>

From storchaka at gmail.com  Fri Mar 14 21:17:58 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 14 Mar 2014 22:17:58 +0200
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <lfvo4j$43j$1@ger.gmane.org>

14.03.14 03:59, Nathaniel Smith ???????(??):
> PEP: XXXX
> Title: Dedicated infix operators for matrix multiplication and matrix power

This is about matrix multiplication and matrix power. But what about 
matrix division? It is needed to distinguish elementwise division from 
left and right matrix divisions. And what about concatenation? Python 
lists and tuples use ``+`` for this, but NumPy arrayse use ``+`` for 
bitwise addition.

Matrix multiplication is only defined on 2d arrays (matrices). But why 
not on arbitrary tensors (except scalars)?



From ncoghlan at gmail.com  Fri Mar 14 21:46:26 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Mar 2014 06:46:26 +1000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <lfvo4j$43j$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <lfvo4j$43j$1@ger.gmane.org>
Message-ID: <CADiSq7dLVn8AJeLFLw8iTHZaitBJ_v7BbYnV7yJDUr9ngyXnxA@mail.gmail.com>

On 15 Mar 2014 06:18, "Serhiy Storchaka" <storchaka at gmail.com> wrote:
>
> 14.03.14 03:59, Nathaniel Smith ???????(??):
>
>> PEP: XXXX
>> Title: Dedicated infix operators for matrix multiplication and matrix
power
>
>
> This is about matrix multiplication and matrix power. But what about
matrix division? It is needed to distinguish elementwise division from left
and right matrix divisions. And what about concatenation? Python lists and
tuples use ``+`` for this, but NumPy arrayse use ``+`` for bitwise addition.
>
> Matrix multiplication is only defined on 2d arrays (matrices). But why
not on arbitrary tensors (except scalars)?

These are covered in the PEP - the relatively simple "just a new matrix
multiplication operator" proposal is due to the fact that, after long
experience, matrix multiplication is the only operator the numeric
community really feel they miss in Python compared to working in more
special purpose languages like Matlab or R.

Everything else can continue to use appropriately named methods and
functions, just as it does today.

That does raise a question for me though: does Julia have syntax for matrix
multiplication? If so, what does that look like?

Cheers,
Nick.

>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/7a0768f0/attachment.html>

From skip at pobox.com  Fri Mar 14 21:51:10 2014
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 14 Mar 2014 15:51:10 -0500
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
Message-ID: <CANc-5Uw+piJx9hFj4VU0h2wxNAm3DiCWWhKFjzk=RNRiKp3XYQ@mail.gmail.com>

On Fri, Mar 14, 2014 at 3:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The method name for "*" is "__mul__" rather than "__star__"

Yes, but
?just think. W
ith a couple extra "@"s we could
?start defining
 __ratatatat__<http://www.merriam-webster.com/audio.php?file=ratata02&word=rat-a-tat-tat&text=%5C%CB%8Cra-t%C9%99-%CB%8Cta(t)-%CB%88tat%5C>
? methods.?
?? :-)

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/d12ac5b4/attachment.html>

From anthony at xtfx.me  Fri Mar 14 22:03:17 2014
From: anthony at xtfx.me (C Anthony Risinger)
Date: Fri, 14 Mar 2014 16:03:17 -0500
Subject: [Python-ideas] Function-like Modules (better control/introspection)
Message-ID: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>

hello,

could we make modules work more like this...

https://gist.github.com/xtfxme/9556806

tl;dr: __code__ object available (potentially a __self__ object), reload by
calling, makes working with modules more natural/safe!

ENCLOSED: modules-next.py

-- 

C Anthony

#----------------------------------------------------------------------------------------(
modules-next.py )
# encoding: utf-8
#
# modules-next
#
# C Anthony Risinger


import sys


m_name = 'modules_next'
m_source = r"""\
if __name__ == '{0}':
    from . import __call__ as reexec
    from . import __code__ as code
    from . import __dict__ as ns
    print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
    __name__ = 'modules_next_reloaded'
    reexec()
    print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
else:
    print('!!! __name__ == {{0}}\n'.format(__name__))

def throw():
    raise Exception(__name__)
""".format(m_name)
m_code = compile(m_source, m_name, 'exec')
m_dict = {
    '__file__': __file__ + '.next',
    '__name__': m_name,
    '__path__': list(),
    '__doc__': None,
    }
#...bind namespace/functions
module = eval('lambda: None', m_dict)
#...automatically expose as attributes
module.__dict__ = module.__globals__
#...must be set twice, __code__ sees __dict__, but we see descriptor
module.__name__ = m_name
#...redefine the function body!
module.__code__ = m_code
#...yay! we haz legitimate module!
sys.modules[m_name] = module


print('--* importing...')
import modules_next
print('>>> {0}: {1}\n{2}\n'.format(
    __name__,
    modules_next.__code__,
    modules_next.__dict__.keys(),
    ))

print('--* importing (for real this time)...')
#...import/reload by calling our func-y-module
module()
print('>>> {0}: {1}\n{2}\n'.format(
    __name__,
    modules_next.__code__,
    modules_next.__dict__.keys(),
    ))


print('--* throwing...')
modules_next.throw()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c8c8bb6b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: modules-next.py
Type: text/x-python
Size: 1521 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c8c8bb6b/attachment-0001.py>

From njs at pobox.com  Fri Mar 14 22:21:12 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 14 Mar 2014 21:21:12 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
Message-ID: <CAPJVwBmm+WDCpsRDAVOZ=YgRX5u7HfWZ+Cs2NWdHev9_v8_PsQ@mail.gmail.com>

On Fri, Mar 14, 2014 at 5:53 PM, Guido van Rossum <guido at python.org> wrote:
> I have now read the PEP, and I think it's good. I think it's a waste of time
> to keep bikeshedding on the choice of operator -- @ is the best compromise.
> I do have a few specific notes:
>
> - Right associativity is not unheard of in Python. E.g. **. If you
>   think that for other reasons @ should be right associative, don't
>   let Python's tradition stop you. But then you need to decide which
>   of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or
>   (a*b)@c? And if you choose the latter, it follows that a at b*c means
>   a@(b*c) -- is that okay? (And similar examples exist for the other
>   choice.)

Like Robert I have the suspicion that the best option is to make @
right-associative and place it just above (more tightly binding) than
*. But I'll poll numpy-discussion and friends and see if anyone has
ideas for more objective measures.

> - Did you consider a duck-typing (is that the word?) attribute?
>   E.g. a*b is elementwise multiplication; a.M*b must be used for
>   matrix multiplication.  (Your use of .T as "transpose" made me think
>   of this.)  Of course the question is, can you get those packages
>   that currently use * for matrix multiply to comply? (I don't consider
>   this a serious counter-proposal. But you list a bunch of rejected
>   alternatives; this could be in that list.

This is an interesting suggestion! I think it hasn't received full
consideration before because the tangle between numpy.ndarray and
numpy.matrix means that the most obvious implementation for an
ndarray.M attribute would be to return a full-fledged numpy.matrix
object... and no-one wants to encourage proliferation of numpy.matrix
objects. But returning a tiny special-purpose class that only
implements __mul__ would avoid that objection. At that point it's
basically a nicer version of the "user-defined infix" '*dot*' operator
idea. It has similar problems with needing an unaesthetically magical
implementation, producing a proliferation of special classes (one
extra one for each array type), requiring an allocation on every call,
etc., but this might well be the next-best thing to a real operator.

All in all, I think we'd rather have a real operator, so if you're
happy to go with @ then I won't put lots of effort into finding
horrible problems that force us to reject .M ;-), but I'll certainly
add it to the PEP in any case.

> - Is @@ really necessary?  It seems you are adding it mostly because
>   it's cute and because of the parallel with **, not because it is
>   actually important enough to add new syntax.  And then later you use
>   it as an argument for @, which seems a bit circular.  Also, if we
>   were to make @ right-associative, the parallel with ** is already
>   imperfect.

@@ hasn't received as much attention as the rest of the proposal, so
I'll check in with numpy-discussion etc. on this as well. But my
personal feeling is +0 on @@ -- all else being equal, it's nicer to
have it than not. Is all else equal? Given the existence of @, the
increase in language complexity is small (or arguably negative, since
people who know *, **, @ may be surprised to find @@ missing, and have
to memorize its non-existence as an extra rule); the opportunity cost
is low (given the existence of @ I can't imagine we'll want to use the
token @@ for something else later); and @@ will be used in real life,
if not as often as @ itself -- 'vec @@ 2' for squared Euclidean length
probably won't be too uncommon, 'matrix @@ n' gets used for things
like simulating random walks on graphs or other markov chains, and the
'matrix @@ -1' notation will probably be a nice win for beginners and
other less-sophisticated programmers who just want to get some
computation done in a way that doesn't require them to keep a more
complicated mapping between math and code notation in their head or
caring about numerical details. I think of @@ being like, say, '%=' --
nothing one would ever add syntax for in isolation, but if designing
an overall system of operators then it makes more sense.

Probably in the end it just comes down to your aesthetic judgement
:-). Numeric folk will be fine either way.

(And I wouldn't consider @@ a *strong* argument for spelling '@' as
'@'; it's just mentioned in that section because there aren't really
*any* strong arguments for preferring one spelling versus another, we
have to make a decision, and so a weak argument is better than
nothing. Two useful operators for the complexity cost of one is a good
deal? :shrug: Obviously if @@ gets dropped I'll drop that bullet point
as well.)

> - For better counts of usages, perhaps Sourcegraph.com might help?  It
>   is a source code query engine that has a Python parser and (limited)
>   type inference built in (also separately available as pysonar on
>   github IIRC). To be clear, I don't need more numbers to be convinced.

Oo, that is shiny, thanks for the tip.

> Once we've decided on associativity and @@, I'm ready to accept.

Wonderful to hear! Thanks for giving this so much time and attention
(on no warning)!

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From ronan.lamy at gmail.com  Fri Mar 14 23:01:54 2014
From: ronan.lamy at gmail.com (Ronan Lamy)
Date: Fri, 14 Mar 2014 22:01:54 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <lfvo4j$43j$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <lfvo4j$43j$1@ger.gmane.org>
Message-ID: <53237C52.80600@gmail.com>

Le 14/03/14 20:17, Serhiy Storchaka a ?crit :
> Matrix multiplication is only defined on 2d arrays (matrices). But why
> not on arbitrary tensors (except scalars)?

The PEP does include a (complicated) definition of "matrix 
multiplication" for arrays of all ranks. In particular, @-multiplication 
of rank-1 arrays is the dot product.


From greg.ewing at canterbury.ac.nz  Fri Mar 14 22:46:33 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 10:46:33 +1300
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <532378B9.2040707@canterbury.ac.nz>

+1000! This all makes total sense.

-- 
Greg


From steve at pearwood.info  Fri Mar 14 23:48:12 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Mar 2014 09:48:12 +1100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
	for matrix multiplication and matrix power
In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
Message-ID: <20140314224812.GR12595@ando>

On Fri, Mar 14, 2014 at 01:59:14AM +0000, Nathaniel Smith wrote:

> This PEP proposes two new binary operators dedicated to matrix
> multiplication and matrix power, spelled ``@`` and ``@@``
> respectively.  (Mnemonic: ``@`` is ``*`` for mATrices.)

When I first started reading the PEP, I was rather dubious about the 
choice of @ as operator, but it surprised me at how quickly I got used 
to it. I suppose because unconsciously I associated it with the common 
shorthand implying (scalar) multiplication:

    5kg of apples @ $2 per kg costs $10.00

so it didn't take me very long to warm to it.

+1


-- 
Steven


From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:16:25 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:16:25 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140314112931.4c38de75@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <20140314112931.4c38de75@fsol>
Message-ID: <53238DC9.1010908@canterbury.ac.nz>

Antoine Pitrou wrote:
> Or simply implement __invert__, so you can write it "~ A".
> (__invert__ doesn't really invert a number, it takes the bitwise
> complement :-))

But that's already taken to mean elementwise bitwise
complement.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:27:11 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:27:11 +1300
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
Message-ID: <5323904F.1000008@canterbury.ac.nz>

Antony Lee wrote:
> A simple suggestion: what about defining the special methods as __at__, 
> __rat__, __iat__

Please, no -- the last thing we need in Python is iyats!

http://www.paravia.com/wiki/index.php5?title=Iyat

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:29:24 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:29:24 +1300
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <20140314190009.GA19829@phdru.name>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com>
 <20140314190009.GA19829@phdru.name>
Message-ID: <532390D4.2000508@canterbury.ac.nz>

Oleg Broytman wrote:
> On Fri, Mar 14, 2014 at 11:53:47AM -0700, Guido van Rossum <guido at python.org> wrote:
 >
>>Ratatatatatat! :-)
> 
>    Is this the sound made by matrices being multiplied? ;-)

I think it's the sound of a PEP idea being lined up against
the wall and shot. :-(

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:41:23 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:41:23 +1300
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <20140314224812.GR12595@ando>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando>
Message-ID: <532393A3.9010005@canterbury.ac.nz>

Steven D'Aprano wrote:
> I suppose because unconsciously I associated it with the common 
> shorthand implying (scalar) multiplication:
> 
>     5kg of apples @ $2 per kg costs $10.00

That analogy actually extends to the matrix case as well.
E.g. if you have a vector q of quantities and a vector
p of prices, then q @ p is the total price.

-- 
Greg

From solipsis at pitrou.net  Sat Mar 15 00:50:32 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 00:50:32 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
Message-ID: <20140315005032.773acc95@fsol>

On Sat, 15 Mar 2014 12:41:23 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Steven D'Aprano wrote:
> > I suppose because unconsciously I associated it with the common 
> > shorthand implying (scalar) multiplication:
> > 
> >     5kg of apples @ $2 per kg costs $10.00
> 
> That analogy actually extends to the matrix case as well.
> E.g. if you have a vector q of quantities and a vector
> p of prices, then q @ p is the total price.

It depends how the vectors are layed out (horizontal @ vertical or
vertical @ horizontal).

Regards

Antoine.



From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:55:38 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:55:38 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315005032.773acc95@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol>
Message-ID: <532396FA.7010601@canterbury.ac.nz>

Antoine Pitrou wrote:

> It depends how the vectors are layed out (horizontal @ vertical or
> vertical @ horizontal).

The definition of @ in the proposal is such that two
1D arrays is interpreted as horizontal @ vertical.

-- 
Greg

From ncoghlan at gmail.com  Sat Mar 15 01:04:46 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Mar 2014 10:04:46 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <532396FA.7010601@canterbury.ac.nz>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
Message-ID: <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com>

On 15 Mar 2014 09:56, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>
> Antoine Pitrou wrote:
>
>> It depends how the vectors are layed out (horizontal @ vertical or
>> vertical @ horizontal).
>
>
> The definition of @ in the proposal is such that two
> 1D arrays is interpreted as horizontal @ vertical.

Oh, nice! Maybe the PEP should include implementing that for lists and our
other 1D sequence types?

Also, I just remembered another minor error in the PEP - as of Python 3.3,
memoryview.cast() allows the creation of multidimensional views, so the PEP
is incorrect in saying that can't be done with the builtins/stdlib.

Cheers,
Nick.

>
> --
> Greg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/9f6609b4/attachment.html>

From solipsis at pitrou.net  Sat Mar 15 01:09:44 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 01:09:44 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
Message-ID: <20140315010944.6b8244e1@fsol>

On Sat, 15 Mar 2014 12:55:38 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Antoine Pitrou wrote:
> 
> > It depends how the vectors are layed out (horizontal @ vertical or
> > vertical @ horizontal).
> 
> The definition of @ in the proposal is such that two
> 1D arrays is interpreted as horizontal @ vertical.

Really? That should be up to the third-party library implementing the @
operator for its types, not to the language itself: Python _suggests_
an use case for @, it doesn't mandate it (especially as there's no
appropriate data type in the stdlib).

Regards

Antoine.



From njs at pobox.com  Sat Mar 15 01:25:07 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 15 Mar 2014 00:25:07 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315010944.6b8244e1@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol>
Message-ID: <CAPJVwB=YZeM-kYr-7w=pMWaScCzz2NGNMwAAfLcO8=OzBHnKvA@mail.gmail.com>

On Sat, Mar 15, 2014 at 12:09 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 15 Mar 2014 12:55:38 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Antoine Pitrou wrote:
>>
>> > It depends how the vectors are layed out (horizontal @ vertical or
>> > vertical @ horizontal).
>>
>> The definition of @ in the proposal is such that two
>> 1D arrays is interpreted as horizontal @ vertical.
>
> Really? That should be up to the third-party library implementing the @
> operator for its types, not to the language itself: Python _suggests_
> an use case for @, it doesn't mandate it (especially as there's no
> appropriate data type in the stdlib).

See: http://legacy.python.org/dev/peps/pep-0465/#intended-usage-details

Which begins: "This section is informative, rather than normative --
it documents the consensus of a number of libraries that provide
array- or matrix-like objects on how the @ and @@ operators will be
implemented. ..."

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From greg.ewing at canterbury.ac.nz  Sat Mar 15 01:27:22 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 13:27:22 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315010944.6b8244e1@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol>
Message-ID: <53239E6A.6020205@canterbury.ac.nz>

Antoine Pitrou wrote:
> On Sat, 15 Mar 2014 12:55:38 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
>>The definition of @ in the proposal is such that two
>>1D arrays is interpreted as horizontal @ vertical.
> 
> Really? That should be up to the third-party library implementing the @
> operator for its types,

It is. It's described as "recommended semantics" in the PEP,
not something defined by the language.

-- 
Greg

From asmeurer at gmail.com  Sat Mar 15 01:48:46 2014
From: asmeurer at gmail.com (Aaron Meurer)
Date: Fri, 14 Mar 2014 17:48:46 -0700 (PDT)
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <3bea8fbc-64b5-42c1-93b6-2d6faf7562c5@googlegroups.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAPJVwBmm+WDCpsRDAVOZ=YgRX5u7HfWZ+Cs2NWdHev9_v8_PsQ@mail.gmail.com>
 <3bea8fbc-64b5-42c1-93b6-2d6faf7562c5@googlegroups.com>
Message-ID: <9e2519ad-0940-478c-bff0-39f47ab1e826@googlegroups.com>



On Friday, March 14, 2014 7:46:11 PM UTC-5, Aaron Meurer wrote:

On Friday, March 14, 2014 4:21:12 PM UTC-5, Nathaniel Smith wrote:

> On Fri, Mar 14, 2014 at 5:53 PM, Guido van Rossum <gu... at python.org> 
> wrote: 
>
>> > I have now read the PEP, and I think it's good. I think it's a waste of 
> time 
>
>> > to keep bikeshedding on the choice of operator -- @ is the best 
> compromise. 
>
>> > I do have a few specific notes: 
>
>> > 
>
>> > - Right associativity is not unheard of in Python. E.g. **. If you 
>
>> >   think that for other reasons @ should be right associative, don't 
>
>> >   let Python's tradition stop you. But then you need to decide which 
>
>> >   of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or 
>
>> >   (a*b)@c? And if you choose the latter, it follows that a at b*c means 
>
>> >   a@(b*c) -- is that okay? (And similar examples exist for the other 
>
>> >   choice.) 
>
>>
> Like Robert I have the suspicion that the best option is to make @ 
>
>> right-associative and place it just above (more tightly binding) than 
>
>> *. But I'll poll numpy-discussion and friends and see if anyone has 
> ideas for more objective measures. 
>

I guess a common case would be a*B at x, where a is a scalar.  Is it more 
efficient or numerically stable to evaluate that one way or the other?

Aaron Meurer



> > - Did you consider a duck-typing (is that the word?) attribute? 
>
>> >   E.g. a*b is elementwise multiplication; a.M*b must be used for 
>
>> >   matrix multiplication.  (Your use of .T as "transpose" made me think 
>
>> >   of this.)  Of course the question is, can you get those packages 
>
>> >   that currently use * for matrix multiply to comply? (I don't consider 
>
>> >   this a serious counter-proposal. But you list a bunch of rejected 
>
>> >   alternatives; this could be in that list. 
>
>>
> This is an interesting suggestion! I think it hasn't received full 
>
>> consideration before because the tangle between numpy.ndarray and 
>
>> numpy.matrix means that the most obvious implementation for an 
>
>> ndarray.M attribute would be to return a full-fledged numpy.matrix 
>
>> object... and no-one wants to encourage proliferation of numpy.matrix 
>
>> objects. But returning a tiny special-purpose class that only 
>
>> implements __mul__ would avoid that objection. At that point it's 
>
>> basically a nicer version of the "user-defined infix" '*dot*' operator 
>
>> idea. It has similar problems with needing an unaesthetically magical 
>
>> implementation, producing a proliferation of special classes (one 
>
>> extra one for each array type), requiring an allocation on every call, 
>
>> etc., but this might well be the next-best thing to a real operator. 
>
>>
> All in all, I think we'd rather have a real operator, so if you're 
>
>> happy to go with @ then I won't put lots of effort into finding 
>
>> horrible problems that force us to reject .M ;-), but I'll certainly 
>
>> add it to the PEP in any case. 
>
>>
> > - Is @@ really necessary?  It seems you are adding it mostly because 
>
>> >   it's cute and because of the parallel with **, not because it is 
>
>> >   actually important enough to add new syntax.  And then later you use 
>
>> >   it as an argument for @, which seems a bit circular.  Also, if we 
>
>> >   were to make @ right-associative, the parallel with ** is already 
>
>> >   imperfect. 
>
>>
> @@ hasn't received as much attention as the rest of the proposal, so 
>
>> I'll check in with numpy-discussion etc. on this as well. But my 
>
>> personal feeling is +0 on @@ -- all else being equal, it's nicer to 
>
>> have it than not. Is all else equal? Given the existence of @, the 
>
>> increase in language complexity is small (or arguably negative, since 
>
>> people who know *, **, @ may be surprised to find @@ missing, and have 
>
>> to memorize its non-existence as an extra rule); the opportunity cost 
>
>> is low (given the existence of @ I can't imagine we'll want to use the 
>
>> token @@ for something else later); and @@ will be used in real life, 
>
>> if not as often as @ itself -- 'vec @@ 2' for squared Euclidean length 
>
>> probably won't be too uncommon, 'matrix @@ n' gets used for things 
>
>> like simulating random walks on graphs or other markov chains, and the 
>
>> 'matrix @@ -1' notation will probably be a nice win for beginners and 
>
>> other less-sophisticated programmers who just want to get some 
>
>> computation done in a way that doesn't require them to keep a more 
>
>> complicated mapping between math and code notation in their head or 
>
>> caring about numerical details. I think of @@ being like, say, '%=' -- 
>
>> nothing one would ever add syntax for in isolation, but if designing 
>
>> an overall system of operators then it makes more sense. 
>
>>
> Probably in the end it just comes down to your aesthetic judgement 
>
>> :-). Numeric folk will be fine either way. 
>
>>
> (And I wouldn't consider @@ a *strong* argument for spelling '@' as 
>
>> '@'; it's just mentioned in that section because there aren't really 
>
>> *any* strong arguments for preferring one spelling versus another, we 
>
>> have to make a decision, and so a weak argument is better than 
>
>> nothing. Two useful operators for the complexity cost of one is a good 
>
>> deal? :shrug: Obviously if @@ gets dropped I'll drop that bullet point 
>
>> as well.) 
>
>>
> > - For better counts of usages, perhaps Sourcegraph.com might help?  It 
>
>> >   is a source code query engine that has a Python parser and (limited) 
>
>> >   type inference built in (also separately available as pysonar on 
>
>> >   github IIRC). To be clear, I don't need more numbers to be convinced. 
>
>>
> Oo, that is shiny, thanks for the tip. 
>
>>
> > Once we've decided on associativity and @@, I'm ready to accept. 
>
>>
> Wonderful to hear! Thanks for giving this so much time and attention 
>
>> (on no warning)! 
>
>>
> -n 
>
>>
> -- 
>
>> Nathaniel J. Smith 
>
>> Postdoctoral researcher - Informatics - University of Edinburgh 
>
>> http://vorpus.org 
>
>> _______________________________________________ 
>
>> Python-ideas mailing list 
>
>> Python... at python.org 
>
>> https://mail.python.org/mailman/listinfo/python-ideas 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/161d878b/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Sat Mar 15 00:16:01 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Mar 2014 12:16:01 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
Message-ID: <53238DB1.7060608@canterbury.ac.nz>

Nathan Schneider wrote:

> How about prefixing * with an innocuous backtick?

Just one little backtick... it's wafer-thin...

-- 
Greg



From cjwelborn at live.com  Sat Mar 15 03:07:18 2014
From: cjwelborn at live.com (Christopher Welborn)
Date: Fri, 14 Mar 2014 21:07:18 -0500
Subject: [Python-ideas] for-loop-if like list comps have?
Message-ID: <lg0ck8$nnh$1@ger.gmane.org>

mylist = ['apple', 'banana', 'orange', 'grapefruit']

# Why is this okay...
     for item in [x for x in mylist if 'p' in x]:
         print(item)

# But this isn't?
     for item in mylist if 'p' in item:
     SyntaxError: invalid syntax

# Instead you have to do another nesting level..
     for item in mylist:
         if 'p' in item:
             print(item)

# Or another way.
     for item in mylist:
         if 'p' not in item:
             continue
         print(item)

...And there is 'filter', 'lambdas', and whatever else
to achieve the same result. But the list comprehension
is already closely related to a for-loop, and it seems
natural to go from:
     [x for x in mylist if x]
to:
     for x in mylist if x:

Just an idea, sorry if it has been mentioned before.

-- 
\?\      /?/\
  \ \/??\/ / / Christopher Welborn (cj)
   \__/\__/ /  cjwelborn at live?com
    \__/\__/   http://welbornprod.com


From steve at pearwood.info  Sat Mar 15 03:37:48 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Mar 2014 13:37:48 +1100
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <lg0ck8$nnh$1@ger.gmane.org>
References: <lg0ck8$nnh$1@ger.gmane.org>
Message-ID: <20140315023747.GA16526@ando>

On Fri, Mar 14, 2014 at 09:07:18PM -0500, Christopher Welborn wrote:
> mylist = ['apple', 'banana', 'orange', 'grapefruit']
> 
> # Why is this okay...
>     for item in [x for x in mylist if 'p' in x]:
>         print(item)
> 
> # But this isn't?
>     for item in mylist if 'p' in item:
>     SyntaxError: invalid syntax


This suggestion has been made, oh, a million times -- not that I would 
ever exaggerate... *wink* -- and has been rejected each time.

Personally, I don't think it gains you too much, all you save is one 
line and one nesting level, and if you're nested so deeply that this 
becomes important, your code is probably in desperate need of 
refactoring.

The downside includes additional complication. It's one more thing for 
people to learn. Not only do you have to learn that there are two 
independent features, the for-statement and the if-statement, which 
being independent can be combined however you like:

    # 1
    for x in seq:
        if cond: 
            ...

    #2
    if cond:
        for x in seq:
            ...

but now you also have to learn a *third* form:

    for x in seq if cond:
        ...

and learn that it is the same as #1 above rather than #2 above. And that 
there *isn't* a fourth form:

    if cond for x in seq: 
        ...

or other combinations like:

    while x if cond:
    if cond while x:
    if cond try:
    for x try: 
    for x if y try:


etc. The way statements work in Python is that you combine them by using 
a new line and starting a new block, not by having special syntactic 
forms for each combination.

List comprehensions are different, because they don't create a new 
block, and they are an expression not a statement.


-- 
Steven

From abarnert at yahoo.com  Sat Mar 15 03:43:32 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 14 Mar 2014 19:43:32 -0700
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <lg0ck8$nnh$1@ger.gmane.org>
References: <lg0ck8$nnh$1@ger.gmane.org>
Message-ID: <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com>

On Mar 14, 2014, at 19:07, Christopher Welborn <cjwelborn at live.com> wrote:

> mylist = ['apple', 'banana', 'orange', 'grapefruit']
> 
> # Why is this okay...
>    for item in [x for x in mylist if 'p' in x]:
>        print(item)
> 
> # But this isn't?
>    for item in mylist if 'p' in item:
>    SyntaxError: invalid syntax
> 
> # Instead you have to do another nesting level..
>    for item in mylist:
>        if 'p' in item:
>            print(item)
> 
> # Or another way.
>    for item in mylist:
>        if 'p' not in item:
>            continue
>        print(item)
> 
> ...And there is 'filter', 'lambdas', and whatever else
> to achieve the same result. But the list comprehension
> is already closely related to a for-loop, and it seems
> natural to go from:
>    [x for x in mylist if x]
> to:
>    for x in mylist if x:
> 
> Just an idea, sorry if it has been mentioned before.

In a listcomp, each for or if is a separate clause, and they can be nested any way you want (except for the restriction that the first clause has to be a for). So, you can do this:

    [x for xs in xss for x in xs]
    [x for x in xs if x if x-1]
    [x for xs in xss if xs for x in xs if x]

Should you be able to write all of those as one-liner statements?

And this isn't just a weird quirk of syntax; each clause is conceptually exactly the same as a nested statement (except for some minor differences with the first one); that's how comprehensions are defined in the language reference.

Not all languages do comprehensions this way; Clojure, Racket, etc. have only for loop clauses in their comprehensions, but every for loop clause has an optional if phrase. Which would make your translation obviously right, instead of wrong for a hard to notice reason. (And it would make comprehensions slightly easier to explain. And it would also make it easier to add things like a while phrase, which people keep asking for.) But that would be a pretty big change for Python at this point.

From bruce at leapyear.org  Sat Mar 15 05:09:02 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 14 Mar 2014 21:09:02 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <53238DB1.7060608@canterbury.ac.nz>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
Message-ID: <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>

I think making @ right associative would make this less suitable for other
uses. Example

someclassobject @ somevalue @ somevalue

won't work with right-associativity. Plus precedence of mixing right and
left can be confusing.

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/7594b8b5/attachment.html>

From steve at pearwood.info  Sat Mar 15 08:35:09 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Mar 2014 18:35:09 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
Message-ID: <20140315073508.GB16526@ando>

On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote:
> I think making @ right associative would make this less suitable for other
> uses. Example
> 
> someclassobject @ somevalue @ somevalue
> 
> won't work with right-associativity. 

Why not? It works with the other right-associative operator:

    x**y**z

"Works" depends on what you expect it to do. Unless you tell us what 
these "other uses" are, how can we know that right-associativity won't 
work?


> Plus precedence of mixing right and left can be confusing.

Perhaps. Perhaps not. This works as expected:

    3*2**5


Are there any other right-associative operators that are confusing?


-- 
Steven

From solipsis at pitrou.net  Sat Mar 15 12:27:33 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 12:27:33 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
Message-ID: <20140315122733.35ed7909@fsol>

On Sat, 15 Mar 2014 18:35:09 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote:
> > I think making @ right associative would make this less suitable for other
> > uses. Example
> > 
> > someclassobject @ somevalue @ somevalue
> > 
> > won't work with right-associativity. 
> 
> Why not? It works with the other right-associative operator:
> 
>     x**y**z
> 
> "Works" depends on what you expect it to do.

The real question is why @ would be right-associative. "**" is very
rarely used in a chained manner as the above, so its associativity
isn't really important (I don't think I have ever written "x**y**z").
@ will be used routinely in a chained manner, so the question is more
important here.

The possible reason given in the PEP is very weak and amounts to
premature optimization:

"""It's been suggested that @ should be right-associative, on the
grounds that for expressions like Mat @ Mat @ vec, the two different
evaluation orders produce the same result, but the right-associative
order Mat @ (Mat @ vec) will be faster and use less memory than the
left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication
is much cheaper than matrix-matrix multiplication)."""

If that's the only reason, then I'd like @ to be left-associative.

Regards

Antoine.



From robert.kern at gmail.com  Sat Mar 15 12:47:48 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Mar 2014 11:47:48 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <lfvo4j$43j$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <lfvo4j$43j$1@ger.gmane.org>
Message-ID: <lg1ekl$11j$1@ger.gmane.org>

On 2014-03-14 20:17, Serhiy Storchaka wrote:
> 14.03.14 03:59, Nathaniel Smith ???????(??):
>> PEP: XXXX
>> Title: Dedicated infix operators for matrix multiplication and matrix power
>
> This is about matrix multiplication and matrix power. But what about matrix
> division? It is needed to distinguish elementwise division from left and right
> matrix divisions.

In our experience, matrix division comes up rather more rarely. For numerical 
problems, these should typically be implemented with a linear solve of some 
type. The best kind of solve varies based on the problem, so that's a good 
reason to continue to implement it with a function call rather than an operator. 
For symbolic problems (a la Sympy), both forms can be spelled reasonably well 
given matrix multiplication and matrix power.

> And what about concatenation? Python lists and tuples use
> ``+`` for this, but NumPy arrayse use ``+`` for bitwise addition.

There are *lots* of ways to concatenate an n-dimensional array, and we have a 
whole complement of functions to do that, vstack(), hstack(), dstack(), 
concatenate(), r_[], c_[]. A single operator would not suffice.

> Matrix multiplication is only defined on 2d arrays (matrices). But why not on
> arbitrary tensors (except scalars)?

The PEP defines how numpy intends to interpret n-dimensional operands and the 
reasons why. For tensors qua tensors you tend to want to do arbitrary 
Einstein-summation, which is hard to do in a single operator. numpy.einsum() 
handles that for us.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From greg.ewing at canterbury.ac.nz  Sat Mar 15 12:55:09 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Mar 2014 00:55:09 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315122733.35ed7909@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
Message-ID: <53243F9D.1060802@canterbury.ac.nz>

Antoine Pitrou wrote:

> The possible reason given in the PEP is very weak and amounts to
> premature optimization:

I don't think it's just a matter of optimization. Often,
matrix @ vector represents a linear operator acting on an
element of a vector space. When you chain them,

    A @ B @ C @ v

conceptually represents acting on v with C, then B,
then A.

-- 
Greg

From oscar.j.benjamin at gmail.com  Sat Mar 15 13:03:10 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Mar 2014 12:03:10 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com>
Message-ID: <CAHVvXxQ4JbriLpn0NxHZnJgbNF6dWZY-wuemUD_DQZS1s=uuWQ@mail.gmail.com>

On 15 March 2014 00:04, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 15 Mar 2014 09:56, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>>
>> Antoine Pitrou wrote:
>>
>>> It depends how the vectors are layed out (horizontal @ vertical or
>>> vertical @ horizontal).
>>
>>
>> The definition of @ in the proposal is such that two
>> 1D arrays is interpreted as horizontal @ vertical.
>
> Oh, nice! Maybe the PEP should include implementing that for lists and our
> other 1D sequence types?

I don't think that that should be included in the PEP. It's unlikely
that many users will want that and if they do it will be subject to
bike-shedding of issues that aren't directly relevant to the problem
that this PEP tries to solve. It'll get brought up on python-ideas
some time in the next couple of years and then interested parties can
bike-shed over it separately (with the benefit of an existing
implementation of the operators in numpy for comparison).

> Also, I just remembered another minor error in the PEP - as of Python 3.3,
> memoryview.cast() allows the creation of multidimensional views, so the PEP
> is incorrect in saying that can't be done with the builtins/stdlib.

I'm getting a bit of  d?j? vu here:
https://mail.python.org/pipermail/python-dev/2013-September/128463.html

Has this changed since then? I'd like to know how to use memoryview to
create a 2D matrix of integers or floats.

Also even if it is possible to create a 2D view in the stdlib I don't
think that the stdlib should try so support matrix multiplication
unless it is prepared to try and do so efficiently. Numpy delegates
this task to the underlying BLAS library which will usually be not
just asymptotically more efficient than a na?ve algorithm but also
heavily micro-optimised. I doubt that the Python core wants to grow a
dependency on having a BLAS library so unless someone wants to
reimplement this part of one it's best to leave it out.

A big +1 on the PEP from me.

I think this brings numpy on a par with Matlab for linear algebra.
Matlab also has other matrix related operators such as matrix left and
right divide with \ and / but I think that these are triviallising
what should be understood as non-trivial operations and are also
overloaded to mean too many different things (given different array
shapes). Similarly Matlab has made a big mistake in having the default
multiplication operator be matrix multiplication which creates a
significant pitfall for new users and becomes a common source of
problems that people have to work through every time they start
debugging their code (it's just a time waster really).

Some people have complained that the @ symbol looks clunky. Personally
I think it's important that this operator should be very distinctive.
Using an innocent-looking operator for matrix multiplication has been
tried before and based on ~7 years of teaching Matlab I would say that
it's a bad idea.


Oscar

From solipsis at pitrou.net  Sat Mar 15 13:06:31 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 13:06:31 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz>
Message-ID: <20140315130631.6f97ec16@fsol>

On Sun, 16 Mar 2014 00:55:09 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Antoine Pitrou wrote:
> 
> > The possible reason given in the PEP is very weak and amounts to
> > premature optimization:
> 
> I don't think it's just a matter of optimization. Often,
> matrix @ vector represents a linear operator acting on an
> element of a vector space. When you chain them,
> 
>     A @ B @ C @ v
> 
> conceptually represents acting on v with C, then B,
> then A.

It can just as well represent "acting" on v with (A @ B @ C).

Of course, mathematically it shouldn't make a difference, but in
computer programming right-associative operators are always a special
case, and therefore an additional cognitive burden.

Regards

Antoine.



From daoust.mj at gmail.com  Sat Mar 15 13:20:59 2014
From: daoust.mj at gmail.com (Mark Daoust)
Date: Sat, 15 Mar 2014 08:20:59 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
Message-ID: <CAFg8gYz1Z27WPkCzqfy+=cCiH1q4KGy=RX1Fg3Rs1=xqkkZNzg@mail.gmail.com>

*>Aaron Meurer* asmeurer at gmail.com
<python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20Fwd%3A%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%0A%20for%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3C9e2519ad-0940-478c-bff0-39f47ab1e826%40googlegroups.com%3E>
>
*>Sat Mar 15 01:48:46 CET 2014*
>
> > Like Robert I have the suspicion that the best option is to make @
> > right-associative and place it just above (more tightly binding) than
> >
> > *. But I'll poll numpy-discussion and friends and see if anyone has
> > ideas for more objective measures.
> >
>
> I guess a common case would be a*B at x, where a is a scalar. Is it more
> efficient or numerically stable to evaluate that one way or the other?

I'm not a numerical linear algebra expert, but I disagree.

If your matrix code needs to be optimized, there's no way around thinking
about your specific situation.

Optimizing matrix expressions is a tricky problem.
 <http://en.wikipedia.org/wiki/Matrix_chain_multiplication>
http://en.wikipedia.org/wiki/Matrix_chain_multiplication
http://www.youtube.com/watch?v=nVt24G_2VC0

If you want a different order use parenthesis, or build a symbolic
expression.

I don't think either choice of priority, or associativity will be the right
often enough be worth an affront to the "principal of least surprise".

Things are nice and clear the way they are.
>From the docs
http://docs.python.org/3.4/reference/expressions.html#binary-arithmetic-operations
:

"""The binary arithmetic operations have the conventional priority
levels... Apart from the power operator[s], there are only two levels, one
for multiplicative operators and one for additive operators"""

let's keep it like this: Vote P/E/MD/AS not P/E/M/MD/AS.

I think that if there is a reason to change is Greg is on the right track:

*>Greg Ewing* greg.ewing at canterbury.ac.nz
<python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%20for%0A%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3C53243F9D.1060802%40canterbury.ac.nz%3E>
*>Sat Mar 15 12:55:09 CET 2014*
>Often,matrix @ vector represents a linear operator acting on an element of
a vector space. When you chain them,
> A @ B @ C @ v
>conceptually represents acting on v with C, then B, then A.




PS:

>*Nathaniel Smith* njs at pobox.com
<python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20Fwd%3A%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%0A%20for%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3CCAPJVwBmm%2BWDCpsRDAVOZ%3DYgRX5u7HfWZ%2BCs2NWdHev9_v8_PsQ%40mail.gmail.com%3E>
>*Fri Mar 14 22:21:12 CET 2014*
>
>The 'matrix @@ -1' notation will probably be a nice win for
>beginners and other less-sophisticated programmers ...

There's a spot where I like the Facade approach: return an object that only
has one method, numpy.solve as __mul__.  Make it hard to do badly.

>[x.M * y] This is an interesting suggestion! I think it hasn't received full consideration before ...
>but this might well be the next-best thing to a real operator.

Maybe It is a real operator, it's just spelled ".M*" instead of "@" ;)



Thanks for all the work on this Nathaniel.



Mark Daoust
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/96bcf9e6/attachment-0001.html>

From oscar.j.benjamin at gmail.com  Sat Mar 15 13:20:42 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Mar 2014 12:20:42 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315130631.6f97ec16@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
Message-ID: <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>

On 15 March 2014 12:06, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > The possible reason given in the PEP is very weak and amounts to
>> > premature optimization:
>>
>> I don't think it's just a matter of optimization. Often,
>> matrix @ vector represents a linear operator acting on an
>> element of a vector space. When you chain them,
>>
>>     A @ B @ C @ v
>>
>> conceptually represents acting on v with C, then B,
>> then A.
>
> It can just as well represent "acting" on v with (A @ B @ C).
>
> Of course, mathematically it shouldn't make a difference, but in
> computer programming right-associative operators are always a special
> case, and therefore an additional cognitive burden.

I don't think it's a premature optimisation. It's a significant
algorithmic optimisation.

A @ (B @ (C @ v)))   # 3 matrix-vector multiplies
(((A @ B) @ C) @ v)  # 2 matrix-matrix multiplies and one matrix-vector multiply

If the matrices are NxN and the vector of length N then the
matrix-vector multiplications can be performed with the asymptotically
optimal N**2 operations but the matrix-matrix multiplications require
something like N**2.5 or worse.

It is possible but unusual for people to write this the other way
round (in which case the optimisation would favour
left-associativity). In any case many of the users of these operators
will not know the difference between left and right associativity and
will either use brackets or just write it out and hope that
Python/numpy know what to do.


Oscar

From robert.kern at gmail.com  Sat Mar 15 13:22:22 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Mar 2014 12:22:22 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315130631.6f97ec16@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
Message-ID: <lg1glg$mc4$1@ger.gmane.org>

On 2014-03-15 12:06, Antoine Pitrou wrote:
> On Sun, 16 Mar 2014 00:55:09 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Antoine Pitrou wrote:
>>
>>> The possible reason given in the PEP is very weak and amounts to
>>> premature optimization:
>>
>> I don't think it's just a matter of optimization. Often,
>> matrix @ vector represents a linear operator acting on an
>> element of a vector space. When you chain them,
>>
>>      A @ B @ C @ v
>>
>> conceptually represents acting on v with C, then B,
>> then A.
>
> It can just as well represent "acting" on v with (A @ B @ C).
>
> Of course, mathematically it shouldn't make a difference, but in
> computer programming right-associative operators are always a special
> case, and therefore an additional cognitive burden.

I think his point was that people doing linear algebra tend to read these 
expressions "right-to-left" anyways because of that conceptual model. I'm not 
entirely sure how supportable that is in the general population, but it's 
certainly the way I think about these expressions. For me, left-associative 
causes an additional cognitive burden, but it's a minor one, either way.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From solipsis at pitrou.net  Sat Mar 15 13:28:11 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 13:28:11 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
Message-ID: <20140315132811.0445e543@fsol>

On Sat, 15 Mar 2014 12:20:42 +0000
Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
> 
> I don't think it's a premature optimisation. It's a significant
> algorithmic optimisation.

You could also make the multiplications lazy (and enforce the optimal
order when computing the final result) rather than enforce that
optimization at the language parsing level. It would be more flexible,
and would also avoid potentially pessimizing other use cases.

Regards

Antoine.



From t_glaessle at gmx.de  Sat Mar 15 13:33:02 2014
From: t_glaessle at gmx.de (=?ISO-8859-1?Q?Thomas_Gl=E4=DFle?=)
Date: Sat, 15 Mar 2014 13:33:02 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
Message-ID: <5324487E.2060505@gmx.de>


> I don't think it's a premature optimisation. It's a significant
> algorithmic optimisation.

Just in idea, any library could optimize itself by delaying the
evaluation of the expression until the result is used or a vector is
right multiplied. So the optimization wouldn't be a real issue here.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1031 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/6f339e14/attachment.sig>

From oscar.j.benjamin at gmail.com  Sat Mar 15 13:36:06 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Mar 2014 12:36:06 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315132811.0445e543@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol>
Message-ID: <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com>

On 15 March 2014 12:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 15 Mar 2014 12:20:42 +0000
> Oscar Benjamin
> <oscar.j.benjamin at gmail.com> wrote:
>>
>> I don't think it's a premature optimisation. It's a significant
>> algorithmic optimisation.
>
> You could also make the multiplications lazy (and enforce the optimal
> order when computing the final result) rather than enforce that
> optimization at the language parsing level. It would be more flexible,
> and would also avoid potentially pessimizing other use cases.

That's true. I believe both blaze and numpypy intend to introduce
optimisations in this style.


Oscar

From oscar.j.benjamin at gmail.com  Sat Mar 15 13:42:51 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Mar 2014 12:42:51 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol>
 <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com>
Message-ID: <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com>

On 15 March 2014 12:36, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 15 March 2014 12:28, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 15 Mar 2014 12:20:42 +0000
>> Oscar Benjamin
>> <oscar.j.benjamin at gmail.com> wrote:
>>>
>>> I don't think it's a premature optimisation. It's a significant
>>> algorithmic optimisation.
>>
>> You could also make the multiplications lazy (and enforce the optimal
>> order when computing the final result) rather than enforce that
>> optimization at the language parsing level. It would be more flexible,
>> and would also avoid potentially pessimizing other use cases.
>
> That's true. I believe both blaze and numpypy intend to introduce
> optimisations in this style.

Just to add to that: I personally would almost always use brackets
rather than rely on left- or right- associativity for something like
this. A similar way that it can come up is with scalar-scalar vs
scalar-array multiplication e.g.:

2 * pi * x / L * A  # A is a big array

I would rewrite that as

(2 * pi * x / L) * A

rather than rely on precedence/associativity.


Oscar

From steve at pearwood.info  Sat Mar 15 14:47:37 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Mar 2014 00:47:37 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com>
References: <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol>
 <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com>
 <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com>
Message-ID: <20140315134736.GC16526@ando>

On Sat, Mar 15, 2014 at 12:42:51PM +0000, Oscar Benjamin wrote:

> Just to add to that: I personally would almost always use brackets
> rather than rely on left- or right- associativity for something like
> this.

> A similar way that it can come up is with scalar-scalar vs
> scalar-array multiplication e.g.:
> 
> 2 * pi * x / L * A  # A is a big array
> 
> I would rewrite that as
> 
> (2 * pi * x / L) * A
> 
> rather than rely on precedence/associativity.

It seems to me that you actually are relying on precedence/ 
associativity, otherwise you would have written it in fully-bracketed 
form like this:

(((2 * pi) * x) / L) * A


It's your choice to include redundant brackets in an expression, but I 
try hard to avoid the extra visual noise.



-- 
Steven

From steve at pearwood.info  Sat Mar 15 15:04:51 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Mar 2014 01:04:51 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <20140315132811.0445e543@fsol>
References: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol>
Message-ID: <20140315140451.GD16526@ando>

On Sat, Mar 15, 2014 at 01:28:11PM +0100, Antoine Pitrou wrote:
> On Sat, 15 Mar 2014 12:20:42 +0000
> Oscar Benjamin
> <oscar.j.benjamin at gmail.com> wrote:
> > 
> > I don't think it's a premature optimisation. It's a significant
> > algorithmic optimisation.
> 
> You could also make the multiplications lazy (and enforce the optimal
> order when computing the final result) rather than enforce that
> optimization at the language parsing level. It would be more flexible,
> and would also avoid potentially pessimizing other use cases.

That sounds to me that you are suggesting that every single 
implementation of a matrix type that supports matrix multiplication 
needs to spend the considerable extra effort to make the multiplication 
lazy, in order to "potentially" avoid hurting hypothetical use-cases 
which don't exist now and may never exist.

That's as clear a case of YAGNI as I've seen for a long time.

We have one solid use-case for a matrix multiplication operator, and 
that's matrix multiplication. If matrix multiplication is most usefully 
treated as right-associative, then we ought to make it right- 
associative, and not burden the matrix multiplication operator with 
restrictions for the sake of hypothetical non-matrix-mult uses.

In the same way that there is one good use-case for the numeric 
exponentiation operator ** , namely numeric exponentiation, and 
consequently it is right-associative because that's how the operator is 
most usefully treated.


-- 
Steven

From oscar.j.benjamin at gmail.com  Sat Mar 15 15:25:10 2014
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 15 Mar 2014 14:25:10 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315134736.GC16526@ando>
References: <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol>
 <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com>
 <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com>
 <20140315134736.GC16526@ando>
Message-ID: <CAHVvXxTR4-dcUTc7PpWdTB7z3xeHTm7O1P32CxaZv_ym2A0qVQ@mail.gmail.com>

On 15 March 2014 13:47, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, Mar 15, 2014 at 12:42:51PM +0000, Oscar Benjamin wrote:
>
>> Just to add to that: I personally would almost always use brackets
>> rather than rely on left- or right- associativity for something like
>> this.
>
>> A similar way that it can come up is with scalar-scalar vs
>> scalar-array multiplication e.g.:
>>
>> 2 * pi * x / L * A  # A is a big array
>>
>> I would rewrite that as
>>
>> (2 * pi * x / L) * A
>>
>> rather than rely on precedence/associativity.
>
> It seems to me that you actually are relying on precedence/
> associativity, otherwise you would have written it in fully-bracketed
> form like this:
>
> (((2 * pi) * x) / L) * A

The point is that I don't care about the order of evaluation for the
scalar part. Wherever you put the brackets works for me:

(2 * pi) * (x / L)
((2 * pi) * x) / L
(2 * (pi * x)) / L
2 * ((pi * x) / L)
2 * (pi * (x / L))

The reason I care about it with A is because A is a big array. Every
operation on A involves an expensive pass over all the elements of the
array as well as a large temporary allocation.

> It's your choice to include redundant brackets in an expression, but I
> try hard to avoid the extra visual noise.

I don't see those brackets as noise. To me it clearly shows that I'm
only doing one pass over the array which is useful information.


Oscar

From solipsis at pitrou.net  Sat Mar 15 15:31:54 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 15:31:54 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com>
 <20140315132811.0445e543@fsol> <20140315140451.GD16526@ando>
Message-ID: <20140315153154.140b44ee@fsol>

On Sun, 16 Mar 2014 01:04:51 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> 
> That sounds to me that you are suggesting that every single 
> implementation of a matrix type that supports matrix multiplication 

No, only implementations that are actually concerned with that
particular optimization.

For an analogy, bytes objects don't implement cheap slicing: if you
want that, you need to use a different type (memoryview).

Regards

Antoine.



From steve at pearwood.info  Sat Mar 15 16:11:43 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Mar 2014 02:11:43 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <20140315122733.35ed7909@fsol>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
Message-ID: <20140315151143.GE16526@ando>

On Sat, Mar 15, 2014 at 12:27:33PM +0100, Antoine Pitrou wrote:

> The real question is why @ would be right-associative.

That's a good question, but I don't think that should be up to us to 
decide. Guido clearly stated that if it is more useful to define @ as 
right-associative, we shouldn't let the fact that most operators are 
left-associative get in the way of doing the right thing here. The only 
people who are in a position to decide that are the users of matrix 
multiplication.


[...]
> The possible reason given in the PEP is very weak and amounts to
> premature optimization:

I don't think it's premature optimization. Sometimes we do know ahead of 
time that a calculation done one way will be faster than doing it 
another way: you don't have to "try it and see" to realise that 
repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for 
using str.join(). [Aside: if you do try it, the string-concat 
reference-counting optimization of CPython may fool you into thinking 
that concatenation is generally fast. It isn't.]

Likewise there is nothing premature about the fact that "Matrix-vector 
multiplication is much cheaper than matrix-matrix multiplication". The 
only question is whether it is more common to write:

    Matrix @ Matrix @ Column_Vector

or

    Row_Vector @ Matrix @ Matrix


I'll leave it to those who do matrix maths to decide which they use more 
often, but personally I've never come across the second case except in 
schoolbook exercises.


> """It's been suggested that @ should be right-associative, on the
> grounds that for expressions like Mat @ Mat @ vec, the two different
> evaluation orders produce the same result, but the right-associative
> order Mat @ (Mat @ vec) will be faster and use less memory than the
> left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication
> is much cheaper than matrix-matrix multiplication)."""
> 
> If that's the only reason, then I'd like @ to be left-associative.

I'm not sure that premature pessimation is much of an improvement over 
premature optimization.

*wink*



-- 
Steven

From solipsis at pitrou.net  Sat Mar 15 16:22:33 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 16:22:33 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
Message-ID: <20140315162233.4d01afd5@fsol>

On Sun, 16 Mar 2014 02:11:43 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> > The possible reason given in the PEP is very weak and amounts to
> > premature optimization:
> 
> I don't think it's premature optimization. Sometimes we do know ahead of 
> time that a calculation done one way will be faster than doing it 
> another way: you don't have to "try it and see" to realise that 
> repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for 
> using str.join().

It's premature optimization because the PEP is proposing to enforce it
at the language level. We didn't change *the language* so that "str +="
allows for O(N) repeated concatenation; instead we tell people that
"".join() should be used for repeated concatenation. Why would our
course of action be different for matrix multiplication?

Regards

Antoine.



From ncoghlan at gmail.com  Sat Mar 15 16:48:44 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Mar 2014 01:48:44 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315162233.4d01afd5@fsol>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol>
Message-ID: <CADiSq7eVnViqjLyD6pyJLNFpRKit6aL2axOzD+rOeiTgWTTtTw@mail.gmail.com>

On 16 March 2014 01:22, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 16 Mar 2014 02:11:43 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>> > The possible reason given in the PEP is very weak and amounts to
>> > premature optimization:
>>
>> I don't think it's premature optimization. Sometimes we do know ahead of
>> time that a calculation done one way will be faster than doing it
>> another way: you don't have to "try it and see" to realise that
>> repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for
>> using str.join().
>
> It's premature optimization because the PEP is proposing to enforce it
> at the language level. We didn't change *the language* so that "str +="
> allows for O(N) repeated concatenation; instead we tell people that
> "".join() should be used for repeated concatenation. Why would our
> course of action be different for matrix multiplication?

That's why Guido's request was for the numeric community to go back
and try to come up with a better rationale one way or the other for
that question - he wanted to make it clear to them that right
associativity *was* potentially acceptable, so they should seriously
consider the question and make the case for right associativity if
they decided they really wanted it, rather than assume that we would
reject the idea out of hand. Left associativity still remains the
default, and this isn't the right list to argue about the choice (as
most of the relevant people aren't here). If the numeric computing
community decide they *do* want right associativity and add that to
the PEP, *then* Guido will need to make the call as to whether or not
he considers their rationale for requesting it convincing (and we will
have the opportunity to chime in with our own perspectives to help him
make up his mind).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Sat Mar 15 17:26:49 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Mar 2014 03:26:49 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <20140315162233.4d01afd5@fsol>
References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol>
Message-ID: <20140315162649.GF16526@ando>

On Sat, Mar 15, 2014 at 04:22:33PM +0100, Antoine Pitrou wrote:
> On Sun, 16 Mar 2014 02:11:43 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
> > > The possible reason given in the PEP is very weak and amounts to
> > > premature optimization:
> > 
> > I don't think it's premature optimization. Sometimes we do know ahead of 
> > time that a calculation done one way will be faster than doing it 
> > another way: you don't have to "try it and see" to realise that 
> > repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for 
> > using str.join().
> 
> It's premature optimization because the PEP is proposing to enforce it
> at the language level.

The PEP leaves the question of left- versus right-associativity open. It 
has to be decided one way or the other. What evidence would you want to 
see before saying "It's not premature optimization, it's a justified 
optimization"?



> We didn't change *the language* so that "str +="
> allows for O(N) repeated concatenation; instead we tell people that
> "".join() should be used for repeated concatenation. Why would our
> course of action be different for matrix multiplication?

Exactly the same argument applies to left-associativity. Why should the 
language enforce optimizing the first case over the second?

    row_vector @ matrix @ matrix

    vs. 

    matrix @ matrix @ column_vector

Since infix operators have to have an associativity, the language 
implicitly has to optimize one case or the other. We can't sit on the 
fence and refuse to favour one over the other.

If the matrix-using community says that both cases are equally common, 
(or uncommon, perhaps) and they are indifferent between left- and 
right-associativity, then it makes sense to stick to what the majority 
of other operators do. I agree with that.

But if the matrix-using community comes down in favour of right- 
associativity, because Practicality Beats Purity and the second case is 
sufficiently more common than the first case, then I think it is a 
*broken design* to force left-associativity on them just to satisfy the 
expectations of people who aren't going to use the @ operator.

I hope I've made my position clear here: Guido has given his blessing to 
the possibility of making @ right-associative, and I think that the only 
people who should care what the associativity of @ ends up being is the 
community of matrix-multiplication users. They should make that 
decision, based on what it useful to them. I don't care what colour this 
bikeshed ends up having, I just want to see that it is the users of that 
shed that make the decision.



-- 
Steven

From cjwelborn at live.com  Sat Mar 15 18:13:16 2014
From: cjwelborn at live.com (Christopher Welborn)
Date: Sat, 15 Mar 2014 12:13:16 -0500
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com>
References: <lg0ck8$nnh$1@ger.gmane.org>
 <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com>
Message-ID: <lg21mu$7fj$1@ger.gmane.org>

On 03/14/2014 09:43 PM, Andrew Barnert wrote:
>      [x for xs in xss for x in xs]
>      [x for x in xs if x if x-1]
>      [x for xs in xss if xs for x in xs if x]
>
> Should you be able to write all of those as one-liner statements?


Fair enough. No I wouldn't like any of those, but thanks
for making things clearer for me.

-- 
\?\      /?/\
  \ \/??\/ / / Christopher Welborn (cj)
   \__/\__/ /  cjwelborn at live?com
    \__/\__/   http://welbornprod.com


From asmeurer at gmail.com  Sat Mar 15 18:14:58 2014
From: asmeurer at gmail.com (Aaron Meurer)
Date: Sat, 15 Mar 2014 12:14:58 -0500
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315122733.35ed7909@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
Message-ID: <CAKgW=6KNoVjJfT2SNyBLCZ0YxFmduwQV-tmjxae6duda9PPmWg@mail.gmail.com>

On Sat, Mar 15, 2014 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 15 Mar 2014 18:35:09 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>> On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote:
>> > I think making @ right associative would make this less suitable for other
>> > uses. Example
>> >
>> > someclassobject @ somevalue @ somevalue
>> >
>> > won't work with right-associativity.
>>
>> Why not? It works with the other right-associative operator:
>>
>>     x**y**z
>>
>> "Works" depends on what you expect it to do.
>
> The real question is why @ would be right-associative. "**" is very
> rarely used in a chained manner as the above, so its associativity
> isn't really important (I don't think I have ever written "x**y**z").
> @ will be used routinely in a chained manner, so the question is more
> important here.

I write that all the time when working with SymPy. And the
associativity of ** is *very* important. If ** were left associative
it would not be useful, because, at least for positive x, y, and z,
(x**y)**z is the same as x**(y*z). If you really want the stacked
power it has to be right associative.

Aaron Meurer

>
> The possible reason given in the PEP is very weak and amounts to
> premature optimization:
>
> """It's been suggested that @ should be right-associative, on the
> grounds that for expressions like Mat @ Mat @ vec, the two different
> evaluation orders produce the same result, but the right-associative
> order Mat @ (Mat @ vec) will be faster and use less memory than the
> left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication
> is much cheaper than matrix-matrix multiplication)."""
>
> If that's the only reason, then I'd like @ to be left-associative.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/aHVlL6BADLY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to python-ideas+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

From solipsis at pitrou.net  Sat Mar 15 18:15:21 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Mar 2014 18:15:21 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol>
 <20140315162649.GF16526@ando>
Message-ID: <20140315181521.33cf22ca@fsol>

On Sun, 16 Mar 2014 03:26:49 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> Exactly the same argument applies to left-associativity. Why should the 
> language enforce optimizing the first case over the second?

It has nothing to do with optimizing, it's just that left-associativity
is the common expectation wrt. operators. Left-associativity doesn't
need any particular justification.

Regards

Antoine.



From cjwelborn at live.com  Sat Mar 15 18:22:37 2014
From: cjwelborn at live.com (Christopher Welborn)
Date: Sat, 15 Mar 2014 12:22:37 -0500
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <20140315023747.GA16526@ando>
References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando>
Message-ID: <lg228f$d92$1@ger.gmane.org>

On 03/14/2014 09:37 PM, Steven D'Aprano wrote:
> This suggestion has been made, oh, a million times -- not that I would
> ever exaggerate... *wink* -- and has been rejected each time.

Oops, I wouldn't have mentioned it if I had known.

> or other combinations like:
>
>      while x if cond:
>      if cond while x:
>      if cond try:
>      for x try:
>      for x if y try:
>

Yeh I could see things getting out of hand if all of
those were added. You both have made good points.
Thank you.


-- 
\?\      /?/\
  \ \/??\/ / / Christopher Welborn (cj)
   \__/\__/ /  cjwelborn at live?com
    \__/\__/   http://welbornprod.com


From alexander.belopolsky at gmail.com  Sat Mar 15 21:50:28 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 15 Mar 2014 16:50:28 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315151143.GE16526@ando>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
Message-ID: <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>

On Sat, Mar 15, 2014 at 11:11 AM, Steven D'Aprano <steve at pearwood.info>wrote:

> The only question is whether it is more common to write:
>
>     Matrix @ Matrix @ Column_Vector
>
> or
>
>     Row_Vector @ Matrix @ Matrix
>
>
> I'll leave it to those who do matrix maths to decide which they use more
> often, but personally I've never come across the second case except in
> schoolbook exercises.
>

Abstractly, 1-dimensional arrays are neither columns nor rows, but Python's
horizontal notation makes them more row-like than column-like.  In
2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a
column-vector.  Which one is more "natural"?

When you have a matrix

A = [[1, 2],
     [3, 4]]

A[1] is [3, 4], which is a row.  To get a column, [2, 4], one has to write
A[:,1] in numpy.

When it comes to matrix - vector multiplication,

[1, 2] @ [[1, 2],
          [3, 4]] -> [7, 10]

has a text-book appearance, while

[[1, 2],
 [3, 4]] @ [1, 2] -> [5, 11]

has to be mentally cast into

([[1, 2],
  [3, 4]] @ [[1],
             [2]])[0] -> [5, 11]

While it is more common in math literature to see Mat @ vec than vec @ Mat,
I don't think anyone who has completed an introductory linear algebra
course would have trouble understanding what [1, 2, 3] @ Mat means.  On the
other hand, novice programmers may find it puzzling why Mat @ [Mat1, Mat2]
is the same as [Mat @ Mat1, Mat @ Mat2], but [Mat @ [vec1, vec2]] is not
[Mat @ vec1, Mat @ vec2].
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/67daccb1/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Mar 15 23:31:49 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Mar 2014 11:31:49 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315130631.6f97ec16@fsol>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
Message-ID: <5324D4D5.5090106@canterbury.ac.nz>

Antoine Pitrou wrote:

> It can just as well represent "acting" on v with (A @ B @ C).

Yes, but if you ask what A @ B @ C means as an operator,
it means "the same thing as acting with C, then B, then A."

The fact that evaluating it right-to-left also happens to
be more efficient in some cases means that there are both
conceptual *and* practical reasons for making @ right
associative.

I was just trying to point out that efficiency is not the
*only* reason to consider this.

There's a counter-argument as well -- it's only more
efficient if the rightmost operand is just a single vector
or a small enough array of vectors. Otherwise, it's more
efficient to calculate a combined operator and then
apply that. So the question would be whether the small
case is common enough in practice to be worth having an
unusual associativity.

A data point from another language, for what it's worth:
In APL, *all* operators are right-associative. I'm not
sure what the reason for that choice was, but it may
result from a similar line of reasoning around how
mathematicians think about things.

-- 
Greg

From alexander.belopolsky at gmail.com  Sat Mar 15 23:44:50 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 15 Mar 2014 18:44:50 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <5324D4D5.5090106@canterbury.ac.nz>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <5324D4D5.5090106@canterbury.ac.nz>
Message-ID: <CAP7h-xZXh2f3PAeeOVNy3fJuug86QjRPdz2kuUC0iTguk_KkLA@mail.gmail.com>

On Sat, Mar 15, 2014 at 6:31 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> In APL, *all* operators are right-associative. I'm not
> sure what the reason for that choice was, but it may
> result from a similar line of reasoning around how
> mathematicians think about things.
>

In APL, there is no operator precedence, so right associativity is the only
way to make expressions that mix unary and binary operation look natural.
 For example, a + -b.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/432d96d7/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Mar 15 23:56:55 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Mar 2014 11:56:55 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lg1glg$mc4$1@ger.gmane.org>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol>
 <lg1glg$mc4$1@ger.gmane.org>
Message-ID: <5324DAB7.8010709@canterbury.ac.nz>

Robert Kern wrote:

> For me, left-associative causes an additional cognitive burden, but it's 
> a minor one, either way.

Wild idea -- provide a way to spell both left and
right associative versions.

It's disappointing that there isn't a character that's
the mirror image of an @. Failing that:

    A @ B     left associative
    A @: B    right associative

(Ducks hail of tomatoes from the no-colons-in-expressions
crowd...)

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat Mar 15 23:56:58 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Mar 2014 11:56:58 +1300
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315151143.GE16526@ando>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
Message-ID: <5324DABA.9020808@canterbury.ac.nz>

Steven D'Aprano wrote:
> or
> 
>     Row_Vector @ Matrix @ Matrix
> 
> personally I've never come across the second case except in 
> schoolbook exercises.

It turns up sometimes in quantum mechanics, at least when
doing the algebra. I don't know whether people who do
QM calculations numerically ever do it that way, though.
Flipping it around would be easy enough if it were more
efficient.

-- 
Greg

From ncoghlan at gmail.com  Sun Mar 16 03:57:06 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Mar 2014 12:57:06 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
Message-ID: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>

On 16 March 2014 06:50, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
>
> Abstractly, 1-dimensional arrays are neither columns nor rows, but Python's
> horizontal notation makes them more row-like than column-like.  In
> 2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a
> column-vector.  Which one is more "natural"?

Folks, please stop trying to argue this one in the abstract. The
decision will likely be made primarily based on the feedback of folks
that have *actually been using and teaching* Python as a tool for
matrix manipulation (etc) for more than a decade.

Guido has asked them to have that discussion and summarise their
conclusions in the PEP - we can lob our armchair opinions into the mix
after the experts have spoken.

This won't be the first time features have been added to the language
core specifically for the benefit of the numeric community - the
ellipsis notation and extended slicing in general were added for their
benefit years ago (I started getting involved in Python core
development around Python 2.3, just as the core sequence types were
being updated to support extended slicing, and being able to use
"s[::-1]" with lists, strings, etc was a novel concept).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Sun Mar 16 03:59:17 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 15 Mar 2014 19:59:17 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
 <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
Message-ID: <CAP7+vJLsGP8cAT5yN+0nQuKdk=Agsaw35bRYyfYr16MWUpZY0g@mail.gmail.com>

Also complex numbers were Jim Hugunin's request for numeric work.

On Saturday, March 15, 2014, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 16 March 2014 06:50, Alexander Belopolsky
> <alexander.belopolsky at gmail.com <javascript:;>> wrote:
> >
> > Abstractly, 1-dimensional arrays are neither columns nor rows, but
> Python's
> > horizontal notation makes them more row-like than column-like.  In
> > 2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a
> > column-vector.  Which one is more "natural"?
>
> Folks, please stop trying to argue this one in the abstract. The
> decision will likely be made primarily based on the feedback of folks
> that have *actually been using and teaching* Python as a tool for
> matrix manipulation (etc) for more than a decade.
>
> Guido has asked them to have that discussion and summarise their
> conclusions in the PEP - we can lob our armchair opinions into the mix
> after the experts have spoken.
>
> This won't be the first time features have been added to the language
> core specifically for the benefit of the numeric community - the
> ellipsis notation and extended slicing in general were added for their
> benefit years ago (I started getting involved in Python core
> development around Python 2.3, just as the core sequence types were
> being updated to support extended slicing, and being able to use
> "s[::-1]" with lists, strings, etc was a novel concept).
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com <javascript:;>   |   Brisbane,
> Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/a34412ba/attachment.html>

From alexander.belopolsky at gmail.com  Sun Mar 16 04:12:52 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 15 Mar 2014 23:12:52 -0400
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
 <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
Message-ID: <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com>

On Sat, Mar 15, 2014 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Folks, please stop trying to argue this one in the abstract. The
> decision will likely be made primarily based on the feedback of folks
> that have *actually been using and teaching* Python as a tool for
> matrix manipulation (etc) for more than a decade.
>

I would think I qualify having been using Numeric/NumPy since 2003.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/5bd40381/attachment.html>

From ncoghlan at gmail.com  Sun Mar 16 04:32:15 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Mar 2014 13:32:15 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
 <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
 <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com>
Message-ID: <CADiSq7feYKs0zK8_xr=B-3eTwe-0zbyBabuSB2pcFmBk+RHnsA@mail.gmail.com>

On 16 March 2014 13:12, Alexander Belopolsky
<alexander.belopolsky at gmail.com> wrote:
>
> On Sat, Mar 15, 2014 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> Folks, please stop trying to argue this one in the abstract. The
>> decision will likely be made primarily based on the feedback of folks
>> that have *actually been using and teaching* Python as a tool for
>> matrix manipulation (etc) for more than a decade.
>
>
> I would think I qualify having been using Numeric/NumPy since 2003.

OK, thanks for the additional context - I didn't know that. However, I
still suggest finding the relevant discussion on the NumPy lists and
contributing there would be a better way to go, rather than getting
Nathaniel to track the discussion in two places at once.

We'll get another go around here and on python-dev after the PEP has
been updated with answers to Guido's questions :)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Sun Mar 16 13:26:34 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 16 Mar 2014 13:26:34 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
 <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
Message-ID: <20140316132634.13d6d589@fsol>

On Sun, 16 Mar 2014 12:57:06 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Guido has asked them to have that discussion and summarise their
> conclusions in the PEP - we can lob our armchair opinions into the mix
> after the experts have spoken.

Debunking armchair opinions is one of the points of the PEP process :-)

Regards

Antoine.



From ncoghlan at gmail.com  Sun Mar 16 14:05:15 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Mar 2014 23:05:15 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140316132634.13d6d589@fsol>
References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol>
 <20140315151143.GE16526@ando>
 <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com>
 <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com>
 <20140316132634.13d6d589@fsol>
Message-ID: <CADiSq7cEPUjj2FmpcjretgzbNnWLFWCjYJqJ44Xxc1A5u4QbgQ@mail.gmail.com>

On 16 March 2014 22:26, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 16 Mar 2014 12:57:06 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> Guido has asked them to have that discussion and summarise their
>> conclusions in the PEP - we can lob our armchair opinions into the mix
>> after the experts have spoken.
>
> Debunking armchair opinions is one of the points of the PEP process :-)

Aye, but at the moment it makes sense to wait and see if there is even
an argument to be had - Nathaniel may decide that even after reviewing
the question seriously, he doesn't want to propose a right associative
operator :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From bruce at leapyear.org  Mon Mar 17 04:46:08 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Sun, 16 Mar 2014 20:46:08 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315073508.GB16526@ando>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
Message-ID: <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>

On Sat, Mar 15, 2014 at 12:35 AM, Steven D'Aprano <steve at pearwood.info>wrote:

> On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote:
> > I think making @ right associative would make this less suitable for
> other
> > uses. Example
> >
> > someclassobject @ somevalue @ somevalue
> >
> > won't work with right-associativity.
>
> Why not? It works with the other right-associative operator:
>
>     x**y**z
>

It won't work because right-associativity means that the two values on the
right are combined together first and they won't be combined properly.
given that the type of someclassobject is not consulted.


>
> "Works" depends on what you expect it to do. Unless you tell us what
> these "other uses" are, how can we know that right-associativity won't
> work?
>

Here's a simple example to illustrate the concept. Suppose I have a class
that is roughly equivalent to a dict mapping values to lists of values,
e.g.,

a = {
    1: [10, 11, 12]
}
b = {
    1: [11, 13]

}


I might have

a + b = {

    1: [10, 11, 12, 11, 13]  # appends

}

a @ b = {

    1: [10, 11, 12, 13]  # appends non-duplicate values

}


but I also want to be able to merge in standard dicts. If @ is
right-associative, a @ x @ y where x and y are dicts, will try to compute x
@ y first and built-in dict doesn't supporr the @ operator. Even if it did,
it's not going to do what I'd want here.

I also want to give a larger example to illustrate the kind of thing I
think this operator would be useful for. SQLAlchemy has query objects where
you can write something like

some_query.filter(...).order_by(...).join(...)


I image that this could use the @ operator:

some_query @ filter(...) @ order_by(...) @ join(...)


The benefit here is that I have filter objects, order_by objects, etc. that
can be passed around rather than only having API calls. In the current API,
it's clumsy to support that as I either need both filter objects and a
query.filter method or I need a clumsy query.apply method. Having an
@operator I think works really well. Left vs. right associativity is
useful
if I wanted to support other kinds of objects on the right hand side of the
@, e.g., if:

some_query @ { column1: value1, column2: value2, ... }


as another spelling of

some_query @ filter(column1=value1, column2=value2, ... }


--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140316/7fb4c9a5/attachment.html>

From ncoghlan at gmail.com  Mon Mar 17 06:03:19 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 17 Mar 2014 15:03:19 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
 <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>
Message-ID: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>

On 17 March 2014 13:46, Bruce Leban <bruce at leapyear.org> wrote:
> I also want to give a larger example to illustrate the kind of thing I think
> this operator would be useful for. SQLAlchemy has query objects where you
> can write something like
>
> some_query.filter(...).order_by(...).join(...)
>
>
> I image that this could use the @ operator:
>
> some_query @ filter(...) @ order_by(...) @ join(...)

There are plenty of existing left-associative operators that could be
used for that if Mike Bayer wanted (or could be convinced) to do so,
so I don't see how that observation is relevant to the question of
whether or not this particular proposal should be for a
right-associative operator.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Mon Mar 17 06:50:32 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 17 Mar 2014 14:50:32 +0900
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lfv7ug$tuv$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando>
 <lfv7ug$tuv$1@ger.gmane.org>
Message-ID: <874n2xl6fr.fsf@uwakimon.sk.tsukuba.ac.jp>

Ron Adam writes:

 > The best I can come up with is to use '^' instead of dot.

This symbol is used to denote the "wedge product" of vectors, so it's
pretty clearly inappropriate.  I forget the exact definition, but it's
related to the area (volume) of the parallelpiped spanned by the
vectors.


From stephen at xemacs.org  Mon Mar 17 06:59:25 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 17 Mar 2014 14:59:25 +0900
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140314180811.35d7cdf6@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <lfvboj$eab$1@ger.gmane.org> <20140314180811.35d7cdf6@fsol>
Message-ID: <8738ihl60y.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > I am personally not fond of @, but I find it ok in that it is
 > distinctive enough without being terribly ugly.

"Not terribly ugly" is the best we're going to do, since Guido has
already ruled out non-ASCII characters.  Well, without another PEP,
and I don't think we should delay PEP 465 for a "Unicode operators"
PEP.


From sturla.molden at gmail.com  Mon Mar 17 09:54:34 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Mon, 17 Mar 2014 08:54:34 +0000 (UTC)
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com>
Message-ID: <585287815416738714.901387sturla.molden-gmail.com@news.gmane.org>

"M.-A. Lemburg" <mal at egenix.com> wrote:

>> Indeed. Numpy already uses a .T property for this.
> 
> Ah, good trick :-)

It is a better trick than you would expect. 

Matlab and Fortran matrix transpose has complexity O(M*N) for a rank M x N
matrix.

NumPy's .T attribute has complexity O(1)

With Numpy, we can put .T into matrix expressions with impunity. Now THAT
is a good trick :-)

I am not aware of any other system where memory access is so flexible that
matrix transposition can be done in O(1) time.


Regards,
Sturla


From robert.kern at gmail.com  Mon Mar 17 11:35:37 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 17 Mar 2014 10:35:37 +0000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
 <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>
 <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>
Message-ID: <lg6j55$7g1$1@ger.gmane.org>

On 2014-03-17 05:03, Nick Coghlan wrote:
> On 17 March 2014 13:46, Bruce Leban <bruce at leapyear.org> wrote:
>> I also want to give a larger example to illustrate the kind of thing I think
>> this operator would be useful for. SQLAlchemy has query objects where you
>> can write something like
>>
>> some_query.filter(...).order_by(...).join(...)
>>
>>
>> I image that this could use the @ operator:
>>
>> some_query @ filter(...) @ order_by(...) @ join(...)
>
> There are plenty of existing left-associative operators that could be
> used for that if Mike Bayer wanted (or could be convinced) to do so,
> so I don't see how that observation is relevant to the question of
> whether or not this particular proposal should be for a
> right-associative operator.

Although this does suggest that having a right-associative operator might 
provide some useful variety in the operator-overloading-as-DSL ecosystem.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From anthony at xtfx.me  Mon Mar 17 16:58:20 2014
From: anthony at xtfx.me (C Anthony Risinger)
Date: Mon, 17 Mar 2014 10:58:20 -0500
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
In-Reply-To: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
Message-ID: <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>

ok, let me try this again, sans distractions:

could we consider adding __code__ and __call__ to module objects? __code__
would contain the code executed in the module __dict__, and __call__ would
reload the module.

thoughts?

-- 

C Anthony


On Fri, Mar 14, 2014 at 4:03 PM, C Anthony Risinger <anthony at xtfx.me> wrote:

> hello,
>
> could we make modules work more like this...
>
> https://gist.github.com/xtfxme/9556806
>
> tl;dr: __code__ object available (potentially a __self__ object), reload
> by calling, makes working with modules more natural/safe!
>
> ENCLOSED: modules-next.py
>
> --
>
> C Anthony
>
> #----------------------------------------------------------------------------------------(
> modules-next.py )
> # encoding: utf-8
> #
> # modules-next
> #
> # C Anthony Risinger
>
>
> import sys
>
>
> m_name = 'modules_next'
> m_source = r"""\
> if __name__ == '{0}':
>     from . import __call__ as reexec
>     from . import __code__ as code
>     from . import __dict__ as ns
>     print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
>     __name__ = 'modules_next_reloaded'
>     reexec()
>     print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
> else:
>     print('!!! __name__ == {{0}}\n'.format(__name__))
>
>  def throw():
>     raise Exception(__name__)
> """.format(m_name)
> m_code = compile(m_source, m_name, 'exec')
> m_dict = {
>     '__file__': __file__ + '.next',
>     '__name__': m_name,
>     '__path__': list(),
>     '__doc__': None,
>     }
> #...bind namespace/functions
> module = eval('lambda: None', m_dict)
> #...automatically expose as attributes
> module.__dict__ = module.__globals__
> #...must be set twice, __code__ sees __dict__, but we see descriptor
> module.__name__ = m_name
> #...redefine the function body!
> module.__code__ = m_code
> #...yay! we haz legitimate module!
> sys.modules[m_name] = module
>
>
> print('--* importing...')
> import modules_next
> print('>>> {0}: {1}\n{2}\n'.format(
>     __name__,
>     modules_next.__code__,
>     modules_next.__dict__.keys(),
>     ))
>
> print('--* importing (for real this time)...')
> #...import/reload by calling our func-y-module
> module()
> print('>>> {0}: {1}\n{2}\n'.format(
>     __name__,
>     modules_next.__code__,
>     modules_next.__dict__.keys(),
>     ))
>
>
> print('--* throwing...')
> modules_next.throw()
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/0550a789/attachment.html>

From lemiant at hotmail.com  Mon Mar 17 17:07:05 2014
From: lemiant at hotmail.com (Alex Rodrigues)
Date: Mon, 17 Mar 2014 10:07:05 -0600
Subject: [Python-ideas] Please reconsider the Boolean
	evaluation	of	midnight
Message-ID: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>

I'm +1 for this idea. It's not about whether the style is good (which its not), but its fundamentally about the fact that it will fix bugs. 		 	   		  

From ncoghlan at gmail.com  Mon Mar 17 17:36:31 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Mar 2014 02:36:31 +1000
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
In-Reply-To: <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
Message-ID: <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>

On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote:
>
> ok, let me try this again, sans distractions:
>
> could we consider adding __code__ and __call__ to module objects?
__code__ would contain the code executed in the module __dict__, and
__call__ would reload the module.
>
> thoughts?

The question to ask is: what would such a change make possible that
importlib doesn't already handle?

This specific proposal wouldn't work right for extension modules, so people
will still need to go through the importlib APIs anyway. If you haven't
seen it yet, you may also want to give PEP 451 a read - we already
overhauled several aspects of the import system for 3.4.

Cheers,
Nick.

>
> --
>
> C Anthony
>
>
> On Fri, Mar 14, 2014 at 4:03 PM, C Anthony Risinger <anthony at xtfx.me>
wrote:
>>
>> hello,
>>
>> could we make modules work more like this...
>>
>> https://gist.github.com/xtfxme/9556806
>>
>> tl;dr: __code__ object available (potentially a __self__ object), reload
by calling, makes working with modules more natural/safe!
>>
>> ENCLOSED: modules-next.py
>>
>> --
>>
>> C Anthony
>>
>>
#----------------------------------------------------------------------------------------(
modules-next.py )
>> # encoding: utf-8
>> #
>> # modules-next
>> #
>> # C Anthony Risinger
>>
>>
>> import sys
>>
>>
>> m_name = 'modules_next'
>> m_source = r"""\
>> if __name__ == '{0}':
>>     from . import __call__ as reexec
>>     from . import __code__ as code
>>     from . import __dict__ as ns
>>     print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
>>     __name__ = 'modules_next_reloaded'
>>     reexec()
>>     print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys()))
>> else:
>>     print('!!! __name__ == {{0}}\n'.format(__name__))
>>
>> def throw():
>>     raise Exception(__name__)
>> """.format(m_name)
>> m_code = compile(m_source, m_name, 'exec')
>> m_dict = {
>>     '__file__': __file__ + '.next',
>>     '__name__': m_name,
>>     '__path__': list(),
>>     '__doc__': None,
>>     }
>> #...bind namespace/functions
>> module = eval('lambda: None', m_dict)
>> #...automatically expose as attributes
>> module.__dict__ = module.__globals__
>> #...must be set twice, __code__ sees __dict__, but we see descriptor
>> module.__name__ = m_name
>> #...redefine the function body!
>> module.__code__ = m_code
>> #...yay! we haz legitimate module!
>> sys.modules[m_name] = module
>>
>>
>> print('--* importing...')
>> import modules_next
>> print('>>> {0}: {1}\n{2}\n'.format(
>>     __name__,
>>     modules_next.__code__,
>>     modules_next.__dict__.keys(),
>>     ))
>>
>> print('--* importing (for real this time)...')
>> #...import/reload by calling our func-y-module
>> module()
>> print('>>> {0}: {1}\n{2}\n'.format(
>>     __name__,
>>     modules_next.__code__,
>>     modules_next.__dict__.keys(),
>>     ))
>>
>>
>> print('--* throwing...')
>> modules_next.throw()
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/258b5334/attachment.html>

From solipsis at pitrou.net  Mon Mar 17 18:11:55 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 17 Mar 2014 18:11:55 +0100
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
 <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>
Message-ID: <20140317181155.5ea09201@fsol>

On Tue, 18 Mar 2014 02:36:31 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote:
> >
> > ok, let me try this again, sans distractions:
> >
> > could we consider adding __code__ and __call__ to module objects?
> __code__ would contain the code executed in the module __dict__, and
> __call__ would reload the module.
> >
> > thoughts?
> 
> The question to ask is: what would such a change make possible that
> importlib doesn't already handle?

I suppose module.__code__ would be a lot less wasteful than
module.__loader__.get_code(module.__name__) (which seems to reload the
bytecode from disk every time).

(while looking at this, I was surprised that dis.dis(some_module)
1) doesn't show a disassembly for the module toplevel 2) shows
disassembly for objects which some_module imports from other modules)

Regards

Antoine.



From guido at python.org  Mon Mar 17 18:24:08 2014
From: guido at python.org (Guido van Rossum)
Date: Mon, 17 Mar 2014 10:24:08 -0700
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
In-Reply-To: <20140317181155.5ea09201@fsol>
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
 <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>
 <20140317181155.5ea09201@fsol>
Message-ID: <CAP7+vJK4hBwDdoPgr2Q333EKrHGO=Vs7RgR7ZHgVA3s7RAr=SA@mail.gmail.com>

I don't know whether it's worth the savings, but the __code__ for a
module's toplevel is thrown away because it is not needed after it has run.
Reading it back from disk is wasteful if you do it every time for every
module, but how often is it used in a situation where speed matters?

Saving __code__ might be useful so you can tell with certainty which code
was run -- the file may have been modified since it was imported.


On Mon, Mar 17, 2014 at 10:11 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> On Tue, 18 Mar 2014 02:36:31 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote:
> > >
> > > ok, let me try this again, sans distractions:
> > >
> > > could we consider adding __code__ and __call__ to module objects?
> > __code__ would contain the code executed in the module __dict__, and
> > __call__ would reload the module.
> > >
> > > thoughts?
> >
> > The question to ask is: what would such a change make possible that
> > importlib doesn't already handle?
>
> I suppose module.__code__ would be a lot less wasteful than
> module.__loader__.get_code(module.__name__) (which seems to reload the
> bytecode from disk every time).
>
> (while looking at this, I was surprised that dis.dis(some_module)
> 1) doesn't show a disassembly for the module toplevel 2) shows
> disassembly for objects which some_module imports from other modules)
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/e25b80ff/attachment.html>

From ncoghlan at gmail.com  Mon Mar 17 18:38:47 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Mar 2014 03:38:47 +1000
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
In-Reply-To: <20140317181155.5ea09201@fsol>
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
 <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>
 <20140317181155.5ea09201@fsol>
Message-ID: <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com>

On 18 Mar 2014 03:16, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> On Tue, 18 Mar 2014 02:36:31 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote:
> > >
> > > ok, let me try this again, sans distractions:
> > >
> > > could we consider adding __code__ and __call__ to module objects?
> > __code__ would contain the code executed in the module __dict__, and
> > __call__ would reload the module.
> > >
> > > thoughts?
> >
> > The question to ask is: what would such a change make possible that
> > importlib doesn't already handle?
>
> I suppose module.__code__ would be a lot less wasteful than
> module.__loader__.get_code(module.__name__) (which seems to reload the
> bytecode from disk every time).

Yep, it's a space/speed trade-off - keeping this info around could waste a
lot of memory, and most code will never need it.

It's a similar reason to why we throw source code away after compiling it,
but then also route inspect.getsource() through the linecache module.

Cheers,
Nick.

>
> (while looking at this, I was surprised that dis.dis(some_module)
> 1) doesn't show a disassembly for the module toplevel 2) shows
> disassembly for objects which some_module imports from other modules)

I don't think we have any tests to see how dis.dis behaves with module
objects - it's likely just hitting the path that is designed to disassemble
all methods on a class. (We throw away the bytecode after execution for
classes as well)

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/18cfdc44/attachment.html>

From solipsis at pitrou.net  Mon Mar 17 18:41:20 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 17 Mar 2014 18:41:20 +0100
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
 <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>
 <20140317181155.5ea09201@fsol>
 <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com>
Message-ID: <20140317184120.5d333d0d@fsol>

On Tue, 18 Mar 2014 03:38:47 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > (while looking at this, I was surprised that dis.dis(some_module)
> > 1) doesn't show a disassembly for the module toplevel 2) shows
> > disassembly for objects which some_module imports from other modules)
> 
> I don't think we have any tests to see how dis.dis behaves with module
> objects - it's likely just hitting the path that is designed to disassemble
> all methods on a class. (We throw away the bytecode after execution for
> classes as well)

A comment in dis.dis's source code does talk about modules, so
apparently someone thought about them, but not very hard :-)

(edit: that someone is you - b2a9a59c2d57 ;-P)

Regards

Antoine.



From njs at pobox.com  Tue Mar 18 00:08:51 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 17 Mar 2014 23:08:51 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
Message-ID: <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>

On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> A few other miscellaneous comments:
>
> - nice work on the PEP Nathaniel!

Thanks!

> - as with others, "@" as the operator doesn't thrill me, but I also think it
> crosses the threshold of "good enough given the constraints"
> - the PEP should probably recommend adding an "operator.matmul" function, a
> "PyObject_MatrixMultiply" C API and consider whether or not the new special
> method should be given a C level type slot.

operator.matmul and PyObject_MatrixMultiply are obvious enough, but
I'm afraid I'm not too clear on the tradeoffs about adding a C level
type slot, or even entirely sure what the alternative is. (I guess I
just assumed that all special methods used C level type slots and
there was nothing to think about.) Do you (or anyone) have any
thoughts?

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From ncoghlan at gmail.com  Tue Mar 18 00:32:55 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Mar 2014 09:32:55 +1000
Subject: [Python-ideas] Function-like Modules (better
	control/introspection)
In-Reply-To: <20140317184120.5d333d0d@fsol>
References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com>
 <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com>
 <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com>
 <20140317181155.5ea09201@fsol>
 <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com>
 <20140317184120.5d333d0d@fsol>
Message-ID: <CADiSq7exvqYYdDHkYiQP-Tkoz1ibkB7w8RMmnP7FrSqACcuTMQ@mail.gmail.com>

On 18 Mar 2014 03:45, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> On Tue, 18 Mar 2014 03:38:47 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> > >
> > > (while looking at this, I was surprised that dis.dis(some_module)
> > > 1) doesn't show a disassembly for the module toplevel 2) shows
> > > disassembly for objects which some_module imports from other modules)
> >
> > I don't think we have any tests to see how dis.dis behaves with module
> > objects - it's likely just hitting the path that is designed to
disassemble
> > all methods on a class. (We throw away the bytecode after execution for
> > classes as well)
>
> A comment in dis.dis's source code does talk about modules, so
> apparently someone thought about them, but not very hard :-)
>
> (edit: that someone is you - b2a9a59c2d57 ;-P)

Huh, clearly I put a lot of thought into that one...

One day I will learn to go check the source rather than speculating from
memory, but apparently yesterday is not that day :)

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/5d098d50/attachment-0001.html>

From bruce at leapyear.org  Tue Mar 18 05:14:03 2014
From: bruce at leapyear.org (Bruce Leban)
Date: Mon, 17 Mar 2014 21:14:03 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>
References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org>
 <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us>
 <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
 <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>
 <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>
Message-ID: <CAGu0AnuCUVV8FOcAOvS1kaYA1eU2erTZK1grQ02wDChp082VOA@mail.gmail.com>

On Sun, Mar 16, 2014 at 10:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> There are plenty of existing left-associative operators that could be
> used for that if Mike Bayer wanted (or could be convinced) to do so,
> so I don't see how that observation is relevant to the question of
> whether or not this particular proposal should be for a
> right-associative operator.
>

I knew I might get in trouble by giving such a specific example. My view is
that a new @ operator is "baggage free" in terms of implied semantics --
since it has defined semantics only for matrices.

To the extent that it has implied semantics of being multiplication-like, I
think it would be surprising that one multiplication operator is
left-associative and one is right-associative. Does @ have higher or lower
precedence than *? The draft PEP says @ is the same as * but it can't be if
it has opposite associativity. The interpretations of these two expressions
depends on the precedence:

A @ B * C
A * B @ C


Someone mentioned that APL is right-associative and has no
operator-precedence. Those two things are related. There's no operator
precedence because there are too many operators for everyone to remember
the exact order of precedence. Given that, all operators need to be left or
right associative. I don't know the reason for sure but I have heard that
one reason is that and given that monadic (unary) operators are right
associative, making all operators right associative means that you can
simply interpret an APL statement reading from right to left. I don't think
APL is the right model to look at.

--- Bruce
Learn how hackers think: http://j.mp/gruyere-security
https://www.linkedin.com/in/bruceleban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/45429967/attachment.html>

From steve at pearwood.info  Tue Mar 18 05:58:04 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 18 Mar 2014 15:58:04 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <lg6j55$7g1$1@ger.gmane.org>
References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com>
 <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com>
 <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com>
 <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com>
 <53238DB1.7060608@canterbury.ac.nz>
 <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com>
 <20140315073508.GB16526@ando>
 <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com>
 <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com>
 <lg6j55$7g1$1@ger.gmane.org>
Message-ID: <20140318045803.GA29663@ando>

On Mon, Mar 17, 2014 at 10:35:37AM +0000, Robert Kern wrote:
> On 2014-03-17 05:03, Nick Coghlan wrote:

> >There are plenty of existing left-associative operators that could be
> >used for that if Mike Bayer wanted (or could be convinced) to do so,
> >so I don't see how that observation is relevant to the question of
> >whether or not this particular proposal should be for a
> >right-associative operator.
> 
> Although this does suggest that having a right-associative operator might 
> provide some useful variety in the operator-overloading-as-DSL ecosystem.


That's a good point. That suggests that adding a second 
right-associative operator may be a good idea even if it's not necessary 
for numpy.


-- 
Steven

From stephen at xemacs.org  Tue Mar 18 07:29:45 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 18 Mar 2014 15:29:45 +0900
Subject: [Python-ideas] Please reconsider the
	Boolean	evaluation	of	midnight
In-Reply-To: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>
References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>
Message-ID: <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp>

Alex Rodrigues writes:

 > I'm +1 for this idea. It's not about whether the style is good
 > (which its not), but its fundamentally about the fact that it will
 > fix bugs.

It's not about fixing bugs.  There are lots of bugs which could be
fixed by changing Python behavior.  It's about the fact that it almost
surely doesn't penalize any correct programs that depend on the
existing behavior.


From ncoghlan at gmail.com  Tue Mar 18 09:02:10 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Mar 2014 18:02:10 +1000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
Message-ID: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>

On 18 Mar 2014 09:08, "Nathaniel Smith" <njs at pobox.com> wrote:
>
> On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > A few other miscellaneous comments:
> >
> > - nice work on the PEP Nathaniel!
>
> Thanks!
>
> > - as with others, "@" as the operator doesn't thrill me, but I also
think it
> > crosses the threshold of "good enough given the constraints"
> > - the PEP should probably recommend adding an "operator.matmul"
function, a
> > "PyObject_MatrixMultiply" C API and consider whether or not the new
special
> > method should be given a C level type slot.
>
> operator.matmul and PyObject_MatrixMultiply are obvious enough, but
> I'm afraid I'm not too clear on the tradeoffs about adding a C level
> type slot, or even entirely sure what the alternative is. (I guess I
> just assumed that all special methods used C level type slots and
> there was nothing to think about.) Do you (or anyone) have any
> thoughts?

I suspect you're going to want one, as without it, the implementation
method ends up in the class dict instead (the context management protocol
works that way).

I suspect the design we will want is a new struct for Py_Matrix slots (akin
to those for numbers, etc). The alternative would be to just add more
"Number" slots, but that isn't really accurate.

I'll come up with a more specific proposal after refreshing my memory of
the exact details of the current layout.

Cheers,
Nick.

>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/63640e51/attachment.html>

From abarnert at yahoo.com  Tue Mar 18 11:39:26 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 18 Mar 2014 03:39:26 -0700
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
	for matrix multiplication and matrix power
In-Reply-To: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
 <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
Message-ID: <68ED2909-56B0-4BF3-A6E9-0FB10435B6F7@yahoo.com>

On Mar 18, 2014, at 1:02, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 18 Mar 2014 09:08, "Nathaniel Smith" <njs at pobox.com> wrote:
> >
> > On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > > A few other miscellaneous comments:
> > >
> > > - nice work on the PEP Nathaniel!
> >
> > Thanks!
> >
> > > - as with others, "@" as the operator doesn't thrill me, but I also think it
> > > crosses the threshold of "good enough given the constraints"
> > > - the PEP should probably recommend adding an "operator.matmul" function, a
> > > "PyObject_MatrixMultiply" C API and consider whether or not the new special
> > > method should be given a C level type slot.
> >
> > operator.matmul and PyObject_MatrixMultiply are obvious enough, but
> > I'm afraid I'm not too clear on the tradeoffs about adding a C level
> > type slot, or even entirely sure what the alternative is. (I guess I
> > just assumed that all special methods used C level type slots and
> > there was nothing to think about.) Do you (or anyone) have any
> > thoughts?
> 
> I suspect you're going to want one, as without it, the implementation method ends up in the class dict instead (the context management protocol works that way).
> 
> I suspect the design we will want is a new struct for Py_Matrix slots (akin to those for numbers, etc). The alternative would be to just add more "Number" slots, but that isn't really accurate.
> 
Someone needs to work out how these changes will affect the numpy C API (the Py_Array dotfunc slot, the PyArray interface, etc.). Whoever is doing that--and the numpy C API users who are interested in it--will probably care more about how matmul fits into the python C API than anyone else does.

As for whether this is another numeric operation or a new kind of operation, I think that depends on how you intuitively interpret matrix operations, or rather which intuition numpy and similar libraries should be encouraging. The fact that the proposal suggests that >2d arrays should handle matmul by broadcasting it over the final 2d subarrays seems like it's treating it fundamentally as a numeric operation on (vectors and) matrices. On the other hand, I don't think anyone implementing a split-complex or quaternion library on top of numpy would want to use @ for multiplying their numbers, even if they used @ on the matrices as the implementation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/402ec756/attachment-0001.html>

From robert.kern at gmail.com  Tue Mar 18 11:47:02 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 18 Mar 2014 10:47:02 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
 <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
Message-ID: <lg986n$n63$1@ger.gmane.org>

On 2014-03-18 08:02, Nick Coghlan wrote:
>
> On 18 Mar 2014 09:08, "Nathaniel Smith"
> <njs at pobox.com
> <mailto:njs at pobox.com>> wrote:
>  >
>  > On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan
> <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
>  > > A few other miscellaneous comments:
>  > >
>  > > - nice work on the PEP Nathaniel!
>  >
>  > Thanks!
>  >
>  > > - as with others, "@" as the operator doesn't thrill me, but I also think it
>  > > crosses the threshold of "good enough given the constraints"
>  > > - the PEP should probably recommend adding an "operator.matmul" function, a
>  > > "PyObject_MatrixMultiply" C API and consider whether or not the new special
>  > > method should be given a C level type slot.
>  >
>  > operator.matmul and PyObject_MatrixMultiply are obvious enough, but
>  > I'm afraid I'm not too clear on the tradeoffs about adding a C level
>  > type slot, or even entirely sure what the alternative is. (I guess I
>  > just assumed that all special methods used C level type slots and
>  > there was nothing to think about.) Do you (or anyone) have any
>  > thoughts?
>
> I suspect you're going to want one, as without it, the implementation method
> ends up in the class dict instead (the context management protocol works that way).
>
> I suspect the design we will want is a new struct for Py_Matrix slots (akin to
> those for numbers, etc). The alternative would be to just add more "Number"
> slots, but that isn't really accurate.

Would it be more palatable if the name were something like __altmul__ or 
__auxmul__ rather than __matmul__? Really, it's just a second 
multiplication-like operator. The leading use case for a second 
multiplication-like operator happens to be matrix multiplication, but I strongly 
suspect it will get used for other mathematical things like symbolic function 
composition or operator application (as in "linear operator", not +-*/) and 
maybe some secondary multiplication types in the weirder groups and fields (you 
can bet I will resurrect my Clifford algebra module to use this operator for one 
of the several types of multiplication they support). Granted, there is still 
some awkwardness in that *none* of the builtin number types will support it.

And hey, look! It makes the @ux or @lt multiplication operator makes a sensible, 
language-independent mnemonic! (not that "auxiliary" or "alternate" work in all 
languages, but at least the 'a' is baked into the special method name)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ncoghlan at gmail.com  Tue Mar 18 12:27:36 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Mar 2014 21:27:36 +1000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <lg986n$n63$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
 <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
 <lg986n$n63$1@ger.gmane.org>
Message-ID: <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>

On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote:
> On 2014-03-18 08:02, Nick Coghlan wrote:
>>  > operator.matmul and PyObject_MatrixMultiply are obvious enough, but
>>  > I'm afraid I'm not too clear on the tradeoffs about adding a C level
>>  > type slot, or even entirely sure what the alternative is. (I guess I
>>  > just assumed that all special methods used C level type slots and
>>  > there was nothing to think about.) Do you (or anyone) have any
>>  > thoughts?
>>
>> I suspect you're going to want one, as without it, the implementation
>> method
>> ends up in the class dict instead (the context management protocol works
>> that way).
>>
>> I suspect the design we will want is a new struct for Py_Matrix slots
>> (akin to
>> those for numbers, etc). The alternative would be to just add more
>> "Number"
>> slots, but that isn't really accurate.


So, here's the change to PyHeapType object that makes the most sense
to me (assuming both "@" and "@@" are added - drop the new "power"
methods if "@@" is dropped from the PEP):

- add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject
- define PyMatrixMethods as:

    typedef struct {
        binaryfunc mt_multiply;
        binaryfunc mt_power;
        binaryfunc mt_inplace_multiply;
        binaryfunc mt_inplace_power;
   } PyMatrixMethods;

This approach increases the size of all type objects by one pointer.

The other way to do it would be to just add four new slots to PyNumberMethods:

        binaryfunc nb_matrix_multiply;
        binaryfunc nb_matrix_power;
        binaryfunc nb_inplace_matrix_multiply;
        binaryfunc nb_inplace_matrix_power;

This approach increases the size of all type objects that define one
or more of the numeric functions by four pointers, and doesn't really
make sense at a conceptual level. The latter is the main reason I
prefer the separate PyMatrixMethods struct.

Other "should probably be listed in the PEP for completeness" change
is that this will need new opcodes and AST nodes. Reviewing the
current opcode list and node names, I would suggest:

    BINARY_MATRIX_MULTIPLY
    BINARY_MATRIX_POWER
    INPLACE_MATRIX_MULTIPLY
    INPLACE_MATRIX_POWER

    MatMult | MatPow

> Would it be more palatable if the name were something like __altmul__ or
> __auxmul__ rather than __matmul__? Really, it's just a second
> multiplication-like operator. The leading use case for a second
> multiplication-like operator happens to be matrix multiplication, but I
> strongly suspect it will get used for other mathematical things like
> symbolic function composition or operator application (as in "linear
> operator", not +-*/) and maybe some secondary multiplication types in the
> weirder groups and fields (you can bet I will resurrect my Clifford algebra
> module to use this operator for one of the several types of multiplication
> they support). Granted, there is still some awkwardness in that *none* of
> the builtin number types will support it.

I think "matmul" is fine. That makes the primary intended use case
clear, without preventing its use for other purposes (like vector dot
products or more exotic things). The magic method names are "add",
"mul", "div", "mod", etc, even though we occasionally use them for
other purposes (e.g. concatenation, sequence repetition, path joining,
interpolation).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From mal at egenix.com  Tue Mar 18 13:18:56 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 18 Mar 2014 13:18:56 +0100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>	<CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>	<CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>	<CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>	<CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>	<CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>	<lg986n$n63$1@ger.gmane.org>
 <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>
Message-ID: <532839B0.7020108@egenix.com>

On 18.03.2014 12:27, Nick Coghlan wrote:
> On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote:
>> On 2014-03-18 08:02, Nick Coghlan wrote:
>>>  > operator.matmul and PyObject_MatrixMultiply are obvious enough, but
>>>  > I'm afraid I'm not too clear on the tradeoffs about adding a C level
>>>  > type slot, or even entirely sure what the alternative is. (I guess I
>>>  > just assumed that all special methods used C level type slots and
>>>  > there was nothing to think about.) Do you (or anyone) have any
>>>  > thoughts?
>>>
>>> I suspect you're going to want one, as without it, the implementation
>>> method
>>> ends up in the class dict instead (the context management protocol works
>>> that way).
>>>
>>> I suspect the design we will want is a new struct for Py_Matrix slots
>>> (akin to
>>> those for numbers, etc). The alternative would be to just add more
>>> "Number"
>>> slots, but that isn't really accurate.
> 
> 
> So, here's the change to PyHeapType object that makes the most sense
> to me (assuming both "@" and "@@" are added - drop the new "power"
> methods if "@@" is dropped from the PEP):
> 
> - add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject
> - define PyMatrixMethods as:
> 
>     typedef struct {
>         binaryfunc mt_multiply;
>         binaryfunc mt_power;
>         binaryfunc mt_inplace_multiply;
>         binaryfunc mt_inplace_power;
>    } PyMatrixMethods;
> 
> This approach increases the size of all type objects by one pointer.
> 
> The other way to do it would be to just add four new slots to PyNumberMethods:
> 
>         binaryfunc nb_matrix_multiply;
>         binaryfunc nb_matrix_power;
>         binaryfunc nb_inplace_matrix_multiply;
>         binaryfunc nb_inplace_matrix_power;
> 
> This approach increases the size of all type objects that define one
> or more of the numeric functions by four pointers, and doesn't really
> make sense at a conceptual level. The latter is the main reason I
> prefer the separate PyMatrixMethods struct.

I don't think that it's a good idea to make all type objects
larger just to address matrix multiplications which none of the
builtin types will actually use, so +1 on adding to the number
methods, even if a matrix isn't a number (they still use numbers,
so it's not that far off :-)), but -1 on adding another slot
struct.

Aside: Mechanisms such as namedtuple add lots of new type
objects to the runtime environment, so it's no longer safe to
assume that the size of type objects doesn't really matter much
in real-life applications.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 18 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          11 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               22 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Tue Mar 18 16:06:35 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 18 Mar 2014 08:06:35 -0700
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>
 <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com>

FWIW what *is* the status of this proposal? It feels reasonable to change
this in 3.5. My friend at SourceGraph found no evidence of the behavior
being relied on in real code (it did find the unit test :-). Although I'm
sure he only searched a limited corpus. If someone wants to do a more
thorough search before deciding I can point them to the software needed.


On Mon, Mar 17, 2014 at 11:29 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> Alex Rodrigues writes:
>
>  > I'm +1 for this idea. It's not about whether the style is good
>  > (which its not), but its fundamentally about the fact that it will
>  > fix bugs.
>
> It's not about fixing bugs.  There are lots of bugs which could be
> fixed by changing Python behavior.  It's about the fact that it almost
> surely doesn't penalize any correct programs that depend on the
> existing behavior.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/bd5bd0fb/attachment.html>

From donald at stufft.io  Tue Mar 18 16:10:10 2014
From: donald at stufft.io (Donald Stufft)
Date: Tue, 18 Mar 2014 11:10:10 -0400
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com>
References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>
 <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com>
Message-ID: <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io>


On Mar 18, 2014, at 11:06 AM, Guido van Rossum <guido at python.org> wrote:

> FWIW what *is* the status of this proposal? It feels reasonable to change this in 3.5. My friend at SourceGraph found no evidence of the behavior being relied on in real code (it did find the unit test :-). Although I'm sure he only searched a limited corpus. If someone wants to do a more thorough search before deciding I can point them to the software needed.
> 
> 
> On Mon, Mar 17, 2014 at 11:29 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Alex Rodrigues writes:
> 
>  > I'm +1 for this idea. It's not about whether the style is good
>  > (which its not), but its fundamentally about the fact that it will
>  > fix bugs.
> 
> It's not about fixing bugs.  There are lots of bugs which could be
> fixed by changing Python behavior.  It's about the fact that it almost
> surely doesn't penalize any correct programs that depend on the
> existing behavior.
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Right now I think it?s waiting on someone to actually do it. Personally I was
waiting on development on 3.5 to open up before I bothered to see if I could
figure out how to do this or not :)

If someone else gets to it before me all the better though!

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/0211e27f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/0211e27f/attachment-0001.sig>

From ethan at stoneleaf.us  Tue Mar 18 16:18:53 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 18 Mar 2014 08:18:53 -0700
Subject: [Python-ideas] Please reconsider the Boolean evaluation of
	midnight
In-Reply-To: <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io>
References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl>
 <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com>
 <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io>
Message-ID: <532863DD.6030202@stoneleaf.us>

On 03/18/2014 08:10 AM, Donald Stufft wrote:
> On Mar 18, 2014, at 11:06 AM, Guido van Rossum wrote:
>>
>> FWIW what *is* the status of this proposal?
>
> Right now I think it?s waiting on someone to actually do it. Personally I was
> waiting on development on 3.5 to open up before I bothered to see if I could
> figure out how to do this or not :)
>
> If someone else gets to it before me all the better though!

It's on my to-do list, but it'll be a couple months before I get to it.

Race ya!  :)

--
~Ethan~

From abarnert at yahoo.com  Tue Mar 18 20:59:30 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 18 Mar 2014 12:59:30 -0700 (PDT)
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
	for matrix multiplication and matrix power
In-Reply-To: <532839B0.7020108@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
 <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
 <lg986n$n63$1@ger.gmane.org>
 <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>
 <532839B0.7020108@egenix.com>
Message-ID: <1395172770.13326.YahooMailNeo@web181001.mail.ne1.yahoo.com>

From: M.-A. Lemburg <mal at egenix.com>

Sent: Tuesday, March 18, 2014 5:18 AM
> On 18.03.2014 12:27, Nick Coghlan wrote:

>>  The other way to do it would be to just add four new slots to
>> PyNumberMethods:
>> 
>> ? ? ? ?  binaryfunc nb_matrix_multiply;
>> ? ? ? ?  binaryfunc nb_matrix_power;
>> ? ? ? ?  binaryfunc nb_inplace_matrix_multiply;
>> ? ? ? ?  binaryfunc nb_inplace_matrix_power;
>> 
>>  This approach increases the size of all type objects that define one
>>  or more of the numeric functions by four pointers, and doesn't really
>>  make sense at a conceptual level. The latter is the main reason I
>>  prefer the separate PyMatrixMethods struct.
> 
> I don't think that it's a good idea to make all type objects
> larger just to address matrix multiplications which none of the
> builtin types will actually use, so +1 on adding to the number
> methods, even if a matrix isn't a number (they still use numbers,
> so it's not that far off :-)), but -1 on adding another slot
> struct.

I don't see how putting them in PyNumberMethods doesn't make sense at a conceptual level. (Matrix, +, @) is a ring, and they form an assocative algebra. Sure, matrices are collections of simpler numbers, and their operations are defined in terms of the operations of those simpler numbers, but the same thing is true for complex numbers, or even rationals. From a mathematical point of view, there's no principled definition of "number" (or, rather, if there is, it's only going to include the naturals or maybe the ordinals). If anything, the float type has a lot more problems as a "number" than a matrix type does. So if there is a good practical reason to call matrix multiplication a numeric operation, why not?


From tarek at ziade.org  Tue Mar 18 20:57:48 2014
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 18 Mar 2014 20:57:48 +0100
Subject: [Python-ideas] A resources tracker ?
Message-ID: <5328A53C.2080604@ziade.org>

Hey

ResourceWarning instances that are created in some classes' __del__
function, like FileIO
are a great tool to track down a program bad behavior.

In network programming, that's even more important to avoid crashes, or
huge leaks.

But some of them are very hard to fix because we don't get much context,
we just
get warnings at the end of the program execution, when gc.collect is called.

sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb'>
sys:1: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb'>

Here I just know that somewhere, 2 file descriptors where not closed.

What I'd like to be able to do is to track down the origin of those
warnings.

Since __del__ is called asynchronously, it's impossible to track it
right now (or I don't know how)

What we need is a way to keep track of any resource allocation *when it
happens*.

Here's an idea: let's add three private functions in Python's io:

def __allocate_resource(fd) => records the file descriptor that was
allocated, along with the current traceback.
def __free_resource(fd) => removes the fd from the list.
def __is_resource_allocated(fd) => tell if the resource is in the list.

These three functions, plugged in somewhere in io's classes, could be
used in conjunction with ResourceWarning:
when __del__ is called, if the resource was not freed - we'd be able to
know where it was created.

Of course these functions are just a brain dump - I have no idea how io
internals work. But unless I missed it,
something like I've just described is missing in Python.

Cheers
Tarek

From abarnert at yahoo.com  Tue Mar 18 21:49:58 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 18 Mar 2014 13:49:58 -0700
Subject: [Python-ideas] A resources tracker ?
In-Reply-To: <5328A53C.2080604@ziade.org>
References: <5328A53C.2080604@ziade.org>
Message-ID: <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com>

On Mar 18, 2014, at 12:57, Tarek Ziad? <tarek at ziade.org> wrote:

> def __allocate_resource(fd) => records the file descriptor that was
> allocated, along with the current traceback.
> def __free_resource(fd) => removes the fd from the list.
> def __is_resource_allocated(fd) => tell if the resource is in the list

I like the general idea. I've actually written wrappers (in Python 2.x and other languages like C++, never Python 3, but similar idea...) to track these kinds of leaks myself.

I don't think you need all of these methods. Just the first one will do it: if you get to __del__ without a close and emit a ResourceWarning, use the info stashed by the allocate function; otherwise, it never gets looked at. And I'm not sure it needs to be a method after all.

What about cases where the io object doesn't actually allocate anything (because you got it from an fd or a socket object or similar?), but you still need to call close. Don't you want the traceback in those cases too? Also, in the most common cases (like open), you're actually creating a chain of two or three objects; do all of them need to store this info?

I don't think you want this on all the time... But when exactly _do_ you want it on? Debug mode? If the warning is enabled at allocation time? A global flag on the io module?

Also, storing an actual traceback is probably a bad idea, as it keeps all kinds of things alive for a very long time. Maybe storing a string representation, or enough info to generate such a representation in the ResourceWarning?

Also, I suspect that knowing the arguments used to allocate the resource (like the filename passed to open or the object's constructor, in the most common case, but also things like the cloexec flag when debugging multi-process apps) might be at least as useful as the traceback, so you might as well add that too.

Anyway, there's pure Python code in Lib/io.py that wraps up the C code. To experiment with this without getting into the C stuff, I think you could edit it to use your own classes that wrap _io.BufferedWriter, etc. instead of just exporting those directly.


From ncoghlan at gmail.com  Tue Mar 18 21:58:20 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 19 Mar 2014 06:58:20 +1000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <532839B0.7020108@egenix.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>
 <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>
 <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>
 <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>
 <lg986n$n63$1@ger.gmane.org>
 <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>
 <532839B0.7020108@egenix.com>
Message-ID: <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com>

On 18 Mar 2014 22:18, "M.-A. Lemburg" <mal at egenix.com> wrote:
>
> On 18.03.2014 12:27, Nick Coghlan wrote:
> > On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote:
> >> On 2014-03-18 08:02, Nick Coghlan wrote:
> >>>  > operator.matmul and PyObject_MatrixMultiply are obvious enough, but
> >>>  > I'm afraid I'm not too clear on the tradeoffs about adding a C
level
> >>>  > type slot, or even entirely sure what the alternative is. (I guess
I
> >>>  > just assumed that all special methods used C level type slots and
> >>>  > there was nothing to think about.) Do you (or anyone) have any
> >>>  > thoughts?
> >>>
> >>> I suspect you're going to want one, as without it, the implementation
> >>> method
> >>> ends up in the class dict instead (the context management protocol
works
> >>> that way).
> >>>
> >>> I suspect the design we will want is a new struct for Py_Matrix slots
> >>> (akin to
> >>> those for numbers, etc). The alternative would be to just add more
> >>> "Number"
> >>> slots, but that isn't really accurate.
> >
> >
> > So, here's the change to PyHeapType object that makes the most sense
> > to me (assuming both "@" and "@@" are added - drop the new "power"
> > methods if "@@" is dropped from the PEP):
> >
> > - add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject
> > - define PyMatrixMethods as:
> >
> >     typedef struct {
> >         binaryfunc mt_multiply;
> >         binaryfunc mt_power;
> >         binaryfunc mt_inplace_multiply;
> >         binaryfunc mt_inplace_power;
> >    } PyMatrixMethods;
> >
> > This approach increases the size of all type objects by one pointer.
> >
> > The other way to do it would be to just add four new slots to
PyNumberMethods:
> >
> >         binaryfunc nb_matrix_multiply;
> >         binaryfunc nb_matrix_power;
> >         binaryfunc nb_inplace_matrix_multiply;
> >         binaryfunc nb_inplace_matrix_power;
> >
> > This approach increases the size of all type objects that define one
> > or more of the numeric functions by four pointers, and doesn't really
> > make sense at a conceptual level. The latter is the main reason I
> > prefer the separate PyMatrixMethods struct.
>
> I don't think that it's a good idea to make all type objects
> larger just to address matrix multiplications which none of the
> builtin types will actually use, so +1 on adding to the number
> methods, even if a matrix isn't a number (they still use numbers,
> so it's not that far off :-)), but -1 on adding another slot
> struct.
>
> Aside: Mechanisms such as namedtuple add lots of new type
> objects to the runtime environment, so it's no longer safe to
> assume that the size of type objects doesn't really matter much
> in real-life applications.

So, here's the problem: operand precedence for sequences implemented in C
is currently broken in certain relatively obscure cases (and has been since
forever - we only found out about due to an SQL Alchemy failure when
porting to PyPy revealed PyPy was right and CPython was wrong). This is why
returning NotImplemented from sq_concat and sq_repeat doesn't work right.
The problem doesn't arise for sequences implemented in Python, because we
always populate the nb_add and nb_multiply slots for those.

I've tried fixing that directly, and it turned abstract.c into an
unmaintainable mess, so I abandoned that approach. My current preferred
solution is now similar to the one we use from Python: drop the direct
calls to sq_concat and sq_repeat from abstract.c, and instead automatically
add nb_add and nb_multiply implementations during type creation that
delegate dynamically to the sequence slots.

So a *lot* of types already have their PyNumberMethods slot allocated
(including sequence types implemented in Python), and I'd like to expand
that to include all sequence types, even those implemented in C.

That makes the space trade-off here substantially less clear, particularly
if matrix power ends up being added in the future (it has been dropped from
the current PEP).

I accept that adding the new slots to PyNumberMethods is more conceptually
coherent than I thought, though. There's also the fact that we already mess
about populating different slots for pragmatic reasons that have nothing to
do with conceptual integrity :)

Cheers,
Nick.

>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Mar 18 2014)
> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2014-03-29: PythonCamp 2014, Cologne, Germany ...          11 days to go
> 2014-04-09: PyCon 2014, Montreal, Canada ...               22 days to go
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/85498646/attachment-0001.html>

From ncoghlan at gmail.com  Tue Mar 18 22:04:25 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 19 Mar 2014 07:04:25 +1000
Subject: [Python-ideas] A resources tracker ?
In-Reply-To: <5328A53C.2080604@ziade.org>
References: <5328A53C.2080604@ziade.org>
Message-ID: <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com>

On 19 Mar 2014 06:09, "Tarek Ziad?" <tarek at ziade.org> wrote:
>
> Hey
>
> ResourceWarning instances that are created in some classes' __del__
> function, like FileIO
> are a great tool to track down a program bad behavior.
>
> In network programming, that's even more important to avoid crashes, or
> huge leaks.
>
> But some of them are very hard to fix because we don't get much context,
> we just
> get warnings at the end of the program execution, when gc.collect is
called.
>
> sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb'>
> sys:1: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb'>
>
> Here I just know that somewhere, 2 file descriptors where not closed.
>
> What I'd like to be able to do is to track down the origin of those
> warnings.
>
> Since __del__ is called asynchronously, it's impossible to track it
> right now (or I don't know how)
>
> What we need is a way to keep track of any resource allocation *when it
> happens*.
>
> Here's an idea: let's add three private functions in Python's io:
>
> def __allocate_resource(fd) => records the file descriptor that was
> allocated, along with the current traceback.
> def __free_resource(fd) => removes the fd from the list.
> def __is_resource_allocated(fd) => tell if the resource is in the list.
>
> These three functions, plugged in somewhere in io's classes, could be
> used in conjunction with ResourceWarning:
> when __del__ is called, if the resource was not freed - we'd be able to
> know where it was created.
>
> Of course these functions are just a brain dump - I have no idea how io
> internals work. But unless I missed it,
> something like I've just described is missing in Python.

You should be able to experiment with something based on tracemalloc
(although it may require patching the io implementation or else installing
a GC callback that looks for particular types).

Cheers,
Nick.

>
> Cheers
> Tarek
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/e746ce73/attachment.html>

From tarek at ziade.org  Tue Mar 18 22:07:28 2014
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Tue, 18 Mar 2014 22:07:28 +0100
Subject: [Python-ideas] A resources tracker ?
In-Reply-To: <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com>
References: <5328A53C.2080604@ziade.org>
 <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com>
Message-ID: <5328B590.80700@ziade.org>

Le 18/03/14 21:49, Andrew Barnert a ?crit :
> On Mar 18, 2014, at 12:57, Tarek Ziad? <tarek at ziade.org> wrote:
>
>> def __allocate_resource(fd) => records the file descriptor that was
>> allocated, along with the current traceback.
>> def __free_resource(fd) => removes the fd from the list.
>> def __is_resource_allocated(fd) => tell if the resource is in the list
> I like the general idea. I've actually written wrappers (in Python 2.x and other languages like C++, never Python 3, but similar idea...) to track these kinds of leaks myself.
>
> I don't think you need all of these methods. Just the first one will do it: if you get to __del__ without a close and emit a ResourceWarning, use the info stashed by the allocate function; otherwise, it never gets looked at. And I'm not sure it needs to be a method after all.
True.
>
> What about cases where the io object doesn't actually allocate anything (because you got it from an fd or a socket object or similar?), but you still need to call close. Don't you want the traceback in those cases too? Also, in the most common cases (like open), you're actually creating a chain of two or three objects; do all of them need to store this info?
I guess the allocate_resource function would be only called when the FD
is created. And we'd want this initial traceback afaik


>
> I don't think you want this on all the time... But when exactly _do_ you want it on? Debug mode? If the warning is enabled at allocation time? A global flag on the io module?
Yeah that would be costly, I was thinking about some kind of debug flag
to activate it. I would not mind having to compile python
--with-debug-leaks just to get that kind
of tooling.  

>
> Also, storing an actual traceback is probably a bad idea, as it keeps all kinds of things alive for a very long time. Maybe storing a string representation, or enough info to generate such a representation in the ResourceWarning?
yeah good point. 
>
> Also, I suspect that knowing the arguments used to allocate the resource (like the filename passed to open or the object's constructor, in the most common case, but also things like the cloexec flag when debugging multi-process apps) might be at least as useful as the traceback, so you might as well add that too.
>
> Anyway, there's pure Python code in Lib/io.py that wraps up the C code. To experiment with this without getting into the C stuff, I think you could edit it to use your own classes that wrap _io.BufferedWriter, etc. instead of just exporting those directly.
According to my attempts, It's hard to make sure all the calls are
really going through those classes. 

Another approach is to hack ResourceWarning itself to give the idea a try

Cheers


From tarek at ziade.org  Tue Mar 18 22:19:06 2014
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 18 Mar 2014 22:19:06 +0100
Subject: [Python-ideas] A resources tracker ?
In-Reply-To: <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com>
References: <5328A53C.2080604@ziade.org>
 <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com>
Message-ID: <5328B84A.5000602@ziade.org>

Le 18/03/14 22:04, Nick Coghlan a ?crit :
>
> ..
>
> You should be able to experiment with something based on tracemalloc
> (although it may require patching the io implementation or else
> installing a GC callback that looks for particular types).
>

We've played a little bit with Victor on this today, using his script
here: https://bitbucket.org/haypo/misc/src/tip/python/res_warn.py

That did not fully work - I still need to investigate, but that's the
general idea.

I suspect modifying ResourceWarning itself to track down things would
ensure we're not missing anything.

Cheers
Tarek



From njs at pobox.com  Tue Mar 18 22:48:27 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 18 Mar 2014 21:48:27 +0000
Subject: [Python-ideas] overloading chained comparison
Message-ID: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>

Hi python-ideas,

Since Nick is being a killjoy and shutting down bikeshedding in the @
thread [1] until the numpy folks finish bikeshedding over it
themselves [2], I thought I'd throw out another idea for...
brainstorming [3].

Guido has suggested that while PEP 335 (overloading 'and', 'or',
'not') is rejected, he might be amenable to making chained comparisons
like a < b < c overloadable [4]. The idea is that right now,
  a < b < c
always expands to
  (a < b) and (b < c)
and the 'and' forces boolean coercion. When working with arrays, '<'
is often used elementwise, so we want the 'and' to apply elementwise
as well; in numpy, this is done using '&' (since we can't overload
'and'). So if a, b, c are numpy arrays, we'd like this to instead
expand to
  (a < b) & (b < c)
Similar considerations apply to other systems that overload the
comparison operators, e.g. DSLs for generating SQL queries.

This seems like a good and straightforward enough idea to me in
principle, but it's not at all clear to me what the best way to
accomplish it in practice is. I thought of three options, but none is
obviously Right.

To have a way to talk about our options precisely, let's pretend that
there's something called operator.chain_comparison, and that the way
it works is that
  a < b <= c
produces a call like
  operator.chain_comparison([a, b, c], [operator.lt, operator.le])

Right now, the semantics are:

# Current
def chain_comparison(args, ops):
    for i, op in enumerate(ops):
        result = op(args[i], args[i + 1]):
        # short-circuit
        if not result:
            return result
    return result

(Of course in reality in CPython the compiler unrolls the loop and
inlines this directly into the bytecode, but whatever, that's an
implementation detail.)

IDEA 1: simple, neat and (sort of) wrong

Let's define a new special method __chain_and__; whenever we do a
chain comparison, we check for the presence of this method, and if
found, we call it instead of using 'and'. The intuition is that if we
have a < b < c then this expands to either
  (a < b) and (b < c)
or
  (a < b).__chain_and__(b < c)
depending on whether hasattr((a < b), "__chain_and__"). Notice that
the first case is short-circuiting, and the second is not. Which seems
totally fine (contra PEP 335), because short-circuiting by definition
requires that you make a boolean decision (quit early/don't quit
early), and the whole point of these overloads is to avoid boolean
coercion. I think in general the semantics here look like:

# __chain_and__
def chain_comparison(args, ops):
    so_far = True
    for i, op in enumerate(ops):
        result = op(args[i], args[i + 1])
        if hasattr(so_far, "__chain_and__"):
            so_far = so_far.__chain_and__(result)
        else:
            so_far = so_far and result
        # short-circuit, but only if the next reduction would use 'and':
        if not hasattr(so_far, "__chain_and__") and not so_far:
            return so_far
    return so_far

So if 'arr' is a numpy array, then code like
  0 < arr < 1
will now work great, because (0 < arr) will return an array of
booleans, and this array will have the __chain_and__ method, so we'll
end up doing
  (0 < arr).__chain_and__(arr < 1)
and successfully return a single array of booleans indicating which
locations in 'arr' are between 0 and 1.

But -- suppose that we have, say, a GUI displaying a barchart, and we
have a horizontal "threshold" line across the barchart that the user
control. The idea is that they can move it up and down, and all the
bars that it overlaps will change color, so we can easily see which
bars are above the threshold and which are below it. So given the
current threshold 't' (an ordinary float or int), and a 1d array
holding the various bar heights, we can request the set of overlapping
bars as:
  0 < x < arr
Okay, first problem is that this fails because here we don't want to do
  (0 < x).__chain_and__(x < arr)
because (0 < x) is just False. So okay, let's say we enhance the above
definition to allow for __rchain_and__, and we get
  (x < arr).__rchain_and__(0 < x)
Great! So we implement it and we test it in our program and it works
-- under ordinary conditions, this spits out a nice 1d array of
booleans, our code continues on its merry way, using the values in
this array to decide how each bar in our plot should be colored.

Until the user slides the threshold line down below zero, at which
point (0 < x) starts returning False, and we short-circuit out before
even evaluating (x < arr). And then our code blows up because instead
of getting an array like we expected, we just get False instead. Oops.
 __rchain_and__ is useless.

So I guess the solution is to write
  arr > x > 0
which will always work? I'm not sure how common this case is in
practice, but I find it somewhat disturbing that with the
__chain_and__ approach, a < b < c and c > b > a can return completely
different values even for completely well-behaved objects, and there's
nothing the person writing overload methods can do about it. Certainly
the bug in our original code is not obvious to the casual reader. (And
AFAICT the proposal in PEP 335 also has this problem -- in fact PEP
335 basically is the same as this proposal -- so no help there.)

IDEA 2: the FULLY GENERAL solution (uh oh)

So we started with a call like
  operator.chain_comparison([a, b, c], [operator.lt, operator.le])
Maybe what we should do is treat *this* as the basic operator, and try
calling a special __chain_comparison__ method on a, b, and/or c.

Of course this immediately runs into a problem, because all python's
existing operators have only 1 or 2 arguments [5], not an indefinite
and varying number of arguments. So we can't use the standard
__X__/__rX__ dispatch strategy. We need something like multimethod
dispatch. (Cue thunder, ominous music.) I know this has been discussed
to death, and I haven't read the discussion, so I guess people can
have fun educating me if they feel like it. But what I'd suggest in
this case is to do what we did in numpy to solve a similar problem
[6]. Instead of using a "real" multimethod system, just directly
generalize the __X__/__rX__ trick: when looking for a candidate object
to call __chain_comparison__ on, take the leftmost one that (a) hasn't
been tried, and (b) doesn't have another object which is a proper
subclass of its type that also hasn't been tried. If you get
NotImplemented, keep trying until you run out of candidates; then fall
back to the traditional 'and'-based semantics.

This does solve our problematic case above: only 'arr' implements
__chain_comparison__, so we have
  0 < x < arr
becoming
  arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt])
and then that can just call the underlying rich comparison operators
in a non-short-circuiting loop, and combine the results using '&'
instead of 'and'.

But it does require a somewhat odd looking piece of machinery for the dispatch.

OPTION 3: the FULLY SPECIFIC solution

A nice thing about the __chain_comparison__ method is that it will
actually be identical for all array-like objects, no matter which
library they're defined in. So, if there are multiple array-like
objects from different libraries in a single chain, it doesn't matter
who gets picked to handle the overall evaluation -- any one of them is
good enough, and then the actual interoperability problems are
delegated to the rich comparison methods, which already have to have a
strategy for dealing with them.

In fact, this same __chain_comparison__ can also probably be used for
just about anyone who wants to define it -- e.g., I think most DB
query DSLs are going to overload & to mean "and", right?

So, maybe all we need is a flag that says: if you see an object with
this flag anywhere in a chain, then switch to these semantics:

# &-flag
def chain_comparison(args, ops):
    for i, op in enumerate(ops):
        result = op(args[i], args[i + 1]):
        if i == 0:
            combined = result
        else:
            combined &= result
    return combined

...and otherwise, use the standard semantics.

So that's what I got. None of these approaches seems obviously Right,
but they're all at least sort of viable. Anyone else got any ideas?

-n

[1] https://mail.python.org/pipermail/python-ideas/2014-March/027174.html
[2] http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069444.html
[3] "We're more of the love, bikeshedding, and rhetoric school. Well,
we can do you bikeshedding and love without the rhetoric, and we can
do you bikeshedding and rhetoric without the love, and we can do you
all three concurrent or consecutive. But we can't give you love and
rhetoric without the bikeshedding. Bikeshedding is compulsory."
[4] https://mail.python.org/pipermail/python-dev/2012-March/117510.html
[5] Except pow(), but that doesn't really count because it never
dispatches on the third argument.
[6] The next release of numpy allows non-numpy array-like classes to
handle all numpy math functions in a generic way by dispatch to a
special __numpy_ufunc__ method; I can provide more details on what's
going on here if anyone's curious:
http://docs.scipy.org/doc/numpy-dev/reference/arrays.classes.html#numpy.class.__numpy_ufunc__

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From ncoghlan at gmail.com  Tue Mar 18 23:35:54 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 19 Mar 2014 08:35:54 +1000
Subject: [Python-ideas] overloading chained comparison
In-Reply-To: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>
References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>
Message-ID: <CADiSq7f4x1Jj9Z9TnE4yqHhRvxH3_=iAaOkA4nRNkKXts0Bndw@mail.gmail.com>

On 19 Mar 2014 07:49, "Nathaniel Smith" <njs at pobox.com> wrote:
> [3] "We're more of the love, bikeshedding, and rhetoric school. Well,
> we can do you bikeshedding and love without the rhetoric, and we can
> do you bikeshedding and rhetoric without the love, and we can do you
> all three concurrent or consecutive. But we can't give you love and
> rhetoric without the bikeshedding. Bikeshedding is compulsory."

I haven't formulated an opinion on the actual question yet, but I just
wanted to say this is one of my favourite descriptions of python-ideas/dev
ever :)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/e1e9e168/attachment.html>

From mal at egenix.com  Wed Mar 19 00:38:54 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Mar 2014 00:38:54 +0100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>	<CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>	<CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>	<CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>	<CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com>	<CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com>	<CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com>	<CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com>	<lg986n$n63$1@ger.gmane.org>	<CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com>	<532839B0.7020108@egenix.com>
 <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com>
Message-ID: <5328D90E.4030603@egenix.com>

On 18.03.2014 21:58, Nick Coghlan wrote:
> On 18 Mar 2014 22:18, "M.-A. Lemburg" <mal at egenix.com> wrote:
>> [...matrix method slots space tradeoff...]
> [... need for implementing number methods even on non-number types ...]
> So a *lot* of types already have their PyNumberMethods slot allocated
> (including sequence types implemented in Python), and I'd like to expand
> that to include all sequence types, even those implemented in C.
> 
> That makes the space trade-off here substantially less clear, particularly
> if matrix power ends up being added in the future (it has been dropped from
> the current PEP).

Ok, point taken :-)

> I accept that adding the new slots to PyNumberMethods is more conceptually
> coherent than I thought, though. There's also the fact that we already mess
> about populating different slots for pragmatic reasons that have nothing to
> do with conceptual integrity :)

Perhaps something to revisit for Python 4 ?!

The refactoring could lead to some space savings, untangle code and
make the C API easier to understand.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 19 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-03-29: PythonCamp 2014, Cologne, Germany ...          10 days to go
2014-04-09: PyCon 2014, Montreal, Canada ...               21 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From greg.ewing at canterbury.ac.nz  Wed Mar 19 01:03:06 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 19 Mar 2014 13:03:06 +1300
Subject: [Python-ideas] overloading chained comparison
In-Reply-To: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>
References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>
Message-ID: <5328DEBA.1040000@canterbury.ac.nz>

Nathaniel Smith wrote:

> This does solve our problematic case above: only 'arr' implements
> __chain_comparison__, so we have
>   0 < x < arr
> becoming
>   arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt])

I don't see how that works, because you need to evaluate
arr in order to tell whether it has a __chain_comparison__
method. So chained comparisons would always have to evaluate
all operands and could never short-circuit.

-- 
Greg

From aj at erisian.com.au  Wed Mar 19 01:54:54 2014
From: aj at erisian.com.au (Anthony Towns)
Date: Wed, 19 Mar 2014 10:54:54 +1000
Subject: [Python-ideas] OrderedDict literals
Message-ID: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>

Hi,

I was re-reading some old threads about ordereddict literals to see if
any of them had gotten anywhere. Amongst them, I came across a post by
Tim Delaney:

  https://mail.python.org/pipermail/python-ideas/2011-January/009049.html

that mentioned an odict literal of ['key': 'value', 'key2': 'value2']
could be confused with slice notation.

>From a syntax point-of-view, that doesn't seem to be true (as
mentioned in some of the replies to that thread), but it seems like
you can abuse the similarity to make it a little easier to declare
ordereddicts:

from collections import OrderedDict
class ODSlicer(object):
    def __getitem__(self, key):
        if type(key) is slice:
            key = [key]
        od = OrderedDict()
        for k in key:
            if type(k) is slice:
                od[k.start] = k.stop
            else:
                od[k] = k
        return od
od = ODSlicer()

print(od[1:2])
print(od["a":"b", "c":5])
print(od['a':'b', 'c':'d', ..., 'a':10, 'e':'f'])


You could then replace:

mydict = {
   'foo':    'bar',
   'baz':   'quux',
}

with:

mydict = od[
   'foo':    'bar',
   'baz':   'quux',
]

if you need to convert a hardcoded dict into a hardcoded ordereddict.
Works fine in python2.7 and python3.4.


At this point, I'd like to note in my defence that this isn't called
the python-good-ideas list :)

What's the actual objection to supporting ['foo': 'bar'] odict
literals? I saw Guido gave a -100, way back in the day, but no actual
explanation for why it was distasteful?

    https://mail.python.org/pipermail/python-ideas/2009-June/004924.html

Cheers,
aj

-- 
Anthony Towns <aj at erisian.com.au>

From njs at pobox.com  Wed Mar 19 02:56:00 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 19 Mar 2014 01:56:00 +0000
Subject: [Python-ideas] overloading chained comparison
In-Reply-To: <5328DEBA.1040000@canterbury.ac.nz>
References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com>
 <5328DEBA.1040000@canterbury.ac.nz>
Message-ID: <CAPJVwB=Hb-i+2ixmpO7gP_E6ugfWpjbhkUSc+iOe4PC7=YhkQQ@mail.gmail.com>

On Wed, Mar 19, 2014 at 12:03 AM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Nathaniel Smith wrote:
>
>> This does solve our problematic case above: only 'arr' implements
>> __chain_comparison__, so we have
>>   0 < x < arr
>> becoming
>>   arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt])
>
>
> I don't see how that works, because you need to evaluate
> arr in order to tell whether it has a __chain_comparison__
> method. So chained comparisons would always have to evaluate
> all operands and could never short-circuit.

Blah, you're absolutely right of course, total brainfart about what
short-circuiting means, even though it was right there in the
disassembly I was looking at.

So yes, I guess the only meaningful part of my message is just the
observation that short-circuiting makes any kind of chained comparison
overload IMHO rather unsatisfying, because it forces you to make the
ugly rule that the special overloading arguments must always occur in
one of the first two positions. Which is probably true by accident 99%
of the time, so as long as short-circuiting exists at all, supporting
chained comparison overload will mean creating an obscure trap.

On the one hand, I am kind of terrified of the idea of code that
depends on short-circuiting here, like:

  0 < x < side_effecting_function_only_called_for_negative_x()

It would be interesting to know how many chained comparisons exist in
the wild with non-trivial 3rd+ arguments.

On the other hand, disabling short-circuiting in general for chained
comparisons would still be a technical compatibility break. Somehow
adding py3's first __future__ feature just for this seems like a lot
to ask...

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From abarnert at yahoo.com  Wed Mar 19 07:44:41 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 18 Mar 2014 23:44:41 -0700
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
Message-ID: <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com>

On Mar 18, 2014, at 17:54, Anthony Towns <aj at erisian.com.au> wrote:

> I was re-reading some old threads about ordereddict literals to see if
> any of them had gotten anywhere.

Skimming the linked thread, there were a bunch of messages with variations of "It might be easier to judge if you came up with a real use case", but nobody ever did.

So, do you have a real use case here? It might look better with headers from some protocol that need to be in order (except I can't think of any such protocol) or whatever you're hoping for than with examples like 'a' or 'foo' or 1.

From aj at erisian.com.au  Wed Mar 19 08:39:11 2014
From: aj at erisian.com.au (Anthony Towns)
Date: Wed, 19 Mar 2014 17:39:11 +1000
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
 <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com>
Message-ID: <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com>

On 19 March 2014 16:44, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Mar 18, 2014, at 17:54, Anthony Towns <aj at erisian.com.au> wrote:
>> I was re-reading some old threads about ordereddict literals to see if
>> any of them had gotten anywhere.
> Skimming the linked thread, there were a bunch of messages with variations of "It might be easier to judge if you came up with a real use case", but nobody ever did.
> So, do you have a real use case here? It might look better with headers from some protocol that need to be in order (except I can't think of any such protocol) or whatever you're hoping for than with examples like 'a' or 'foo' or 1.

Oh, sure. The case that I've done most recently was when writing a
tool to let me convert yaml files to and from jira issues. I want to
specify how to convert each field between yaml and jira, and I also
want to specify the order the fields end up in in the yaml file so
that they're predictable. That's static information that I want to
include in my source, and having it as a single ordereddict is the
easiest way to write and store it. (The reason I want the ordering is
both so that (a) if I dump an issue from jira to a file one day, and
store it in git, then do it again the next day, the diff catches any
intentional changes, not just random ordering changes, and (b) when
I'm looking at the yaml file, the fields are in an order that makes
sense to me so I know where to look for particular information).

The code I ended up with was:

 class InterestingFields(OrderedDict):
    def __init__(self, extras):
        super(InterestingFields, self).__init__()

        D_key = D("key")
        D_name = D("name")
        s = S()
        t = T()
        f = self

        f["project"] =      D_key
        f["issuetype"] =    D_name
        f["summary"] =      s
        # 18 more key/value pairs ...

        for k,v in extras:
            f[k] = v

where D,S,T are classes that provide functions to convert from the
Jira REST representation to the python version of the YAML
representation I want and vice-versa.

It took me a while to do the above in a way I didn't totally hate
("oh, I'll just do an OrderedDict( project=D_key, issuetype=D_name,
...)... huh, why didn't that work?"), and I can't say I *liked* what I
ended up with. Better than specifying the fields twice (once for how
to convert, once for the ordering), and better than all the
heiroglyphics involved in writing it as a list of pairs though.

The use case that was crossing my mind this morning was being able to write:

    return { "result": "ok", "details": "everything worked wonderfully" }

from a function whose result gets turned into json and returned over a
REST API, and have the order preserved so that when debugging with
curl, I wouldn't see the "details" come first and confuse myself.
Haven't decided how much I care about that yet.

Cheers,
aj

-- 
Anthony Towns <aj at erisian.com.au>

From mertz at gnosis.cx  Wed Mar 19 09:00:51 2014
From: mertz at gnosis.cx (David Mertz)
Date: Wed, 19 Mar 2014 01:00:51 -0700
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
Message-ID: <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com>

It doesn't feel to me like the existing (non-literal) spelling is
particularly bad:

>>> o1 = OrderedDict(foo=1, bar=2, baz=3)
>>> o2 = OrderedDict([('foo',1), ('bar',2), ('baz',3)])
>>> o1
OrderedDict([('bar', 2), ('foo', 1), ('baz', 3)])
>>> o2
OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])

So sure, you can't use the kw=val style of initialization, but the "list of
pairs" style works fine.  It's not that much longer than a literal would be.


On Tue, Mar 18, 2014 at 5:54 PM, Anthony Towns <aj at erisian.com.au> wrote:

> Hi,
>
> I was re-reading some old threads about ordereddict literals to see if
> any of them had gotten anywhere. Amongst them, I came across a post by
> Tim Delaney:
>
>   https://mail.python.org/pipermail/python-ideas/2011-January/009049.html
>
> that mentioned an odict literal of ['key': 'value', 'key2': 'value2']
> could be confused with slice notation.
>
> From a syntax point-of-view, that doesn't seem to be true (as
> mentioned in some of the replies to that thread), but it seems like
> you can abuse the similarity to make it a little easier to declare
> ordereddicts:
>
> from collections import OrderedDict
> class ODSlicer(object):
>     def __getitem__(self, key):
>         if type(key) is slice:
>             key = [key]
>         od = OrderedDict()
>         for k in key:
>             if type(k) is slice:
>                 od[k.start] = k.stop
>             else:
>                 od[k] = k
>         return od
> od = ODSlicer()
>
> print(od[1:2])
> print(od["a":"b", "c":5])
> print(od['a':'b', 'c':'d', ..., 'a':10, 'e':'f'])
>
>
> You could then replace:
>
> mydict = {
>    'foo':    'bar',
>    'baz':   'quux',
> }
>
> with:
>
> mydict = od[
>    'foo':    'bar',
>    'baz':   'quux',
> ]
>
> if you need to convert a hardcoded dict into a hardcoded ordereddict.
> Works fine in python2.7 and python3.4.
>
>
> At this point, I'd like to note in my defence that this isn't called
> the python-good-ideas list :)
>
> What's the actual objection to supporting ['foo': 'bar'] odict
> literals? I saw Guido gave a -100, way back in the day, but no actual
> explanation for why it was distasteful?
>
>     https://mail.python.org/pipermail/python-ideas/2009-June/004924.html
>
> Cheers,
> aj
>
> --
> Anthony Towns <aj at erisian.com.au>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/430b9bac/attachment.html>

From abarnert at yahoo.com  Wed Mar 19 19:08:54 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 19 Mar 2014 11:08:54 -0700
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
 <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com>
 <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com>
Message-ID: <6A8172AE-7BBF-4005-9D6F-74D5B9259A41@yahoo.com>

On Mar 19, 2014, at 0:39, Anthony Towns <aj at erisian.com.au> wrote:

> class InterestingFields(OrderedDict):
>   def __init__(self, extras):
>       super(InterestingFields, self).__init__()
> 
>       D_key = D("key")
>       D_name = D("name")
>       s = S()
>       t = T()
>       f = self
> 
>       f["project"] =      D_key
>       f["issuetype"] =    D_name
>       f["summary"] =      s
>       # 18 more key/value pairs ...
> 
>       for k,v in extras:
>           f[k] = v
> 
> where D,S,T are classes that provide functions to convert from the
> Jira REST representation to the python version of the YAML
> representation I want and vice-versa.

My first question is: why are you retrieving all 21 fields into named variables, and then putting them all into the dict? Why not just:

   f["project"] = D("key")

Also, why does this have to be an OrderedDict in the first place? This isn't a collection of arbitrary fields where you want to preserve the order you found them in, it's primarily a collection of a fixed set of fields, which may have some extra fields at the end. In other words, an object. 

If the only issue is that you want to dump all your fields to YAML, you just need a reduce-type method that dumps them in the right order. You could do that in a variety of different ways; using an OrderedDict for your instance __dict__ is the obvious one, but your could also use a namedtuple instead of a normal class, use __slots__ (and store the extras in a single OrderedDict attribute), etc.

If the problem is how to take the extras in a way that's nicer than an iterable of pairs so you don't need all those parens and brackets when you construct it, you can remove either the brackets or the parens or both by just using *extras at the end and taking the pairs as args, or taking the keys and values as alternating args.

I don't mean to nitpick here, but I suspect this is exactly the kind of thing people were expecting--use cases that do unpythonic things like blur the line between objects and dicts ala JavaScript. Of course when you're dealing in JSON, or to some extent YAML, it's hard to completely avoid that blurring, but blurring it even farther isn't the solution.

From ericsnowcurrently at gmail.com  Wed Mar 19 22:24:16 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 19 Mar 2014 15:24:16 -0600
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
 <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com>
Message-ID: <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com>

On Wed, Mar 19, 2014 at 2:00 AM, David Mertz <mertz at gnosis.cx> wrote:
> It doesn't feel to me like the existing (non-literal) spelling is
> particularly bad:
>
>>>> o1 = OrderedDict(foo=1, bar=2, baz=3)
>>>> o2 = OrderedDict([('foo',1), ('bar',2), ('baz',3)])
>>>> o1
> OrderedDict([('bar', 2), ('foo', 1), ('baz', 3)])
>>>> o2
> OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])
>
> So sure, you can't use the kw=val style of initialization,

FYI I'm working on a proposal to support preserving the call order of
**kwargs.  This would allow initializing an OrderedDict correctly
using keyword args.  I may have something to show for it by the end of
the PyCon sprints.

-eric

From random832 at fastmail.us  Wed Mar 19 23:01:20 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 19 Mar 2014 18:01:20 -0400
Subject: [Python-ideas] OrderedDict literals
In-Reply-To: <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com>
References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com>
 <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com>
 <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com>
Message-ID: <1395266480.19830.96554629.30A054FF@webmail.messagingengine.com>

On Wed, Mar 19, 2014, at 17:24, Eric Snow wrote:
> FYI I'm working on a proposal to support preserving the call order of
> **kwargs.  This would allow initializing an OrderedDict correctly
> using keyword args.  I may have something to show for it by the end of
> the PyCon sprints.

I know that it's possible to support this* in dict, but is kwargs
required to be a dict? What route are you going: modify dict to allow
this and require dict to support it; make a new class for kwargs and
require this behavior of kwargs; modify dict on CPython to allow it but
only require that kwargs support it "somehow" on non-CPython?

*where "this" is e.g. "preservation of original insertion order for
iterating when there have been no deletions"

From ericsnowcurrently at gmail.com  Thu Mar 20 00:32:27 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 19 Mar 2014 17:32:27 -0600
Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict
	literals)
Message-ID: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>

On Wed, Mar 19, 2014 at 4:01 PM,  <random832 at fastmail.us> wrote:
> On Wed, Mar 19, 2014, at 17:24, Eric Snow wrote:
>> FYI I'm working on a proposal to support preserving the call order of
>> **kwargs.  This would allow initializing an OrderedDict correctly
>> using keyword args.  I may have something to show for it by the end of
>> the PyCon sprints.
>
> I know that it's possible to support this* in dict, but is kwargs
> required to be a dict?

I'm not talking about the ** unpacking syntax, but about the function
definition syntax for **kwargs:

  def spam(a, b, **kwargs): ...

> What route are you going: modify dict to allow
> this and require dict to support it; make a new class for kwargs and
> require this behavior of kwargs; modify dict on CPython to allow it but
> only require that kwargs support it "somehow" on non-CPython?

Hopefully I'll have time to write a proto-PEP on this in the next
couple weeks, but the gist is that I see 2 options:

1. Change kwargs to an OrderedDict for functions decorated a special decorator:

  @preserve_kwargs_order
  def spam(a, b, **kwargs):
     ...  # kwargs will be an OrderedDict

2. Store the ordered keys in a list and bind that to a special local
variable, but only for functions that have **kwargs:

  def spam(a, b, **kwargs):
      kwargs = OrderedDict((k, kwargs[k]) for k in __kwargs_order__)
      ...

My gut feeling is that option 2 is the better choice.  And before you
ask, Guido has already vetoed making **kwargs always be an
OrderedDict.  There are a few other details to work out, but the above
is the main thing.

Also, an OrderedDict implemented in C will likely be a prerequisite,
but luckily I've had a patch up since July(?) (#16991).  It doesn't
apply cleanly at present, but I expect that's only barely and I should
be able to get a clean patch up when I have a chance.  There wasn't a
lot of interest in reviewing such a large patch so I'd back-burnered
it until Python 3.4 got released (which just happened).  Once I get a
couple reviews I should be able to get that in (wink wink nudge
nudge).

-eric

From rosuav at gmail.com  Thu Mar 20 01:13:28 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 20 Mar 2014 11:13:28 +1100
Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict
	literals)
In-Reply-To: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
Message-ID: <CAPTjJmoKzMB+cef-1RD8FwBYVpAf6GQQBmBD-y=ayJocGLO7vw@mail.gmail.com>

On Thu, Mar 20, 2014 at 10:32 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> 1. Change kwargs to an OrderedDict for functions decorated a special decorator:
>
>   @preserve_kwargs_order
>   def spam(a, b, **kwargs):
>      ...  # kwargs will be an OrderedDict
>
> 2. Store the ordered keys in a list and bind that to a special local
> variable, but only for functions that have **kwargs:
>
>   def spam(a, b, **kwargs):
>       kwargs = OrderedDict((k, kwargs[k]) for k in __kwargs_order__)
>       ...
>
> My gut feeling is that option 2 is the better choice.

Strongly against any proposal that makes argument passing behave
differently based on the target - it means the process of building up
the arguments has to check some attribute on the function.

Order of kwargs is a problem for any function that passes args through
to another function.

def wrap_spam(*args, **kwargs):
    try:
        return spam(*args, **kwargs)
    except EggsError:
        return False

Does wrap_spam have to be aware that spam cares about keyword argument
order? Should it preserve order "just in case"? Should the
__kwargs_order__ list be automatically propagated? And what happens if
the args aren't passed through perfectly, but some are added/removed?

ChrisA

From abarnert at yahoo.com  Thu Mar 20 02:58:30 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 19 Mar 2014 18:58:30 -0700
Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict
	literals)
In-Reply-To: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
Message-ID: <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>

On Mar 19, 2014, at 16:32, Eric Snow <ericsnowcurrently at gmail.com> wrote:

> Hopefully I'll have time to write a proto-PEP on this in the next
> couple weeks, but the gist is that I see 2 options:

Last time around, someone suggested that ***kwargs could get you the args as a list of pairs, or maybe an OrderedDict. While that would require a bit of complexity in the CALL_FUNCTION code, it seems like it would be simpler than your option 2.

But it doesn't solve the main problem. Right now, you can forward any arguments perfectly by doing this:

    def wrapper(*args, **kwargs):
        return wrappee(*args, **kwargs)

Your option 2 would require much more verbose code to forward perfectly. The triple-star idea makes it a lot nicer (assuming that ** passing respects the order, or a new *** is added there as well), but it would still break the thousands of wrapper functions out there written with **kwargs.


From python at mrabarnett.plus.com  Thu Mar 20 03:42:54 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 20 Mar 2014 02:42:54 +0000
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
Message-ID: <532A55AE.5040509@mrabarnett.plus.com>

On 2014-03-20 01:58, Andrew Barnert wrote:> On Mar 19, 2014, at 16:32, 
Eric Snow <ericsnowcurrently at gmail.com> wrote:
 >
 >> Hopefully I'll have time to write a proto-PEP on this in the next
 >> couple weeks, but the gist is that I see 2 options:
 >
 > Last time around, someone suggested that ***kwargs could get you the
 > args as a list of pairs, or maybe an OrderedDict. While that would
 > require a bit of complexity in the CALL_FUNCTION code, it seems like
 > it would be simpler than your option 2.
 >
 > But it doesn't solve the main problem. Right now, you can forward
 > any arguments perfectly by doing this:
 >
 >      def wrapper(*args, **kwargs):
 >          return wrappee(*args, **kwargs)
 >
 > Your option 2 would require much more verbose code to forward
 > perfectly. The triple-star idea makes it a lot nicer (assuming that
 > ** passing respects the order, or a new *** is added there as well),
 > but it would still break the thousands of wrapper functions out
 > there written with **kwargs.
 >

Wouldn't it be a problem only if the dict were unpacked and then
repacked? In the code above, it's merely passing the input kwargs on to
'wrappee'.

From antony.lee at berkeley.edu  Thu Mar 20 07:03:14 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Wed, 19 Mar 2014 23:03:14 -0700
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <532A55AE.5040509@mrabarnett.plus.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
 <532A55AE.5040509@mrabarnett.plus.com>
Message-ID: <CAGRr6BFtZStpvqSZCsrwYo9hp82Si2wWfxsVSRWtwpmzKfHnMw@mail.gmail.com>

Somehow it feels wrong to add a fairly complex new feature to the language
(namely, ordered keyword arguments) to handle the very specific case of
ordered dict constructors (or is there some other (clearly) different use
intended?).  If anything, I prefer the abuse of "slicing" syntax (so that
one can write "od['a': 'b', 'c': 'd']" which is not much worse than "{'a':
'b', 'c': 'd'}").

Let's say we start with

def wrapper(**kwargs):
    <some code that manipulates kwargs>
    return g(**kwargs)

def g(**kwargs): pass

wrapper(a=1, b=2)

Nothing requires ordered keyword arguments, wrapper doesn't bother saving
the argument order, everything is fine (but because there is some code that
manipulates kwargs, an actual dict must be created, not just some sort of
proxy)... now, just after wrapper is executed and before g gets called,
globals()["g"] gets changed (by another thread...) to a function that does
require ordered keyword arguments.  What now?

Antony


2014-03-19 19:42 GMT-07:00 MRAB <python at mrabarnett.plus.com>:

> On 2014-03-20 01:58, Andrew Barnert wrote:> On Mar 19, 2014, at 16:32,
> Eric Snow <ericsnowcurrently at gmail.com> wrote:
> >
> >> Hopefully I'll have time to write a proto-PEP on this in the next
> >> couple weeks, but the gist is that I see 2 options:
> >
> > Last time around, someone suggested that ***kwargs could get you the
> > args as a list of pairs, or maybe an OrderedDict. While that would
> > require a bit of complexity in the CALL_FUNCTION code, it seems like
> > it would be simpler than your option 2.
> >
> > But it doesn't solve the main problem. Right now, you can forward
> > any arguments perfectly by doing this:
> >
> >      def wrapper(*args, **kwargs):
> >          return wrappee(*args, **kwargs)
> >
> > Your option 2 would require much more verbose code to forward
> > perfectly. The triple-star idea makes it a lot nicer (assuming that
> > ** passing respects the order, or a new *** is added there as well),
> > but it would still break the thousands of wrapper functions out
> > there written with **kwargs.
> >
>
> Wouldn't it be a problem only if the dict were unpacked and then
> repacked? In the code above, it's merely passing the input kwargs on to
> 'wrappee'.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/1b016632/attachment.html>

From abarnert at yahoo.com  Thu Mar 20 07:43:21 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 19 Mar 2014 23:43:21 -0700 (PDT)
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <532A55AE.5040509@mrabarnett.plus.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
 <532A55AE.5040509@mrabarnett.plus.com>
Message-ID: <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com>

From: MRAB <python at mrabarnett.plus.com>

Sent: Wednesday, March 19, 2014 7:42 PM


> On 2014-03-20 01:58, Andrew Barnert wrote:
>> On Mar 19, 2014, at 16:32, 
>> Eric Snow <ericsnowcurrently at gmail.com> wrote:
>>>  Hopefully I'll have time to write a proto-PEP on this in the next
>>>  couple weeks, but the gist is that I see 2 options:
>> 
>>  But it doesn't solve the main problem. Right now, you can forward
>>  any arguments perfectly by doing this:
>> 
>> ? ? ? def wrapper(*args, **kwargs):
>> ? ? ? ? ? return wrappee(*args, **kwargs)
>> 
>>  Your option 2 would require much more verbose code to forward
>>  perfectly.
>> ? it would still break the thousands of wrapper functions out

>>  there written with **kwargs.
> 
> Wouldn't it be a problem only if the dict were unpacked and then
> repacked? In the code above, it's merely passing the input kwargs on to
> 'wrappee'.

No, a **kwargs parameter packs the keyword arguments into a dict, and a **kwargs argument unpacks a dict into the keyword arguments; that's exactly what they do. And dicts have arbitrary order.

Besides, the whole point of option 2 is that kwargs?is still just a plain dict, and the order is passed via a magic hidden parameter __kwargs_order__. So, even if you _were_ just passing kwargs on without doing anything to it, that still wouldn't help.?As long as wrapper (and every other general-purpose wrapper every written) doesn't do anything with __kwargs_order__, the order is not going to get passed to wrapped.

The obvious way to fix this is to make a **kwargs parameter pack the keyword arguments into an OrderedDict, but Guido has already rejected that, which is why Eric Snow had to come up with his two other options. But they don't solve the problem.

From jared.grubb at gmail.com  Thu Mar 20 08:41:40 2014
From: jared.grubb at gmail.com (Jared Grubb)
Date: Thu, 20 Mar 2014 00:41:40 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
Message-ID: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>

A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions")

Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it.

I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special?

I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :)) 

Jared

QUICK SUMMARY OF ANDREW'S EMAIL:
(Not a proposal for syntax or naming)

    m `cross` n
    m `dot` n

    a `Pair` b
    a `Tree` (b `Tree` c `Tree` d) `Tree` e

And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too:

    m `mmul` n
    m `mdiv` n


From ncoghlan at gmail.com  Thu Mar 20 09:02:54 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 20 Mar 2014 18:02:54 +1000
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
Message-ID: <CADiSq7d8727d+rK7oVX0M-D2-BUd_+thAnBXUZr2G5F-VLw5hQ@mail.gmail.com>

On 20 March 2014 17:41, Jared Grubb <jared.grubb at gmail.com> wrote:
> A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions")
>
> Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it.
>
> I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special?

Yes, basically - a large portion of the PEP is about pointing out that
matrix multiplication really is that special. All past attempts at
getting syntax for it *have* been generalised along that lines of this
suggestion from Andrew, and have ended up failing on the grounds of
"just use a method or function instead".

Nathaniel's PEP is about looking at those past proposals and the
experience of the numeric computing community and asking "Which
operator have we *really* missed?". And they realised that it was only
one: a way to spell matrix multiplication, since "*" has already been
claimed for element-wise multiplication.

The generalised proposals failed because they were still clumsy for
the matrix multiplication use case, and there wasn't a compelling
justification for the extra complexity that came with the
generalisation.

By contrast, Nathaniel's latest PEP looked at solving the *exact*
problem the numeric community has: a clean and concise way to spell
matrix multiplication. It hit the sweet spot of design proposals: a
clean targeted solution that is as simple as it can possibly be, while
still solving the problem that needs to be solved.

It may have *looked* deceptively fast in terms of core devs saying
"yes, that's a good idea", but the speed makes more sense if you start
the timer from some of the earlier PEPs referenced from Nathaniel's
one - the "matrix multiplication is clumsy compared to MATLAB" problem
has been around for as long as people have been doing numeric
computing in Python.

Cheers,
Nick.

>
> I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :))
>
> Jared
>
> QUICK SUMMARY OF ANDREW'S EMAIL:
> (Not a proposal for syntax or naming)
>
>     m `cross` n
>     m `dot` n
>
>     a `Pair` b
>     a `Tree` (b `Tree` c `Tree` d) `Tree` e
>
> And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too:
>
>     m `mmul` n
>     m `mdiv` n
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rosuav at gmail.com  Thu Mar 20 09:58:47 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 20 Mar 2014 19:58:47 +1100
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
Message-ID: <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com>

On Thu, Mar 20, 2014 at 6:41 PM, Jared Grubb <jared.grubb at gmail.com> wrote:
> But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :))
>
> Jared
>
> QUICK SUMMARY OF ANDREW'S EMAIL:
> (Not a proposal for syntax or naming)
>
>     m `cross` n
>     m `dot` n

Sometimes a proposal fails because no good syntax can be found for it.
The strongest objection to my PEP 463 on exception expressions is "I
don't like the syntax", and to date nobody's come up with a syntax
that everyone likes. (Four proposals in the PEP itself, all somewhat
flawed.)

In the above case, the proposal somewhat falls down for the same
reason. Backticks have been, by BDFL fiat, banned from syntax; either
that decision has to be reversed, or some other syntax has to be
figured out. People will love or hate the proposal based on the
tentative syntax, and the "grit on Tim's monitor" check, and such.

ChrisA

From steve at pearwood.info  Thu Mar 20 11:24:44 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 20 Mar 2014 21:24:44 +1100
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
Message-ID: <20140320102443.GS16526@ando>

On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote:

> A couple months back, Andrew Barnet brought up the idea of adding a 
> way to apply functions in an infix style. It really didnt gain much 
> traction. (Subject line "Infix Functions")
> 
> Nathaniel Smith has just proposed a PEP to add an infix matrix 
> multiply operator, and everyone seems to love it.
> 
> I honestly am surprised at the difference in reaction. Why are we so 
> quick to add a single-purpose punctuation mark, but reject a named 
> infix operator that works for many general cases? Is the 
> matrix-multiply use case so special?

Yes.

Every day, there are probably thousands of numpy users (to say nothing 
of other matrix-multiplication library users) who do matrix 
multiplication. I can't think of the last time I've wanted to invent my 
own infix operator, and if I ever did, I'd probably be the only person 
using it. I don't think I'm too unusual. Custom infix operators for 
arbitrary functions are extremely niche, because apart from matrix 
multiplication, nearly all the common, useful ones are already 
available. We have + - * / & ^ | etc., that covers the common cases. The 
ones remaining are (apart from matrix multiplication) uncommon.

The beauty of infix operators is that they give an extremely compact 
representation, which is valuable for mathematics but not so much for 
general purpose computing. Consequently, it would be useful to write 
something like:

    matrix @ vector

versus

    matrix.mult(vector)

because the symbol for the operator is so compact. Having to write it 
as this instead:

    matrix`mult`vector

saves you one character over the method syntax, or even costs you one 
character if you write it like this:

    matrix `mult` vector

which is not enough of a saving over the method syntax to make it 
worthwhile. Having operators like + and * is a good thing, but if we 
didn't have them, and had to write this instead:

    x`plus`y
    a`times`b

there'd be no advantage to using pseudo-operators as shown. You might 
as well use regular method calls:

    x.plus(y)
    a.times(b)

(which you'll note is already pseudo-infix). So, in my opinion, until 
such time as Python supports Unicode symbols (around Python 5000 
perhaps?) as operators, there is no point in trying to fake them with 
spelled-out names. If you're going to use a spelled-out name, use a 
method, or function.



-- 
Steven

From robert.kern at gmail.com  Thu Mar 20 11:32:33 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 20 Mar 2014 10:32:33 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
Message-ID: <lgeg3i$2o9$1@ger.gmane.org>

On 2014-03-14 17:53, Guido van Rossum wrote:

> - Did you consider a duck-typing (is that the word?) attribute?
>    E.g. a*b is elementwise multiplication; a.M*b must be used for
>    matrix multiplication.  (Your use of .T as "transpose" made me think
>    of this.)  Of course the question is, can you get those packages
>    that currently use * for matrix multiply to comply? (I don't consider
>    this a serious counter-proposal. But you list a bunch of rejected
>    alternatives; this could be in that list.

Apparently this *was* considered in PEP 225 under "Alternatives to adding new 
operators", numbers 3 and 4.

   http://legacy.python.org/dev/peps/pep-0225/

Of course, number 6 is "Introducing a single operator, such as @, for matrix 
multiplication." :-)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From python at mrabarnett.plus.com  Thu Mar 20 13:58:27 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 20 Mar 2014 12:58:27 +0000
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
 <532A55AE.5040509@mrabarnett.plus.com>
 <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com>
Message-ID: <532AE5F3.9040508@mrabarnett.plus.com>

On 2014-03-20 06:43, Andrew Barnert wrote:
 > From: MRAB <python at mrabarnett.plus.com>
 >
 > Sent: Wednesday, March 19, 2014 7:42 PM
 >
 >
 > > On 2014-03-20 01:58, Andrew Barnert wrote:
 > >> On Mar 19, 2014, at 16:32,
 > >> Eric Snow <ericsnowcurrently at gmail.com> wrote:
 > >>>  Hopefully I'll have time to write a proto-PEP on this in the
 > >>>  next couple weeks, but the gist is that I see 2 options:
 > >>
 > >>  But it doesn't solve the main problem. Right now, you can forward
 > >>  any arguments perfectly by doing this:
 > >>
 > >>       def wrapper(*args, **kwargs):
 > >>           return wrappee(*args, **kwargs)
 > >>
 > >>  Your option 2 would require much more verbose code to forward
 > >>  perfectly.
 > >> ? it would still break the thousands of wrapper functions out
 >
 > >>  there written with **kwargs.
 > >
 > > Wouldn't it be a problem only if the dict were unpacked and then
 > > repacked? In the code above, it's merely passing the input kwargs
 > > on to 'wrappee'.
 >
 > No, a **kwargs parameter packs the keyword arguments into a dict, and
 >  a **kwargs argument unpacks a dict into the keyword arguments;
 > that's exactly what they do. And dicts have arbitrary order.
 >
When I decompile, what I get is:

 >>> dis(compile('''wrappee(**kwargs)''', '<string>', 'exec'))
   1           0 LOAD_NAME                0 (wrappee)
               3 LOAD_NAME                1 (kwargs)
               6 CALL_FUNCTION_KW         0 (0 positional, 0 keyword pair)
               9 POP_TOP
              10 LOAD_CONST               0 (None)
              13 RETURN_VALUE

It looks like the dict 'kwargs' is being passed straight to 'wrappee'.

 > Besides, the whole point of option 2 is that kwargs is still just a
 > plain dict, and the order is passed via a magic hidden parameter
 > __kwargs_order__. So, even if you _were_ just passing kwargs on
 > without doing anything to it, that still wouldn't help. As long as
 > wrapper (and every other general-purpose wrapper every written)
 > doesn't do anything with __kwargs_order__, the order is not going to
 > get passed to wrapped.
 >
 > The obvious way to fix this is to make a **kwargs parameter pack the
 > keyword arguments into an OrderedDict, but Guido has already rejected
 > that, which is why Eric Snow had to come up with his two other
 > options. But they don't solve the problem.
 >


From abarnert at yahoo.com  Thu Mar 20 17:06:09 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 20 Mar 2014 09:06:09 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <20140320102443.GS16526@ando>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
Message-ID: <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>

On Mar 20, 2014, at 3:24, Steven D'Aprano <steve at pearwood.info> wrote:

> On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote:
> 
> The beauty of infix operators is that they give an extremely compact 
> representation, which is valuable for mathematics but not so much for 
> general purpose computing. Consequently, it would be useful to write 
> something like:
> 
>    matrix @ vector
> 
> versus
> 
>    matrix.mult(vector)
> 
> because the symbol for the operator is so compact. Having to write it 
> as this instead:
> 
>    matrix`mult`vector
> 
> saves you one character over the method syntax,

It's not about saving one character, it's about saving the (possibly overly-nested) parentheses. The PEP explains that, as does the earlier thread, and all of the previous threads and PEPs on related ideas, so I won't go into it again.

However, the other answers are right; matrix multiplication _is_ special. As an alternative to the 2000 PEP that proposed a suite of new operators by allowing ~ as a prefix to 11 existing operators )or to the other proposal to allow any string of operator characters to be defined as an operator), I think general infix wins. However, the NumPy community is no longer suggesting either of those; after 14 years of living with the limitations of the current system, they've decided that only one operator is really a problem. And against that proposal, general infix loses. One is a good number of special cases for the same reason 11 is a bad number of special cases.

Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway.

From abarnert at yahoo.com  Thu Mar 20 17:28:34 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 20 Mar 2014 09:28:34 -0700
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <532AE5F3.9040508@mrabarnett.plus.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
 <532A55AE.5040509@mrabarnett.plus.com>
 <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com>
 <532AE5F3.9040508@mrabarnett.plus.com>
Message-ID: <87C2AE45-4FD2-418D-9288-B5BD51D06F27@yahoo.com>

On Mar 20, 2014, at 5:58, MRAB <python at mrabarnett.plus.com> wrote:

> > No, a **kwargs parameter packs the keyword arguments into a dict, and
> >  a **kwargs argument unpacks a dict into the keyword arguments;
> > that's exactly what they do. And dicts have arbitrary order.
> >
> When I decompile, what I get is:
> 
> >>> dis(compile('''wrappee(**kwargs)''', '<string>', 'exec'))
>  1           0 LOAD_NAME                0 (wrappee)
>              3 LOAD_NAME                1 (kwargs)
>              6 CALL_FUNCTION_KW         0 (0 positional, 0 keyword pair)
>              9 POP_TOP
>             10 LOAD_CONST               0 (None)
>             13 RETURN_VALUE

This is basically an optimization, where the unpacking happens inside CALL_FUNCTION_KW instead of in a bunch of separate bytecodes that pushed them onto the stack to be popped off by CALL_FUNCTION. The end result is identical. Inside CALL_FUNCTION_KW, instead of starting with an empty dict and then popping keywords into it, it starts with a copy of kwargs and then pops keywords into it. 

If that weren't true, you wouldn't be able to do this:

    def foo(a, **kwargs): pass
    foo(**{'a': 1, 'b': 2})

Or:

    def foo(**kwargs): pass
    foo(b=2, **{'a': 1})

(If you replace a or b with self, this should look more familiar.)

And, even if you don't have any missing or extra arguments, you are still going to get a new dict inside the callee. Otherwise this would do the wrong thing:

    def foo(**kwargs): kwargs['a'] = 3
    d = {'a': 1, 'b': 2}
    foo(**d)

And, even if you changed Python so  that, in the case where there are no extra or missing keywords and the dict isn't mutated inside foo, you just passed it as-is and skipped the copy, that _still_ wouldn't solve the forwarding problem, because it's still just a dict in arbitrary order. And adding a separate hidden parameter to pass the order along still wouldn't help unless you also forwarded _that_ parameter somehow.

And as far as I can see, there is no way to solve this problem. The only way to add keyword order information without breaking all existing forwarding functions is to make the **kwargs parameter always preserve order (and make the **kwargs calling syntax preserve order if present)--that is, to make it always an OrderedDict, which Guido has already rejected.


From ron3200 at gmail.com  Thu Mar 20 21:14:35 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Thu, 20 Mar 2014 16:14:35 -0400
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
Message-ID: <lgfi6t$45l$1@ger.gmane.org>



On 03/20/2014 12:06 PM, Andrew Barnert wrote:
> Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway.


I like that idea also, but ...

    s = 'a' 'b' 'c'

Is 'b' a string op or a string?


To make it work, it needs an introducer.  Which kind of defeats some of the 
advantage.


Or it needs to be defined with a keyword.  This actually makes sense as 
it's changes the effect of a name.  (like local, non-local, and global.)

   def foo(...):
      symbol b

      s = 'a' b 'c'

One of the objects would need a method to get the symbol, and the other 
argument.

Cheers,
    Ron


From bwmaister at gmail.com  Thu Mar 20 22:04:25 2014
From: bwmaister at gmail.com (Brandon W Maister)
Date: Thu, 20 Mar 2014 17:04:25 -0400
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <lgfi6t$45l$1@ger.gmane.org>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
Message-ID: <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>

On Thu, Mar 20, 2014 at 4:14 PM, Ron Adam <ron3200 at gmail.com> wrote:

>    def foo(...):
>      symbol b
>
>      s = 'a' b 'c'
>

Actually makes me think of decorators, which means that you wouldn't even
need new tokens or keywords:

    from functools import make_operator

    bop = make_operator(b)

    s = 'a' bop 'b'

    class Ops:
        @make_operator
        def cat(left, right):
            return left + right

    t = 'a' Ops.cat 'b'

It reduces the grit, but also reduces certain kinds of obviousness.

bwm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140320/800cf2e9/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Thu Mar 20 22:34:59 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 21 Mar 2014 10:34:59 +1300
Subject: [Python-ideas] Preserving **kwargs order
In-Reply-To: <532AE5F3.9040508@mrabarnett.plus.com>
References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com>
 <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com>
 <532A55AE.5040509@mrabarnett.plus.com>
 <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com>
 <532AE5F3.9040508@mrabarnett.plus.com>
Message-ID: <532B5F03.90409@canterbury.ac.nz>

MRAB wrote:
> It looks like the dict 'kwargs' is being passed straight to 'wrappee'.

But the interpreter doesn't just pass the object
given as a ** argument straight to the function's
** parameter; it gets repacked into a new dict:

 >>> def f(**kw):
...  print id(kw)
...
 >>> d = {'a':1, 'b':2}
 >>> print id(d)
2816160
 >>> f(**d)
2817744

So even if you pass an OrderedDict or some other
custom object as a ** argument, it's a plain dict
by the time it comes out the other end.

-- 
Greg

From steve at pearwood.info  Fri Mar 21 01:15:39 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Mar 2014 11:15:39 +1100
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
Message-ID: <20140321001539.GU16526@ando>

On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote:

> Actually makes me think of decorators, which means that you wouldn't even
> need new tokens or keywords:

But you would require new syntax. Your suggested syntax 

    s = 'a' bop 'b'

is currently a syntax error.

Keep in mind that any changes to syntax would allow bop to be any 
arbitrary object. Python cannot tell at compile-time whether bop is an 
"operator" or a list or a string or a float, only at run-time.



>     from functools import make_operator
>     bop = make_operator(b)

Do you have any suggestion as to how make_operator would work? What 
should it do? What sort of object is bop?



-- 
Steven

From bwmaister at gmail.com  Fri Mar 21 02:28:03 2014
From: bwmaister at gmail.com (Brandon W Maister)
Date: Thu, 20 Mar 2014 21:28:03 -0400
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <20140321001539.GU16526@ando>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
Message-ID: <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>

On Mar 20, 2014 8:16 PM, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote:
>
> > Actually makes me think of decorators, which means that you wouldn't
even
> > need new tokens or keywords:
>
> But you would require new syntax. Your suggested syntax
>
>     s = 'a' bop 'b'
>
> is currently a syntax error.

100% agree, I was careful with my language, if not my thinking :-)


> Keep in mind that any changes to syntax would allow bop to be any
> arbitrary object. Python cannot tell at compile-time whether bop is an
> "operator" or a list or a string or a float, only at run-time.

I admit that when writing I didn't think about the fact that I was
suggesting adding dynamic names to what is currently a static table.

> >     from functools import make_operator
> >     bop = make_operator(b)
>
> Do you have any suggestion as to how make_operator would work? What
> should it do?

Originally I was thinking something along the lines of "just add it to the
'operators' dict" but obviously that won't work since there's no such thing.

Before I defend the idea any further I need to point out that I don't think
it's a *good* idea. I kind of like the idea of arbitrary infix functions,
so I threw out an idea that I haven't seen before.

We could, hypothetically, create an "operators" dict (or dicts) alongside
locals and globals. Then, if an unexpected token shows up in operator
position (which is probably well-defined?) before a SyntaxError is raised
the token is looked up in the operators dict.

I think that the biggest problem I have with that solution is that
philosophically it seems like it would take Python a bit far into the
lisp-2 camp, which I would not like.

> What sort of object is bop?
A regular function that takes one or two args. Honestly the nice thing
about the "look it up somewhere if it causes the right kind of SyntaxError"
solution is that it would probably also allow arbitrary unary operators.
Although why anyone would want *that* is beyond me, so maybe it's not nice
:-)

bwm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140320/48bd111a/attachment.html>

From stephen at xemacs.org  Fri Mar 21 02:39:00 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 21 Mar 2014 10:39:00 +0900
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com>
Message-ID: <87mwgkib4b.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > figured out. People will love or hate the proposal based on the
 > tentative syntax, and the "grit on Tim's monitor" check, and such.

Why the lets-use-some-punctuation-for-syntax crowd doesn't just buy
Tim a lifetime subscription to monitor-protecting clearsheets so he
can remove the grit every day, I just don't know!  Win-win AFAICS....


From rosuav at gmail.com  Fri Mar 21 03:49:12 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Mar 2014 13:49:12 +1100
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
Message-ID: <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>

On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com> wrote:
> We could, hypothetically, create an "operators" dict (or dicts) alongside
> locals and globals. Then, if an unexpected token shows up in operator
> position (which is probably well-defined?) before a SyntaxError is raised
> the token is looked up in the operators dict.

The SyntaxError is raised at compilation time; locals and globals are
at run time. The only way that would work is if you do something like
this:

operators["b"] = lambda left, right: whatever
import module_using_b

ChrisA

From abarnert at yahoo.com  Fri Mar 21 04:40:46 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 20 Mar 2014 20:40:46 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
Message-ID: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>

On Mar 20, 2014, at 19:49, Chris Angelico <rosuav at gmail.com> wrote:

> On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com> wrote:
>> We could, hypothetically, create an "operators" dict (or dicts) alongside
>> locals and globals. Then, if an unexpected token shows up in operator
>> position (which is probably well-defined?) before a SyntaxError is raised
>> the token is looked up in the operators dict.
> 
> The SyntaxError is raised at compilation time; locals and globals are
> at run time. The only way that would work is if you do something like
> this:
> 
> operators["b"] = lambda left, right: whatever
> import module_using_b

Or, alternatively, instead of a SyntaxError this would compile into a call to b, which would then give a NameError at runtime. And I suppose you could argue that's just taking dynamic typing one step further. But I seriously doubt anyone actually wants typos like "esle" to compile successfully...

Also, imagine what it would look like to chain these:

    spam spam eggs baked beans spam spam

Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have? Obviously that example is over the top, but try it with a tree constructor or the other examples from this thread and it's still painful.

The Haskell-style backticks solve all of these problems: the coder can't accidentally write an infix operator when he meant something else, the parser can distinguish infix operators from typos at compile time, and the reader can tell (unless he's stolen Tim's monitor grit) which things are operators and which are operands. (Of course you could deliberately confuse people with weird spacing, but Python doesn't have to make it impossible to write intentionally obfuscated code, it just has to avoid making it easy to write accidentally obfuscated code.)

From ethan at stoneleaf.us  Fri Mar 21 06:01:11 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 20 Mar 2014 22:01:11 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
Message-ID: <532BC797.1060106@stoneleaf.us>

On 03/20/2014 08:40 PM, Andrew Barnert wrote:
>
> Also, imagine what it would look like to chain these:
>
>      spam spam eggs baked beans spam spam
>
> Maybe the parser can figure out that the spam, baked, and spam are
>  operators and the spam, eggs, beans, and spam are operands, but what
>  hope does any human have?

Why would spam be able to be both a variable and an operator?  Hmm, perhaps because of the first-class citizen thing -- 
so the leading grit says "make it work" and without it is just passing it around.

--
~Ethan~

From rosuav at gmail.com  Fri Mar 21 06:43:48 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Mar 2014 16:43:48 +1100
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <532BC797.1060106@stoneleaf.us>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <532BC797.1060106@stoneleaf.us>
Message-ID: <CAPTjJmpGZqZkmdsNX-AE7+O96rZ1wp6hCHbD=gisKJNFivWZFg@mail.gmail.com>

On Fri, Mar 21, 2014 at 4:01 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 03/20/2014 08:40 PM, Andrew Barnert wrote:
>>
>>
>> Also, imagine what it would look like to chain these:
>>
>>      spam spam eggs baked beans spam spam
>>
>> Maybe the parser can figure out that the spam, baked, and spam are
>>  operators and the spam, eggs, beans, and spam are operands, but what
>>  hope does any human have?
>
>
> Why would spam be able to be both a variable and an operator?  Hmm, perhaps
> because of the first-class citizen thing -- so the leading grit says "make
> it work" and without it is just passing it around.

If the creation of an operator is done by putting something into the
current namespace, then it has to be available as some sort of object.
On the other hand, if it's something done as a declaration, or a
pre-import keyword mapping, then it should be a keyword (like other
word operators, "if" and "else" and such), and unavailable for
assignment.

In theory, this could be done as:

from __aether__ import spam as operator

where "as operator", instead of specifying the name to bind to,
specifies that it be something other than a name. (Preferably, it
should use a keyword, rather than "operator", which is a valid name.)
This would go immediately under future statements, and would affect
the compilation of that module: any instance of the word "spam" is now
a keyword operator, so the above would be invalid. I'm still not
convinced it would be a good thing, but this at least gives a
per-module way to create operators.

ChrisA

From Andy.Henshaw at gtri.gatech.edu  Fri Mar 21 15:19:42 2014
From: Andy.Henshaw at gtri.gatech.edu (Henshaw, Andy)
Date: Fri, 21 Mar 2014 14:19:42 +0000
Subject: [Python-ideas] Infix matrix-multiply,
 but not general infix operators?
In-Reply-To: <532BC797.1060106@stoneleaf.us>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <532BC797.1060106@stoneleaf.us>
Message-ID: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org>

From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman

>On 03/20/2014 08:40 PM, Andrew Barnert wrote:
>>
>> Also, imagine what it would look like to chain these:
>>
>>      spam spam eggs baked beans spam spam
>>
>> Maybe the parser can figure out that the spam, baked, and spam are  
>> operators and the spam, eggs, beans, and spam are operands, but what  
>> hope does any human have?
>
> Why would spam be able to be both a variable and an operator?  
> Hmm, perhaps because of the first-class citizen thing -- so the
> leading grit says "make it work" and without it is just passing it around.

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo


From cfkaran2 at gmail.com  Fri Mar 21 17:24:28 2014
From: cfkaran2 at gmail.com (Cem Karan)
Date: Fri, 21 Mar 2014 12:24:28 -0400
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <532BC797.1060106@stoneleaf.us>
 <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org>
Message-ID: <33AF08DB-28BF-4290-89D4-5FFA485AF8C2@gmail.com>


On Mar 21, 2014, at 10:19 AM, "Henshaw, Andy" <Andy.Henshaw at gtri.gatech.edu> wrote:

> From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman
> 
>> On 03/20/2014 08:40 PM, Andrew Barnert wrote:
>>> 
>>> Also, imagine what it would look like to chain these:
>>> 
>>>     spam spam eggs baked beans spam spam
>>> 
>>> Maybe the parser can figure out that the spam, baked, and spam are  
>>> operators and the spam, eggs, beans, and spam are operands, but what  
>>> hope does any human have?
>> 
>> Why would spam be able to be both a variable and an operator?  
>> Hmm, perhaps because of the first-class citizen thing -- so the
>> leading grit says "make it work" and without it is just passing it around.
> 
> Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

For those whose first language is not USA English:

http://en.wikipedia.org/wiki/Buffalo_Buffalo_Buffalo_Buffalo_Buffalo_Buffalo

Thanks,
Cem Karan

From ethan at stoneleaf.us  Fri Mar 21 17:47:21 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 21 Mar 2014 09:47:21 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <532BC797.1060106@stoneleaf.us>
 <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org>
Message-ID: <532C6D19.5050803@stoneleaf.us>

On 03/21/2014 07:19 AM, Henshaw, Andy wrote:
> From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman
>
>> On 03/20/2014 08:40 PM, Andrew Barnert wrote:
>>>
>>> Also, imagine what it would look like to chain these:
>>>
>>>       spam spam eggs baked beans spam spam
>>>
>>> Maybe the parser can figure out that the spam, baked, and spam are
>>> operators and the spam, eggs, beans, and spam are operands, but what
>>> hope does any human have?
>>
>> Why would spam be able to be both a variable and an operator?
>> Hmm, perhaps because of the first-class citizen thing -- so the
>> leading grit says "make it work" and without it is just passing it around.
>
> Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

Python, however, is not English, and would only recognize two different objects from the above, so that would be a 
SyntaxError.

--
~Ethan~

From daoust.mj at gmail.com  Fri Mar 21 21:06:54 2014
From: daoust.mj at gmail.com (Mark Daoust)
Date: Fri, 21 Mar 2014 16:06:54 -0400
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
Message-ID: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com>

>>> spam `spam` eggs `baked` beans `spam` spam

I'm not sure if the backticks help.

When you read that, I think you're reading the spacing not the ticks.

You can already do something very similar by using built-in operators to
"quote" the custom "operators", if you set the appropriate operator
overloads.

 >>> spam *spam* eggs <baked> beans /spam/ spam

"<spam>" is potentially the most readable, for it's parenthesis like shape
, but you can't overload the "and" it expands into.




Mark Daoust



On Thu, Mar 20, 2014 at 11:40 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Mar 20, 2014, at 19:49, Chris Angelico <rosuav at gmail.com> wrote:
>
> > On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com>
> wrote:
> >> We could, hypothetically, create an "operators" dict (or dicts)
> alongside
> >> locals and globals. Then, if an unexpected token shows up in operator
> >> position (which is probably well-defined?) before a SyntaxError is
> raised
> >> the token is looked up in the operators dict.
> >
> > The SyntaxError is raised at compilation time; locals and globals are
> > at run time. The only way that would work is if you do something like
> > this:
> >
> > operators["b"] = lambda left, right: whatever
> > import module_using_b
>
> Or, alternatively, instead of a SyntaxError this would compile into a call
> to b, which would then give a NameError at runtime. And I suppose you could
> argue that's just taking dynamic typing one step further. But I seriously
> doubt anyone actually wants typos like "esle" to compile successfully...
>
> Also, imagine what it would look like to chain these:
>
>     spam spam eggs baked beans spam spam
>
> Maybe the parser can figure out that the spam, baked, and spam are
> operators and the spam, eggs, beans, and spam are operands, but what hope
> does any human have? Obviously that example is over the top, but try it
> with a tree constructor or the other examples from this thread and it's
> still painful.
>
> The Haskell-style backticks solve all of these problems: the coder can't
> accidentally write an infix operator when he meant something else, the
> parser can distinguish infix operators from typos at compile time, and the
> reader can tell (unless he's stolen Tim's monitor grit) which things are
> operators and which are operands. (Of course you could deliberately confuse
> people with weird spacing, but Python doesn't have to make it impossible to
> write intentionally obfuscated code, it just has to avoid making it easy to
> write accidentally obfuscated code.)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140321/c7353cbc/attachment.html>

From skip at pobox.com  Fri Mar 21 21:14:21 2014
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 21 Mar 2014 15:14:21 -0500
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com>
 <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com>
Message-ID: <CANc-5UzOXGG5SZGb=hxYXHADbmREvm0kZMe0We=zuocvLB2rHA@mail.gmail.com>

On Fri, Mar 21, 2014 at 3:06 PM, Mark Daoust <daoust.mj at gmail.com> wrote:
>>>> spam `spam` eggs `baked` beans `spam` spam
>
> I'm not sure if the backticks help.

Backticks never help. They hide amongst the beefier characters. That's
one reason why $(...) is preferred by most Bash users and repr() by
most Python users. Let them die the quiet death they deserve.

Skip

From abarnert at yahoo.com  Fri Mar 21 23:09:06 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 21 Mar 2014 15:09:06 -0700
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
In-Reply-To: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com>
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com>
Message-ID: <6AC16A81-BCF8-4690-954C-4A14DA704993@yahoo.com>

On Mar 21, 2014, at 13:06, Mark Daoust <daoust.mj at gmail.com> wrote:

> >>> spam `spam` eggs `baked` beans `spam` spam
> 
> I'm not sure if the backticks help.
> 
> When you read that, I think you're reading the spacing not the ticks. 

No, you're reading a combination of the two. Compare it to the same without backticks:

    >>> spam `spam` eggs `baked` brand `spam` spam
    >>> spam spam eggs baked beans spam spam
    >>> spam  spam  eggs  baked  beans  spam  spam

There's no way to differentiate the operators from the operands with _just_ spacing, but with spacing plus anything else, there is.

And lest you think this is just an artifact of passing spam to spam, here it is with more realistic names;

    >>> mata mmul matb mmul matb mmul vec
    >>> mata `mmul` matb `mmul` matb `mmul` vec

No matter how much you like or dislike the latter, the former is clearly a lot worse.

> You can already do something very similar by using built-in operators to "quote" the custom "operators", if you set the appropriate operator overloads.

Yes, I mentioned the |op| hack the last time I brought this up, and two other people suggested it after that. But if you want to go over it again: no symbol has a precedence right below the bitwise ops, comparison operators don't work because of chaining, there are types that will handle __op__ so your fake operator doesn't get a chance to do its __rop__, there's no way to prevent "2 |op" from being a valid but confusing expression (and there's a temptation to use it as a half-assed form of operator sectioning), it's hard for syntax highlighters and linters and the like to understand |op| is an operator, there's no way to restrict arbitrary expressions as operators, you have to declare all infixable operators with a decorator (as opposed to having the language choose whether all functions are infixable, or we use a decorator, or something else), any symbol you use already has another meaning that will confuse people, it's a bit less efficient, and, finally, it's not a consistent language feature (having |op| be an operator in one module, *op* in another, etc. makes the language harder to learn).

From vernondcole at gmail.com  Sun Mar 23 10:31:03 2014
From: vernondcole at gmail.com (Vernon D. Cole)
Date: Sun, 23 Mar 2014 03:31:03 -0600
Subject: [Python-ideas] alternative-implementation friendly changes to
	Python Launcher for Windows
Message-ID: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com>

A new issue was recently raised on the IronPython group addressing the idea
of including a copy of the *Python Launcher for Windows* in the IronPython
binary distribution.  http://www.python.org/dev/peps/pep-0397/

I would like to get feedback from this group, also, before opening a work
item for the suggestion.  Please add any comments or suggestions you may
have.

Here is a partial clip from our discussion...

vernondcole <http://www.codeplex.com/site/users/view/vernondcole> wrote Fri
> at 2:32 AM
> I made a clone of the git version of PLW to look this idea over. It is a
> smallish C++ program and should not be too hard to patch to make
> alternate-implementation friendly. The problem today is that it is CPython
> specific and searches the Windows registry to determine which versions of
> Python are present. Alternate implementations are supported only using
> entries in the py.ini file and setting the #! line in your source code
> approprietly. (Which works great. I even have mine launching perl scripts,
> just to proove that it can.)
>
> I propose adding two features to PLW.
>
> 1) extend the first command-line switch to also pick items from the py.ini
> file. I would like to type:
> py -ipy myprogram.py
> and have the command associated with "ipy" in my py.ini file executed.
>
> 2) PLW should have (in py.ini) a [default] section identifying which
> command in the [commands] section should be run in the absence of a
> command-line switch.
>
> Alternate implementations should then install (or modify) py.ini to
> support their interpreter.
>
> The extended PLW version ought to be backported to the CPython
> distribution, so that any implementation of Python will work. I don't think
> that the CPython group would object.
>
> Note 1: I have not compared to make sure the stand-alone PLW is identical
> with the Python 3.3 version.
> Note 2: We need to make sure that PLW will work correctly even if no
> CPython versions are present.
> Note 3: I fear that I have just volunteered to write the patch. Is someone
> else out there willing? It's been a very long time since I wrote commercial
> quality C code.
>
> jdhardy <http://www.codeplex.com/site/users/view/jdhardy> wrote Fri at
> 5:33 AM
>  Sounds good. I remember from the original PLW discussion they want it to
> work for other implementations but weren't sure what would be needed. It
> sounds like you're making that list. :)
>
> It's probably worth opening bugs on the Python tracker describing what
> needs to change, since it's not IronPython specific. If you don't feel
> comfortable writing the code perhaps someone over there can, or at least
> review the code for you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/d8411396/attachment.html>

From sturla.molden at gmail.com  Sun Mar 23 11:40:10 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sun, 23 Mar 2014 11:40:10 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140315010944.6b8244e1@fsol>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol>
Message-ID: <lgmdm0$2ng$1@ger.gmane.org>

On 15/03/14 01:09, Antoine Pitrou wrote:

> Really? That should be up to the third-party library implementing the @
> operator for its types, not to the language itself: Python _suggests_
> an use case for @, it doesn't mandate it (especially as there's no
> appropriate data type in the stdlib).

array.array is an appropriate type for supporting @ for matrix 
multiplication.


Sturla




From jeanpierreda at gmail.com  Sun Mar 23 11:51:24 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 23 Mar 2014 03:51:24 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <lgmdm0$2ng$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
Message-ID: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>

On Sun, Mar 23, 2014 at 3:40 AM, Sturla Molden <sturla.molden at gmail.com> wrote:
> On 15/03/14 01:09, Antoine Pitrou wrote:
>
>> Really? That should be up to the third-party library implementing the @
>> operator for its types, not to the language itself: Python _suggests_
>> an use case for @, it doesn't mandate it (especially as there's no
>> appropriate data type in the stdlib).
>
>
> array.array is an appropriate type for supporting @ for matrix
> multiplication.

I was thinking the function type, as a composition operator. Matrix
multiplication corresponds exactly with composition of linear
functions, so it makes some sort of sense. And it has some kind of
neat similarity to decorators, which generalize function composition.

If you rename "matmul" to "compose" and "matpow" to something like
"iterate" (except less confusing), then you've generalized the
operators a little and given them more potential use cases.

-- Devin

From p.f.moore at gmail.com  Sun Mar 23 15:36:30 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 23 Mar 2014 14:36:30 +0000
Subject: [Python-ideas] alternative-implementation friendly changes to
 Python Launcher for Windows
In-Reply-To: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com>
References: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com>
Message-ID: <CACac1F-WRYu+xcOLt5oVGXCS+G5uiQVKP0+PFHFz_9b4vuCZxQ@mail.gmail.com>

On 23 March 2014 09:31, Vernon D. Cole <vernondcole at gmail.com> wrote:
> A new issue was recently raised on the IronPython group addressing the idea
> of including a copy of the Python Launcher for Windows in the IronPython
> binary distribution.  http://www.python.org/dev/peps/pep-0397/
[...]
>> I propose adding two features to PLW.
>>
>> 1) extend the first command-line switch to also pick items from the py.ini
>> file. I would like to type:
>> py -ipy myprogram.py
>> and have the command associated with "ipy" in my py.ini file executed.
>>
>> 2) PLW should have (in py.ini) a [default] section identifying which
>> command in the [commands] section should be run in the absence of a
>> command-line switch.
>>
>> Alternate implementations should then install (or modify) py.ini to
>> support their interpreter.

I like the idea of extending the version switch.
I don't have a problem with a default command (although I probably
wouldn't use it).

I don't particularly like the idea of implementations modifying py.ini
- that's a user configuration file and should not be modified by the
system. Just document what needs to be added by the user for a
particular implementation.

The standalone version of the launcher is maintained by Vinay Sajip at
https://bitbucket.org/pypa/pylauncher. I'd suggest adding issues to
the tracker there. I don't know how (or if) that version gets folded
back into the one distributed with Python, that's something that maybe
Vinay can comment on.

Paul

From mistersheik at gmail.com  Sun Mar 23 23:20:41 2014
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 23 Mar 2014 15:20:41 -0700 (PDT)
Subject: [Python-ideas] Make __reduce__ to correspond to __getnewargs_ex__
Message-ID: <fe676ada-d7bd-451c-b518-71d4cfb6411f@googlegroups.com>

Currently __reduce__<http://docs.python.org/3.4/library/pickle.html#object.__reduce__>returns up to five things:

(1) self.__new__ (or a substitute)
(2) the result of __getnewargs__<http://docs.python.org/3.4/library/pickle.html#object.__getnewargs__>, 
which returns a tuple of positional arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__>
,
(3) the result of __getstate__<http://docs.python.org/3.4/library/pickle.html#object.__getstate__>, 
which returns an object to be passed to __setstate__<http://docs.python.org/3.4/library/pickle.html#object.__setstate__>
(4) an iterator of values for appending to a sequence
(5) an iterator of key-value pairs for setting on a string.

Python 3.4 added the very useful (for me) __getnewargs_ex__<http://docs.python.org/3.4/library/pickle.html#object.__getnewargs_ex__>, 
which returns a pair:
(1) a tuple of positional arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__>
(2) a dict of keyword arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__>

Therefore, I am proposing that __reduce__ return somehow these keyword 
arguments for __new__.

Best,
Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/5fac6a92/attachment.html>

From abarnert at yahoo.com  Sun Mar 23 23:48:04 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 23 Mar 2014 15:48:04 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
Message-ID: <6E79C662-4C99-40BD-90AC-AFF39EFC8F04@yahoo.com>

On Mar 23, 2014, at 3:51, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:

> On Sun, Mar 23, 2014 at 3:40 AM, Sturla Molden <sturla.molden at gmail.com> wrote:
>> On 15/03/14 01:09, Antoine Pitrou wrote:
>> 
>>> Really? That should be up to the third-party library implementing the @
>>> operator for its types, not to the language itself: Python _suggests_
>>> an use case for @, it doesn't mandate it (especially as there's no
>>> appropriate data type in the stdlib).
>> 
>> 
>> array.array is an appropriate type for supporting @ for matrix
>> multiplication.
> 
> I was thinking the function type, as a composition operator. Matrix
> multiplication corresponds exactly with composition of linear
> functions, so it makes some sort of sense. And it has some kind of
> neat similarity to decorators, which generalize function composition.
> 
> If you rename "matmul" to "compose" and "matpow" to something like
> "iterate" (except less confusing), then you've generalized the
> operators a little and given them more potential use cases.

This sounds nice from a purity point of view, but I'm not sure it's a good idea.

First, matrix multiplication only corresponds exactly with function composition if matrix*vector multiplication corresponds exactly with function calling, which is generally not true in programming. You can't assign a matrix to f, then write f(v) to apply the matrix as an operation; you write f @ v.

Meanwhile, Python doesn't even have compose in functools, much less in builtins, and I don't think anyone had seriously proposed it in decades. If we don't even need it in the stdlib, do we really need is as an operator?

Also, is it right-compose or left-compose? (That is, does (f at g)(2) call
f then g, or g then f?) One answer is obvious to people who think mathematically, the other to novices. And even if you argue that the right answer to that is to teach the novices the right way to think about it clearly, reverse composition is still often useful; that's why many languages that have a compose function or operator also have rcompose and/or make it trivial to write as flip(compose).

Also, how exactly do you define compose? This is easy in a language like Haskell, where every function takes one argument and returns one value (which, because of currying, may actually be a function that takes the next argument...), but in Python, where functions can take any number of arguments, with default values and *args and keyword-only arguments and so on, it's a lot less clear. Does the second function need to be a single-argument function, or do tuple returns match to *args, or something different? Different third-party function libraries use different answers to that, sometimes offering more than one under different names, but obviously the operator has to be either compose1 or composestar, and then the other one won't even be in the stdlib?

Meanwhile, looking at languages that care about mathematical purity and about functional composition: you don't write . for matrix multiplication in Haskell, you write `multStd` or, more likely, define *, **, `mult`, or something else more readable. 



From steve at pearwood.info  Mon Mar 24 01:49:43 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 24 Mar 2014 11:49:43 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <lgmdm0$2ng$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
Message-ID: <20140324004943.GX16526@ando>

On Sun, Mar 23, 2014 at 11:40:10AM +0100, Sturla Molden wrote:
> On 15/03/14 01:09, Antoine Pitrou wrote:
> 
> >Really? That should be up to the third-party library implementing the @
> >operator for its types, not to the language itself: Python _suggests_
> >an use case for @, it doesn't mandate it (especially as there's no
> >appropriate data type in the stdlib).
> 
> array.array is an appropriate type for supporting @ for matrix 
> multiplication.

No it isn't. The PEP even discusses it:


    [quote]
    array objects cannot represent multidimensional data at all, 
    which makes ``__matmul__`` much less useful.  Second, 
    providing a quality implementation of matrix multiplication 
    is highly non-trivial.  Naive nested loop implementations 
    are very slow and providing one in CPython would just create
    a trap for users.  But the alternative -- providing a modern,
    competitive matrix multiply -- would require that CPython link
    to a BLAS library, which brings a set of new complications.  
    In particular, several popular BLAS libraries (including the
    one that ships by default on OS X) currently break the use of 
    ``multiprocessing`` [#blas-fork]_.  And finally, we'd have to
    add quite a bit beyond ``__matmul__`` before ``memoryview``
    or  ``array.array`` would be useful for numeric work -- like
    elementwise versions of the other arithmetic operators, just
    to start.
    [end quote]


I don't think we should be looking for additional use-cases for @. 
Either the PEP stands on its own, and @ is approved for matrix 
multiplication (and possibly @@ for exponentiation), or it isn't. If the 
PEP is accepted -- and I think it should be -- then people will invent 
new uses for @. Or they won't. Either way, it doesn't matter.


-- 
Steven

From steve at pearwood.info  Mon Mar 24 02:08:24 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 24 Mar 2014 12:08:24 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
Message-ID: <20140324010824.GY16526@ando>

On Sun, Mar 23, 2014 at 03:51:24AM -0700, Devin Jeanpierre wrote:

> If you rename "matmul" to "compose" and "matpow" to something like
> "iterate" (except less confusing), then you've generalized the
> operators a little and given them more potential use cases.

-1

The dunder methods are typically named for the most common or 
typical use-case, not generalised or marginal ones e.g.:

+  __add__ not __concat__
*  __mul__  not __repeat__ or __repetition__
<  __lt__ not __subset__
&  __and__ not __ampersand__ or __conjunction__




-- 
Steven

From jeanpierreda at gmail.com  Mon Mar 24 02:47:40 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 23 Mar 2014 18:47:40 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140324010824.GY16526@ando>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
 <20140324010824.GY16526@ando>
Message-ID: <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>

On Sun, Mar 23, 2014 at 6:08 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> -1
>
> The dunder methods are typically named for the most common or
> typical use-case, not generalised or marginal ones e.g.:
>
> +  __add__ not __concat__
> *  __mul__  not __repeat__ or __repetition__
> <  __lt__ not __subset__
> &  __and__ not __ampersand__ or __conjunction__

Thanks for taking issue with the not-silly point.

Ironically enough, all of those but "and" are actually general names
for operators, but I agree that probably wasn't intentional. Anyway,
in practice people seem to get what it means for something to be
"and"ed with another thing. There's an intuition that many things
using the operator treat it in a roughly analogous way, and the same
is true for many other overloadable operators in Python. These
conventions help make code more understandable and are great.

I'm not sure such a thing would happen with an operator for which the
stdlib and documentation and everything insists is only for matrix
multiplication. It'd be like if we had
"__twos_complement_bitwise_and__". It's one thing to generalize an
operator name, and another to not make it very specific.

-- Devin

From cjwilliams43 at gmail.com  Mon Mar 24 03:55:16 2014
From: cjwilliams43 at gmail.com (Colin J. Williams)
Date: Sun, 23 Mar 2014 22:55:16 -0400
Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199
In-Reply-To: <mailman.43259.1395623316.18129.python-ideas@python.org>
References: <mailman.43259.1395623316.18129.python-ideas@python.org>
Message-ID: <532F9E94.4090509@gmail.com>

#4 seems to refer to the Python basic system.

However numpy.array does deal with multi-dimension arrays.
On 23-Mar-2014 9:08 PM, python-ideas-request at python.org wrote:
> ------------------------------ Message: 4 Date: Mon, 24 Mar 2014 
> 11:49:43 +1100 From: Steven D'Aprano <steve at pearwood.info> To: 
> python-ideas at python.org Subject: Re: [Python-ideas] [RFC] draft PEP: 
> Dedicated infix operators for matrix multiplication and matrix power 
> Message-ID: <20140324004943.GX16526 at ando> Content-Type: text/plain; 
> charset=us-ascii On Sun, Mar 23, 2014 at 11:40:10AM +0100, Sturla 
> Molden wrote:
>> On 15/03/14 01:09, Antoine Pitrou wrote:
>>
>>> Really? That should be up to the third-party library implementing the @
>>> operator for its types, not to the language itself: Python _suggests_
>>> an use case for @, it doesn't mandate it (especially as there's no
>>> appropriate data type in the stdlib).
>> array.array is an appropriate type for supporting @ for matrix
>> multiplication.
> No it isn't. The PEP even discusses it:
>
>
>      [quote]
>      array objects cannot represent multidimensional data at all,
>      which makes ``__matmul__`` much less useful.  Second,
>      providing a quality implementation of matrix multiplication
>      is highly non-trivial.  Naive nested loop implementations
>      are very slow and providing one in CPython would just create
>      a trap for users.  But the alternative -- providing a modern,
>      competitive matrix multiply -- would require that CPython link
>      to a BLAS library, which brings a set of new complications.
>      In particular, several popular BLAS libraries (including the
>      one that ships by default on OS X) currently break the use of
>      ``multiprocessing`` [#blas-fork]_.  And finally, we'd have to
>      add quite a bit beyond ``__matmul__`` before ``memoryview``
>      or  ``array.array`` would be useful for numeric work -- like
>      elementwise versions of the other arithmetic operators, just
>      to start.
>      [end quote]
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/61eade49/attachment.html>

From ben+python at benfinney.id.au  Mon Mar 24 04:00:41 2014
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 24 Mar 2014 14:00:41 +1100
Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199
References: <mailman.43259.1395623316.18129.python-ideas@python.org>
 <532F9E94.4090509@gmail.com>
Message-ID: <85vbv4qp0m.fsf@benfinney.id.au>

"Colin J. Williams"
<cjwilliams43 at gmail.com> writes:

> #4 seems to refer to the Python basic system.

Howdy Colin. To participate in a discussion, please respond to
individual messages so the threads of discussion make sense.

To do that, first change your subscription to disable ?digest? mode
<URL:http://www.list.org/mailman-member/node27.html> so that you receive
the individual messages in the discussion.

-- 
 \     ?If I had known what it would be like to have it all... I might |
  `\     have been willing to settle for less.? ?Jane Wagner, via Lily |
_o__)                                                           Tomlin |
Ben Finney


From steve at pearwood.info  Mon Mar 24 04:12:42 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 24 Mar 2014 14:12:42 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>
References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
 <20140324010824.GY16526@ando>
 <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>
Message-ID: <20140324031242.GZ16526@ando>

On Sun, Mar 23, 2014 at 06:47:40PM -0700, Devin Jeanpierre wrote:

> I'm not sure such a thing would happen with an operator for which the
> stdlib and documentation and everything insists is only for matrix
> multiplication.

They would *insist* would they? The documentation would come with a 
great warning "Thou Shalt Not Use __matmul__ For Anything Except Matrix 
Multiplication"? I suppose that will be like the similar (imaginary) 
warning that __pow__ is only permitted to be used for numerical 
exponentiation and that any other use is forbidden on pain of being 
slapped with a fish.

I think it is highly unlikely that people will be frightened off from 
overloading @ by the name. If people happily use __lt__ for subset 
checking, which is *nothing* like less-than, I'm sure they'll use @ as 
well. People do far weirder things -- there's even a recipe out there on 
the Internet for implementing arbitrary named pseudo-operators using 
__or__ :

    vector |dot| vector


But suppose you're right, and people don't use it for anything but 
matrix multiplication. So what? Who cares? It's not like anyone is made 
worse off by the change. None of my code currently uses the @ operator. 
If numpy starts to use it, and I don't, I'll be no worse off than I am 
now.


> It'd be like if we had "__twos_complement_bitwise_and__". 

Having __matmul__ would be like if we had  __xor__ __pow__ 
__sub__ __div__ __mod__ __pos__ or __neg__ 



-- 
Steven


From jeanpierreda at gmail.com  Mon Mar 24 04:48:49 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 23 Mar 2014 20:48:49 -0700
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140324031242.GZ16526@ando>
References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
 <20140324010824.GY16526@ando>
 <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>
 <20140324031242.GZ16526@ando>
Message-ID: <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com>

On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I think it is highly unlikely that people will be frightened off from
> overloading @ by the name. If people happily use __lt__ for subset
> checking, which is *nothing* like less-than,

Actually, no. <, or "less than", is the exact way it's spelled for any
partial order, and subset relations are probably the most famous
non-numeric example of a partial order.

In fact, if you look at the most common construction of the natural
numbers (0 = frozenset(), succ(x) = frozenset(x) | x), it is more
complicated than it needs to be only so that the < operator means the
same thing for natural numbers when treated as numbers or as sets.

>> It'd be like if we had "__twos_complement_bitwise_and__".
>
> Having __matmul__ would be like if we had  __xor__ __pow__
> __sub__ __div__ __mod__ __pos__ or __neg__

Then we disagree.

-- Devin

From tjreedy at udel.edu  Mon Mar 24 05:58:03 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 24 Mar 2014 00:58:03 -0400
Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199
In-Reply-To: <85vbv4qp0m.fsf@benfinney.id.au>
References: <mailman.43259.1395623316.18129.python-ideas@python.org>
 <532F9E94.4090509@gmail.com> <85vbv4qp0m.fsf@benfinney.id.au>
Message-ID: <lgoe0l$fbf$2@ger.gmane.org>

On 3/23/2014 11:00 PM, Ben Finney wrote:
> "Colin J. Williams"
> <cjwilliams43 at gmail.com> writes:
>
>> #4 seems to refer to the Python basic system.
>
> Howdy Colin. To participate in a discussion, please respond to
> individual messages so the threads of discussion make sense.
>
> To do that, first change your subscription to disable ?digest? mode
> <URL:http://www.list.org/mailman-member/node27.html> so that you receive
> the individual messages in the discussion.

Or disable mail messages and access the list as newsgroup 
gmane.comp.python.ideas as news.gmane.org (as I am because I also do not 
want a flood of messages to delete).

-- 
Terry Jan Reedy



From steve at pearwood.info  Mon Mar 24 13:06:31 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 24 Mar 2014 23:06:31 +1100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
	matrix multiplication and matrix power
In-Reply-To: <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com>
References: <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol>
 <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol>
 <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
 <20140324010824.GY16526@ando>
 <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>
 <20140324031242.GZ16526@ando>
 <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com>
Message-ID: <20140324120631.GA16526@ando>

On Sun, Mar 23, 2014 at 08:48:49PM -0700, Devin Jeanpierre wrote:
> On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> > I think it is highly unlikely that people will be frightened off from
> > overloading @ by the name. If people happily use __lt__ for subset
> > checking, which is *nothing* like less-than,
> 
> Actually, no. <, or "less than", is the exact way it's spelled for any
> partial order, and subset relations are probably the most famous
> non-numeric example of a partial order.

I will accept that subsets are an example of partial order and so I was 
wrong to say that that it has nothing to do with __lt__ and __gt__.

But I disagree that < is the usual symbol for subset. Neither Wikipedia 
nor Wolfram Mathworld mention it:

https://en.wikipedia.org/wiki/Subset
http://mathworld.wolfram.com/Subset.html

I was taught to use symbols ? and ? for proper subset and subset-or-
equal. I've also seen ? and ? used instead. But prior to Python, I've 
never seen < used. If it's used in mathematics, it's a niche use.



-- 
Steven

From ncoghlan at gmail.com  Mon Mar 24 14:28:06 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 24 Mar 2014 23:28:06 +1000
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140324120631.GA16526@ando>
References: <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol>
 <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol>
 <lgmdm0$2ng$1@ger.gmane.org>
 <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com>
 <20140324010824.GY16526@ando>
 <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com>
 <20140324031242.GZ16526@ando>
 <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com>
 <20140324120631.GA16526@ando>
Message-ID: <CADiSq7eKPM75=efthpSYn_i9+mB6TWV0tAVix69z-5X0p5V1sQ@mail.gmail.com>

On 24 March 2014 22:06, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Mar 23, 2014 at 08:48:49PM -0700, Devin Jeanpierre wrote:
>> On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> > I think it is highly unlikely that people will be frightened off from
>> > overloading @ by the name. If people happily use __lt__ for subset
>> > checking, which is *nothing* like less-than,
>>
>> Actually, no. <, or "less than", is the exact way it's spelled for any
>> partial order, and subset relations are probably the most famous
>> non-numeric example of a partial order.
>
> I will accept that subsets are an example of partial order and so I was
> wrong to say that that it has nothing to do with __lt__ and __gt__.

I normally let python-ideas subthreads run indefinitely without
commenting, but in this case... *really* not a productive tangent :)

Ellipsis, extended slicing and memory views were added to the core
language and C API definitions for the numeric computing folks without
a compelling stdlib use case (at least at the time). Adding __matmul__
for their benefit really isn't much of a stretch, and it makes the
distinction *they* care about clear (__mul__ = element-wise
multiplication, __matmul__ = matrix multiplication). Previous
proposals along these lines failed because they overgeneralised
without compelling benefits from the extra complexity, while this one
hits the sweet spot of solving the *actual* problem to be solved with
just some minor technical details to work out.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From sturla.molden at gmail.com  Tue Mar 25 07:05:04 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Tue, 25 Mar 2014 06:05:04 +0000 (UTC)
Subject: [Python-ideas] Infix matrix-multiply,
	but not general infix operators?
References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com>
 <20140320102443.GS16526@ando>
 <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org>
 <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com>
 <20140321001539.GU16526@ando>
 <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com>
 <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com>
 <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com>
 <532BC797.1060106@stoneleaf.us>
 <CAPTjJmpGZqZkmdsNX-AE7+O96rZ1wp6hCHbD=gisKJNFivWZFg@mail.gmail.com>
Message-ID: <564858166417420257.372728sturla.molden-gmail.com@news.gmane.org>

Chris Angelico <rosuav at gmail.com> wrote:

> where "as operator", instead of specifying the name to bind to,
> specifies that it be something other than a name. (Preferably, it
> should use a keyword, rather than "operator", which is a valid name.)
> This would go immediately under future statements, and would affect
> the compilation of that module: any instance of the word "spam" is now
> a keyword operator, so the above would be invalid. I'm still not
> convinced it would be a good thing, but this at least gives a
> per-module way to create operators.

Cluttering the syntax with arbitrary operators is a very bad idea, IMHO.

Sturla


From techtonik at gmail.com  Tue Mar 25 07:29:44 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 25 Mar 2014 09:29:44 +0300
Subject: [Python-ideas] PEP feedback loop (with tracker account)
Message-ID: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>

It is not that hard to implement an approval function for
the PEP site (to improve the visibility and the process).

For example, on the page
http://legacy.python.org/dev/peps/pep-0231/
the field "Status: Rejected" can be linkified to bring up a
popup window with short info about quantity of people who
approved the PEP (content details can be extended later).

It is not that hard to add a disqus-style JS form to set
status of your position towards PEP using tracker
account.

Will that be useful for Python community?
-- 
anatoly t.

From rosuav at gmail.com  Tue Mar 25 10:07:07 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 25 Mar 2014 20:07:07 +1100
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
Message-ID: <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com>

On Tue, Mar 25, 2014 at 5:29 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> For example, on the page
> http://legacy.python.org/dev/peps/pep-0231/
> the field "Status: Rejected" can be linkified to bring up a
> popup window with short info about quantity of people who
> approved the PEP (content details can be extended later).

By "approved" do you mean those who wish it hadn't been rejected? I
don't think that's terribly useful, and it implies a democratic
approach to Python's development, which doesn't exist. More useful
might be for the "Status: Rejected" to link to a python-dev post
formally rejecting it, but that's about all. Can you elaborate on both
what this would show, and how it would be useful?

ChrisA

From techtonik at gmail.com  Tue Mar 25 17:13:32 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 25 Mar 2014 19:13:32 +0300
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com>
Message-ID: <CAPkN8xJ6bZgoP9V+nN0-Db1aFRBo9RhU+XS9JstPtF0jt3u7XQ@mail.gmail.com>

On Tue, Mar 25, 2014 at 12:07 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Mar 25, 2014 at 5:29 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> For example, on the page
>> http://legacy.python.org/dev/peps/pep-0231/
>> the field "Status: Rejected" can be linkified to bring up a
>> popup window with short info about quantity of people who
>> approved the PEP (content details can be extended later).
>
> By "approved" do you mean those who wish it hadn't been rejected? I
> don't think that's terribly useful, and it implies a democratic
> approach to Python's development, which doesn't exist. More useful
> might be for the "Status: Rejected" to link to a python-dev post
> formally rejecting it, but that's about all. Can you elaborate on both
> what this would show, and how it would be useful?

By "approved" I mean that people sign the specific PEP version as
"fully read and agree with all the clauses". For example, I don't have
time to read PEPs thoroughly and I wonder how many people have.
There might be a problem that PEPs don't receive enough exposure
and timely feedback. I don't know - it is uncertain.

Also having some kind of recorded "achievement" will help to get some
motivation to do thorough reviews.

Also many people don't imagine how many heads are working on a
single PEP and think that there are a lot of people already. It may be
that knowing the stats they will be more interested to contribute.

The fourth reason is that this basic feedback system can be extended
to include specific facts to be addressed, but that out of scope for now.
-- 
anatoly t.

From stephen at xemacs.org  Tue Mar 25 20:16:46 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 26 Mar 2014 04:16:46 +0900
Subject: [Python-ideas]  PEP feedback loop (with tracker account)
In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
Message-ID: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:

 > It is not that hard to implement an approval function for
 > the PEP site (to improve the visibility and the process).

I gather you mean "thumbs up/down" buttons.

 > Will that be useful for Python community?

No.  As Chris points out, the PEP process is not a voting process.  (I
think that's what he meant.  I would say that it is "democratic" in
the sense that as much as possible we try to come to consensus on a
"best" proposal.)  And people do switch as PEPs evolve, but I doubt
they'd be likely to change their "vote" on the PEP site.

Links to the discussion (or perhaps even blog posts of folks who
strongly dissent) are far more relevant.


From rosuav at gmail.com  Tue Mar 25 20:58:59 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 26 Mar 2014 06:58:59 +1100
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmryKhpBj9yeh9z1Dc87H6ShttZYteVibg86j7nF1RnSCw@mail.gmail.com>

On Wed, Mar 26, 2014 at 6:16 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> No.  As Chris points out, the PEP process is not a voting process.  (I
> think that's what he meant.  I would say that it is "democratic" in
> the sense that as much as possible we try to come to consensus on a
> "best" proposal.)  And people do switch as PEPs evolve, but I doubt
> they'd be likely to change their "vote" on the PEP site.

Yes, that's what I meant. Python is built, as most open source
projects are, on a model of "a king with his ministers and citizens".
The king, of course, is normally Guido [1]; core devs are Guido's
ministers and close advisers; everyone else who uses Python is a
citizen. (Drawing the line between ministers and citizens is a tad
harder than in a parliamentary structure, so I'm going to horrendously
cheat and say that ministers can be appointed and/or can step down
very easily and quickly.) A PEP is basically a petition, raised and
discussed. It has a champion (at least by the time it becomes a PEP),
and the Champion will seek to demonstrate how the proposal will be of
great benefit. This doesn't strictly equate to "number of signatures",
which is how these sorts of petitions are often treated in modern
politics; it's more about strength of argument. So not only is this
system not democratic in the obvious sense of "majority rules" (if you
get more + votes than -, the PEP is approved), it's also almost never
even about gathering numerical support (which is what this *would* be
able to show). There have been a few times when popular support is
canvassed (PEP 308, for instance), but even then, it's certainly not
"most popular option wins" (it wasn't then, although the chosen syntax
was clearly in the popular group rather than the less-popular group).

Recording who has fully read the PEP and agrees with every little part
of it would be nearly impossible as the PEP evolves, so by the latest
definition, it's impossible to calculate, as well as being
almost-never-useful. Searching the python-ideas archive for the PEP
number and reading through the discussion is probably the only way to
find out after the event, so I sympathize with the desire to simply
eyeball a figure and see; but I can't imagine it being truly useful in
all that many cases.

ChrisA

[1] Some subprojects (like IDLE?) function identically with a different king.

From barry at python.org  Tue Mar 25 21:27:12 2014
From: barry at python.org (Barry Warsaw)
Date: Tue, 25 Mar 2014 16:27:12 -0400
Subject: [Python-ideas] PEP feedback loop (with tracker account)
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmryKhpBj9yeh9z1Dc87H6ShttZYteVibg86j7nF1RnSCw@mail.gmail.com>
Message-ID: <20140325162712.3f6ba000@anarchist.wooz.org>

From PEP 1:

What is a PEP?

PEP stands for Python Enhancement Proposal. A PEP is a design document
providing information to the Python community, or describing a new feature for
Python or its processes or environment. The PEP should provide a concise
technical specification of the feature and a rationale for the feature.

We intend PEPs to be the primary mechanisms for proposing major new features,
for collecting community input on an issue, and for documenting the design
decisions that have gone into Python. The PEP author is responsible for
building consensus within the community and documenting dissenting opinions.

Because the PEPs are maintained as text files in a versioned repository, their
revision history is the historical record of the feature proposal [1].

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140325/cd332abe/attachment.sig>

From g.brandl at gmx.net  Tue Mar 25 23:46:52 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 25 Mar 2014 23:46:52 +0100
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
Message-ID: <lgt10d$h6l$1@ger.gmane.org>

Am 25.03.2014 07:29, schrieb anatoly techtonik:
> It is not that hard to implement an approval function for
> the PEP site (to improve the visibility and the process).
> 
> For example, on the page
> http://legacy.python.org/dev/peps/pep-0231/
> the field "Status: Rejected" can be linkified to bring up a
> popup window with short info about quantity of people who
> approved the PEP (content details can be extended later).
> 
> It is not that hard to add a disqus-style JS form to set
> status of your position towards PEP using tracker
> account.
> 
> Will that be useful for Python community?

No.  What is useful is to keep the "Post-History" header up to
date, which refers people to the python-dev threads where all the
discussion and "voting" takes place.

I could also imagine adding direct links to the archive; however
many PEPs already add references in the text where appropriate.

Georg


From ethan at stoneleaf.us  Wed Mar 26 03:21:07 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 25 Mar 2014 19:21:07 -0700
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <lgt10d$h6l$1@ger.gmane.org>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <lgt10d$h6l$1@ger.gmane.org>
Message-ID: <53323993.7030505@stoneleaf.us>

On 03/25/2014 03:46 PM, Georg Brandl wrote:
>
> No.  What is useful is to keep the "Post-History" header up to
> date, which refers people to the python-dev threads where all the
> discussion and "voting" takes place.

So should that header just include the dates when new threads were spawned, or if significant time had passed since the 
last posting in the "current" thread?

--
~Ethan~

From abarnert at yahoo.com  Wed Mar 26 06:22:23 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 25 Mar 2014 22:22:23 -0700
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <lgt10d$h6l$1@ger.gmane.org>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <lgt10d$h6l$1@ger.gmane.org>
Message-ID: <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com>

On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote:

> Am 25.03.2014 07:29, schrieb anatoly techtonik:
>> It is not that hard to implement an approval function for
>> the PEP site (to improve the visibility and the process).
>> 
>> For example, on the page
>> http://legacy.python.org/dev/peps/pep-0231/
>> the field "Status: Rejected" can be linkified to bring up a
>> popup window with short info about quantity of people who
>> approved the PEP (content details can be extended later).
>> 
>> It is not that hard to add a disqus-style JS form to set
>> status of your position towards PEP using tracker
>> account.
>> 
>> Will that be useful for Python community?
> 
> No.  What is useful is to keep the "Post-History" header up to
> date, which refers people to the python-dev threads where all the
> discussion and "voting" takes place.
> 
> I could also imagine adding direct links to the archive; however
> many PEPs already add references in the text where appropriate.

Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful.

I can't think of anything else that would really be interesting at all. A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs.

From g.brandl at gmx.net  Wed Mar 26 07:17:17 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 26 Mar 2014 07:17:17 +0100
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <53323993.7030505@stoneleaf.us>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <lgt10d$h6l$1@ger.gmane.org> <53323993.7030505@stoneleaf.us>
Message-ID: <lgtrcu$t6i$1@ger.gmane.org>

Am 26.03.2014 03:21, schrieb Ethan Furman:
> On 03/25/2014 03:46 PM, Georg Brandl wrote:
>>
>> No.  What is useful is to keep the "Post-History" header up to
>> date, which refers people to the python-dev threads where all the
>> discussion and "voting" takes place.
> 
> So should that header just include the dates when new threads were spawned, or if significant time had passed since the 
> last posting in the "current" thread?

PEP 1 says:

"""
The Created header records the date that the PEP was assigned a
number, while Post-History is used to record the dates of when new
versions of the PEP are posted to python-list and/or python-dev.  Both
headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001.
"""

And new threads are usually accompanied by a posting of a revision of the PEP.

Georg


From abarnert at yahoo.com  Wed Mar 26 09:34:01 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 26 Mar 2014 01:34:01 -0700
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <lgt10d$h6l$1@ger.gmane.org> <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com>
 <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com>
Message-ID: <33AA2C5B-A9AF-4B05-A967-0FD41CD6C6B5@yahoo.com>

On Mar 26, 2014, at 1:07, anatoly techtonik <techtonik at gmail.com> wrote:

> On Wed, Mar 26, 2014 at 8:22 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote:
>>> 
>>> No.  What is useful is to keep the "Post-History" header up to
>>> date, which refers people to the python-dev threads where all the
>>> discussion and "voting" takes place.
>>> 
>>> I could also imagine adding direct links to the archive; however
>>> many PEPs already add references in the text where appropriate.
>> 
>> Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful.
> 
> The goal to link threads and PEP versions is reachable at the next
> iteration once status per person is implemented. The benefit from
> the system is that you don't need to dedicate a person to do this.
> 
>> A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs.
> 
> It is not a post. It is status per revision. The difference between
> post and status is that status gives immediate overview (diffstat)
> while post needs English skills + time to bring an up-to-date
> image in you head. So, status is declarative and can be processed
> by machines. Post is procedural and requires manual human labor.

The other difference is that nobody cares about a count of +/- status, so it doesn't really matter how easy it is to collect. If someone is trying to decide whether or not the PEP discussion is missing his point of view, a vote isn't going to tell him that. If Guido is trying to decide whether to approve or reject the PEP, he's not going to go by the vote. So what's the point of it?

From techtonik at gmail.com  Wed Mar 26 08:35:30 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 26 Mar 2014 10:35:30 +0300
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPkN8xJiND9FFQiqfd4gV4u2BH=6n7FWV7AgNeJPYW_YTfmxdQ@mail.gmail.com>

On Tue, Mar 25, 2014 at 10:16 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> anatoly techtonik writes:
>
>  > It is not that hard to implement an approval function for
>  > the PEP site (to improve the visibility and the process).
>
> I gather you mean "thumbs up/down" buttons.

It is not primitive "thumbs up/down" button and not voting.
The feedback loop goal to get information about community
awareness - not for making decision to accept or reject a
proposal. Therefore it is not democracy.

The approval is not a simple boolean - it is the status object,
which contains fields to make feedback loop actually useful,
and may provide information, such as:

  1. LGTM
  2. Have questions
  3. Have comments
  4. Will watch later
  5. tl;dr
  6. meh
  7. .... <link> to discussion
  8. ...

The status sums up opinion for a given person at a given
moment. Benefits:

  - see who is interested
  - see who was able to review
  - see who was able to read it
  - see which version has questions

I don't want to get into much featurecreeping, which is
incrementally possible.

Will that idea be useful for Python community?

> Links to the discussion (or perhaps even blog posts of folks who
> strongly dissent) are far more relevant.

This can be added, so that folks can tie their posts to PEP
feedback loop.

  9. Dissent <link>
  (new word for me, sounds strange)

-- 
anatoly t.

From techtonik at gmail.com  Wed Mar 26 09:07:32 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 26 Mar 2014 11:07:32 +0300
Subject: [Python-ideas] PEP feedback loop (with tracker account)
In-Reply-To: <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com>
References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com>
 <lgt10d$h6l$1@ger.gmane.org> <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com>
Message-ID: <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com>

On Wed, Mar 26, 2014 at 8:22 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote:
>>
>> No.  What is useful is to keep the "Post-History" header up to
>> date, which refers people to the python-dev threads where all the
>> discussion and "voting" takes place.
>>
>> I could also imagine adding direct links to the archive; however
>> many PEPs already add references in the text where appropriate.
>
> Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful.

The goal to link threads and PEP versions is reachable at the next
iteration once status per person is implemented. The benefit from
the system is that you don't need to dedicate a person to do this.

> A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs.

It is not a post. It is status per revision. The difference between
post and status is that status gives immediate overview (diffstat)
while post needs English skills + time to bring an up-to-date
image in you head. So, status is declarative and can be processed
by machines. Post is procedural and requires manual human labor.

Status can serve as intermediate entrypoint to join current problem
field (as opposed to rereading posts from the start).
-- 
anatoly t.

From g.rodola at gmail.com  Wed Mar 26 15:37:40 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Wed, 26 Mar 2014 15:37:40 +0100
Subject: [Python-ideas] Turn signal.SIG* constants into enums
Message-ID: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com>

In Python 3.4 some module constants were turned into enums.
The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*).
For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_* constants:
http://bugs.python.org/issue21068
To me signal module looks like another good candidate which could benefit
from this.
Here's a real world example:
http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm

Thoughts?

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140326/4e681b06/attachment.html>

From guido at python.org  Wed Mar 26 16:37:53 2014
From: guido at python.org (Guido van Rossum)
Date: Wed, 26 Mar 2014 08:37:53 -0700
Subject: [Python-ideas] Turn signal.SIG* constants into enums
In-Reply-To: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com>
References: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com>
Message-ID: <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com>

SGTM.
On Mar 26, 2014 7:38 AM, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote:

> In Python 3.4 some module constants were turned into enums.
> The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*).
> For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_* constants:
> http://bugs.python.org/issue21068
> To me signal module looks like another good candidate which could benefit
> from this.
> Here's a real world example:
>
> http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm
>
> Thoughts?
>
> --
> Giampaolo - http://grodola.blogspot.com
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140326/85e7fa14/attachment.html>

From sturla.molden at gmail.com  Wed Mar 26 19:19:15 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Wed, 26 Mar 2014 19:19:15 +0100
Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for
 matrix multiplication and matrix power
In-Reply-To: <20140324004943.GX16526@ando>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz>
 <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz>
 <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org>
 <20140324004943.GX16526@ando>
Message-ID: <lgv5ml$ifu$1@ger.gmane.org>

On 24/03/14 01:49, Steven D'Aprano wrote:

> No it isn't. The PEP even discusses it:

(...)

Well, even with a single dimension, matrix multiplication has a useful 
interpretation (inner product). At least the intention in numpy is to 
interpret "vector @ vector" as the inner product. Thus it is useful both 
for list and array in the standard library. A possible usecase for it in 
the standard library would be to simplify the code in the statistics module.

But the main consideration is to make @ benign to the rest of the Python 
community.


> I don't think we should be looking for additional use-cases for @.
> Either the PEP stands on its own, and @ is approved for matrix
> multiplication (and possibly @@ for exponentiation),

The consensus on the NumPy list seems to be that @@ can wait. It is also 
ambigous. For example matrix @@ 0.5 could be interpreted as the Cholesky 
factorization or the matrix square root, depending on context.

Also an expression like Z = (X @@ -1) @ Y should for numerical stability 
(and speed) be computed as "solve X @ Z = Y" e.g. with LU, SVD or QR 
factorization. This would require calling a function like solve(X,Y). 
Matlab actually has a special back-divide operator for this reason: X \ Y.

This means that @@ is ambiguous except for positive integer powers or 
-1. But the common case of matrix @@ -1 is not as useful as we might 
think. Then we are left with exponentiation to positive integer powers, 
which is not nearly common enough to justify a special operator.


Sturla













From sturla.molden at gmail.com  Wed Mar 26 19:59:52 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Wed, 26 Mar 2014 19:59:52 +0100
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
Message-ID: <lgv82q$i25$1@ger.gmane.org>

On 14/03/14 18:53, Guido van Rossum wrote:

> - Did you consider a duck-typing (is that the word?) attribute?
>    E.g. a*b is elementwise multiplication; a.M*b must be used for
>    matrix multiplication.

This is exactly what has been tried with the types numpy.matrix and 
numpy.array. For numpy.matrix the * operator means matrix multiplication 
and for numpy.array it means Hadamard product.

In practice, it does not work as we want. That is the raison d'etre for 
this PEP.

First, we often needs both operators in the same expression. This makes 
it very hard to read, in fact a function call is always more readable.

Second, NumPy often has to return temporary arrays from binary 
operators, and the duck-typing of these must be controlled too. If we 
are unlucky a temporary array will have the wrong type, and we end up 
with errors that are very hard to track down.

So for any serious code, matrix multiplication is written with a 
function call: numpy.dot.

numpy.matrix is for this reason mostly useful for teaching students, not 
for serious numerical hacking with Python. It is due for retirement from 
NumPy.

A special operator will work, however, because languages like Matlab and 
R have proven that it does. Many in the NumPy community have extensive 
experience with Matlab and R, and we know it works with two 
multiplication operators. In contrast, we have proven that duck-typing 
is not useful in this context.

Though this PEP is written by Nathaniel, the whole NumPy dev team is 
behind it. It has been planned and discussed for a long time, both on 
the NumPy mailing list and on Github. But Nathaniel took the job to 
write it down. So other options has been considered very carefully.


> - Is @@ really necessary?

It seems the consensus on the NumPy list is that asking for @@ can wait.

- An expression like matrix @@ 0.5 is ambiguous because it could mean 
the matrix square root or the Cholesky factorization.

- An expression like Z = (X @@ -1) @ Y should for numerical stability 
ans speed be written as solving a set of equations: X @ Z = Y. 
Symbolically it is the same, but numerically it is not.

So for most practical purposes, matrix exponentiation will require a 
function call anyway. The unambiguous case of positive integer powers 
might not be common enough.


Sturla





From robert.kern at gmail.com  Wed Mar 26 20:14:09 2014
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 26 Mar 2014 19:14:09 +0000
Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators
 for matrix multiplication and matrix power
In-Reply-To: <lgv82q$i25$1@ger.gmane.org>
References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com>
 <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com>
 <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com>
 <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com>
 <lgv82q$i25$1@ger.gmane.org>
Message-ID: <lgv8th$t0h$1@ger.gmane.org>

On 2014-03-26 18:59, Sturla Molden wrote:
> On 14/03/14 18:53, Guido van Rossum wrote:
>
>> - Did you consider a duck-typing (is that the word?) attribute?
>>    E.g. a*b is elementwise multiplication; a.M*b must be used for
>>    matrix multiplication.
>
> This is exactly what has been tried with the types numpy.matrix and numpy.array.

For the record, no it hasn't. The proposal here is different than the 
numpy.matrix status quo. The proposed result of `a.M*b` would be an ndarray, not 
whatever the .M type is. The .M type would not be intended to be used outside of 
a multiplication expression. It is unlikely to propagate through ones code the 
way that numpy.matrix does (at least, it would be considered a bug to let a free 
.M type loose). One could safely write their functions expecting only 
numpy.ndarrays.

While this proposal has been considered and rejected, it is not the same as 
numpy.matrix and has been rejected for different reasons.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From g.rodola at gmail.com  Thu Mar 27 17:00:54 2014
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Thu, 27 Mar 2014 17:00:54 +0100
Subject: [Python-ideas] Turn signal.SIG* constants into enums
In-Reply-To: <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com>
References: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com>
 <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com>
Message-ID: <CAFYqXL9d67v7zDQS8tsETEatRp4cuOixNn9kSpbB6Remk6NFOw@mail.gmail.com>

Here's a patch: http://bugs.python.org/issue21076

On Wed, Mar 26, 2014 at 4:37 PM, Guido van Rossum <guido at python.org> wrote:

> SGTM.
> On Mar 26, 2014 7:38 AM, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote:
>
>> In Python 3.4 some module constants were turned into enums.
>> The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*).
>> For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_*
>> constants:
>> http://bugs.python.org/issue21068
>> To me signal module looks like another good candidate which could benefit
>> from this.
>> Here's a real world example:
>>
>> http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm
>>
>> Thoughts?
>>
>> --
>> Giampaolo - http://grodola.blogspot.com
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140327/a38510d9/attachment.html>

From clay.gerrard at gmail.com  Thu Mar 27 21:16:50 2014
From: clay.gerrard at gmail.com (Clay Gerrard)
Date: Thu, 27 Mar 2014 13:16:50 -0700
Subject: [Python-ideas] iterating over a generator while sending values
Message-ID: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>

Hi,

I often use generators when I want to bake some smarts into my producers in
a typical producer and consumer pattern.  Twice in the past year or so on
different projects after a few iterations on my producer/consumer
code ("iterations", ha!) I would eventually find I wanted my consumer to
send some simple feedback to the producer - when using generators send
works great for this!

I normally start here:

q = producer()

def consume(q):
    for item in q:
        success = handle(item)

But then ended up here:

def consume(q):
    success = None
    while True:
        try:
            item = q.send(success)
        except StopIteration:
            break
        success = handle(item)

But this always feels clunky; almost "out of order" - I've normally got
some logging and other bits in there.  I'm just trying to iterate over the
queue, the StopIteration just feels like noise.

It seems I always started out with a simple iteration and then end up
refactoring in the send function once I realize I need to add something
extra into my producer:

I want to just say:

def consume(q):
    for item in q:
        success = handle(item)
        q.send(success)  # oh how I wish I could just add this here!

But I can just "tack on" the send because it consumes too, so I have to
rephrase the "simple" iteration into a "while True try except StopIteration
break" - yuk.

I normally just end up adding a little helper function, which I've named
"sending" and use it like this:

def consume(q):
    for item in sending(q, lambda: success):
        success = handle(item)

^ much closer to the original iterative code!

Here's the implementation if you didn't already guess:

# this function is missing from stdlib for some reason...
def sending(g, sender):
    """
    Iterate over g with send

    :params g: the iterable
    :params sender: the callable returning the value to send
    """
    yield g.next()
    while True:
        yield g.send(sender())

This will raise StopIteration as soon as g is exhausted, but since the
caller is already consuming "sending" with a for loop it all works ok in
the end.

Does anyone else have a different way of iterating over a generator while
sending values?  Does everyone just already have a similar function stuck
in their bag?  Does this happen to other folks frequently enough that a
generalizable solution could be added to stdlib?  Maybe it's too simple to
be worth anyone's time or i'm missing something that already works better...

Thanks!

-clayg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140327/6748d86d/attachment.html>

From tjreedy at udel.edu  Thu Mar 27 23:24:54 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 27 Mar 2014 18:24:54 -0400
Subject: [Python-ideas] iterating over a generator while sending values
In-Reply-To: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>
References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>
Message-ID: <lh28fj$iif$1@ger.gmane.org>

On 3/27/2014 4:16 PM, Clay Gerrard wrote:
> Hi,
>
> I often use generators when I want to bake some smarts into my producers
> in a typical producer and consumer pattern.  Twice in the past year or
> so on different projects after a few iterations on my producer/consumer
> code ("iterations", ha!) I would eventually find I wanted my consumer to
> send some simple feedback to the producer - when using generators send
> works great for this!
>
> I normally start here:
>
> q = producer()
>
> def consume(q):
>      for item in q:
>          success = handle(item)
>
> But then ended up here:
>
> def consume(q):
>      success = None
>      while True:
>          try:
>              item = q.send(success)
>          except StopIteration:
>              break
>          success = handle(item)
>
> But this always feels clunky; almost "out of order" - I've normally got
> some logging and other bits in there.  I'm just trying to iterate over
> the queue, the StopIteration just feels like noise.

For Python, the normal mode of iteration is 'pull' mode -- the consumer 
pulls from the producer. I believe the send method was mostly designed 
for inversion of control with 'push' mode -- the producer pushes to the 
consumer with .send. The consumer might yield feedback to the producer. 
Sending feedback in pull mode is a secondary use.

The iterator protocol was designed for for statements: the initial call 
to iter(), multiple calls to next() and catching of StopIteration are 
all handled internally. Any time you want to do anything non-standard, 
other that stop before the iterator is exhausted, you have to deal with 
iter, next, and StopIteration yourself. StopIteration is an essential 
part of the protocol, not noise. An example:

def min_f(iterable):
     it = iter(iterable)
     try:
         val = next(it)
     except StopIteration:
         raise ValueError("an empty iterable has no minimum")
     for ob in it:
         if ob < val:
             val = ob
     return val

> It seems I always started out with a simple iteration and then end up
> refactoring in the send function once I realize I need to add something
> extra into my producer:
>
> I want to just say:
>
> def consume(q):
>      for item in q:
>          success = handle(item)
>          q.send(success)  # oh how I wish I could just add this here!
>
> But I can just "tack on" the send because it consumes too,

I think you meant "can't", but the problem with .send is that is returns 
the next items prematurely in addition to receiving something. That is 
for the design reason given above. You could just 'tack on' if you wrote 
an iterator class with .__iter__, .__next__, and .send methods, where 
.send only received (and returned an ignorable None).

> so I have to rephrase the "simple" iteration into a "while True try except
> StopIteration break" - yuk.

Catching StopIteration is part of explicit use of the iterator protocol, 
as in min_f above. In this case, one could make everything implicit 
again by using functools.reduce, with in its two parameter form, 
abstracts away the code pattern used in min_f.

from functools import reduce
def min_r(iterable):
     return reduce(lambda v, o: v if v <= o else o, iterable)

s = [5,6,4,6,3]
print(min_f(s), min_r(s))
 >>>
3, 3

> I normally just end up adding a little helper function, which I've named
> "sending" and use it like this:
>
> def consume(q):

        success = None

>      for item in sending(q, lambda: success):
>          success = handle(item)

> # this function is missing from stdlib for some reason...

Because it is short, special purpose, and pretty easy to write.

> def sending(g, sender):
>      """
>      Iterate over g with send
>
>      :params g: the iterable
>      :params sender: the callable returning the value to send
>      """
>      yield g.next()
>      while True:
>          yield g.send(sender())

Writing an iterator or generator that wrap an iterator or, as in this 
case, specifically a generator, is common. Some common patterns that 
work for iterators as input are included in itertools. More are given in 
the recipes section of the doc. The criterion for being in the module is 
that the abstraction either be common or useful in composition with 
others in the module. A specialized generator-input-only iterator would 
not fit.

It is possible that a 'gentools' module would be useful, but it should 
start on PyPI.

> This will raise StopIteration as soon as g is exhausted,

as per the protocol

> but since the caller is already consuming "sending" with a for loop
 > it all works ok in the end.

It does not matter whether the caller uses your generator in a for loop 
or explicit iter-while-next-except-StopIteration construct.

-- 
Terry Jan Reedy


From greg.ewing at canterbury.ac.nz  Fri Mar 28 00:17:43 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 28 Mar 2014 12:17:43 +1300
Subject: [Python-ideas] iterating over a generator while sending values
In-Reply-To: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>
References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>
Message-ID: <5334B197.2050300@canterbury.ac.nz>

Clay Gerrard wrote:

> def consume(q):
>     success = None
>     while True:
>         try:
>             item = q.send(success)
>         except StopIteration:
>             break
>         success = handle(item)

My first thought would be to ask whether you really need
to do things this way round. Instead of the consumer pulling
things from the producer, can you have the producer push
them to the consumer? Then the response is just a function
return value.

Assuming you do need to pull, my solution would be
something like this (not tested):

class Feedbackerator:

    def __init__(self, base):
       self.base = base
       self.response = None

    def __next__(self):
       return self.base.send(self.response)

def consume(q):
    f = Feedbackerator(q)
    for item in f:
       f.response = handle(item)

-- 
Greg

From aj at erisian.com.au  Fri Mar 28 05:22:01 2014
From: aj at erisian.com.au (Anthony Towns)
Date: Fri, 28 Mar 2014 14:22:01 +1000
Subject: [Python-ideas] iterating over a generator while sending values
In-Reply-To: <5334B197.2050300@canterbury.ac.nz>
References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com>
 <5334B197.2050300@canterbury.ac.nz>
Message-ID: <CAJS_LCVWKbFppGcwtwrFMyp1rL0Vup+HM8xPJswezayfrwvumA@mail.gmail.com>

On 28 March 2014 09:17, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Clay Gerrard wrote:
>> def consume(q):
>>     success = None
>>     while True:
>>         try:
>>             item = q.send(success)
>>         except StopIteration:
>>             break
>>         success = handle(item)
>
> Assuming you do need to pull, my solution would be
> something like this (not tested):
> class Feedbackerator:
>    def __init__(self, base):
>       self.base = base
>       self.response = None
>    def __next__(self):
>       return self.base.send(self.response)
> def consume(q):
>    f = Feedbackerator(q)
>    for item in f:
>       f.response = handle(item)

Ooo, nice. Minor tweaks to make it work in py2, and not require you to
set the response every iteration:

class Feedbackerator(object):
    def __init__(self, iterator):
        self.iter = iter(iterator)
        self.resp = None
    def next(self):
        r, self.resp = self.resp, None
        return self.iter.send(r)
    def __iter__(self):
        return self

def consume(q):
    qi = Feedbackerator(q)
    for item in qi:
        res = handle(item)
        if needs_feedback(res):
            qi.resp = feedback_for(res)

Cheers,
aj

-- 
Anthony Towns <aj at erisian.com.au>

From ncoghlan at gmail.com  Fri Mar 28 11:27:33 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Mar 2014 20:27:33 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
Message-ID: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>

One of the current annoyances with the bytes type in Python 3 is the
way the constructor handles integers:

>>> bytes(3)
b'\x00\x00\x00'

It would be far more consistent with the behaviour of other bytes
interfaces if the result of that call was instead b'\x03'. Instead, to
get that behaviour, you currently have to wrap it in a list or other
iterable:

>>> bytes([3])
b'\x03'

The other consequence of this is that's currently no neat way to
convert the integers produced by various bytes APIs back to a length
one bytes object - we have no binary equivalent of "chr" to convert an
integer in the range 0-255 inclusive to its bytes counterpart. The
acceptance of PEP 361 means we'll get another option (b"%c".__mod__)
but that's hardly what anyone would call obvious.

However, during a conversation today, a possible solution occurred to
me: a "bytes.chr" class method, that served as an alternate
constructor. That idea results in the following 3 part proposal:

1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
361 defined "b'%c' % x"

2. Add "bytearray.allnull" and "bytes.allnull" such that
"bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
handling

3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
not only ambiguous, but actually a genuine bug magnet (it's way too
easy to accidentally pass a large integer and try to allocate a
ridiculously large bytes object)

For point 2, I also considered the following alternative names before
settling on "allnull":

- bytes.null sounds too much like an alias for b"\x00"
- bytes.nulls just sounded too awkward to say (too many sibilants)
- bytes.zeros I can never remember how to spell (bytes.zeroes?)
- bytearray.cleared sort of worked, but bytes.cleared?
- ditto for bytearray.prealloc and bytes.prealloc (latter makes no sense)

That last is also a very C-ish name (although it is a rather C-ish operation).

Anyway, what do people think? Does anyone actually *like* the way the
bytes constructor in Python 3 currently handles integers and want to
keep it forever? Does the above proposal sound like a reasonable
suggestion for improvement in 3.5? Does this hit PEP territory, since
it's changing the signature and API of a builtin?

Cheers,
Nick.




-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Fri Mar 28 11:50:28 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Mar 2014 21:50:28 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <20140328105027.GH16526@ando>

On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote:
> One of the current annoyances with the bytes type in Python 3 is the
> way the constructor handles integers:
[...]

> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
> 361 defined "b'%c' % x"

+1 on the concept, but please not "chr". How about bytes.byte(x)? That 
emphasises that you are dealing with a single byte, and avoids 
conflating chars (strings of length 1) with bytes.


> 2. Add "bytearray.allnull" and "bytes.allnull" such that
> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
> handling

+1 on bytearray.allnull, with a mild preference for spelling it "zeroes" 
instead.

I'm indifferent about bytes.allnull. I'm not sure why I would use it 
in preference to b'\0'*x, or for that matter why I would use it at all. 
I suppose if there's a bytearray.allnul, for symmetry there should be a 
bytes.allnul as well.


> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
> not only ambiguous, but actually a genuine bug magnet (it's way too
> easy to accidentally pass a large integer and try to allocate a
> ridiculously large bytes object)

+1


[...]
> Anyway, what do people think? Does anyone actually *like* the way the
> bytes constructor in Python 3 currently handles integers and want to
> keep it forever? 

Not me!


> Does the above proposal sound like a reasonable
> suggestion for improvement in 3.5? Does this hit PEP territory, since
> it's changing the signature and API of a builtin?

On the vanishingly small chance that there is universal agreement on 
this, and a minimum of bike-shedding, I think it would still be useful 
to write up a brief half page PEP linking to the discussion here.


-- 
Steven

From ncoghlan at gmail.com  Fri Mar 28 12:00:44 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Mar 2014 21:00:44 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328105027.GH16526@ando>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
Message-ID: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>

On 28 March 2014 20:50, Steven D'Aprano <steve at pearwood.info> wrote:
>> 2. Add "bytearray.allnull" and "bytes.allnull" such that
>> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
>> handling
>
> +1 on bytearray.allnull, with a mild preference for spelling it "zeroes"
> instead.

If I could consistently remember whether to include the "e" or not,
"zeroes" would be my preference as well, but almost every time I go to
type it, I have to pause...

> I'm indifferent about bytes.allnull. I'm not sure why I would use it
> in preference to b'\0'*x, or for that matter why I would use it at all.
> I suppose if there's a bytearray.allnul, for symmetry there should be a
> bytes.allnul as well.

It's actually in the proposal solely because "bytes(x)" already works
that way, and "this is deprecated, use bytes.allnull instead" is a
much easier deprecation to sell.

>> Does the above proposal sound like a reasonable
>> suggestion for improvement in 3.5? Does this hit PEP territory, since
>> it's changing the signature and API of a builtin?
>
> On the vanishingly small chance that there is universal agreement on
> this, and a minimum of bike-shedding, I think it would still be useful
> to write up a brief half page PEP linking to the discussion here.

Yeah, that's make sense, so regardless of what happens in this thread,
I'll still be putting a PEP together and asking Guido for his verdict.
Heaps of time before 3.5 though...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Fri Mar 28 12:23:06 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 28 Mar 2014 20:23:06 +0900
Subject: [Python-ideas]  Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <874n2ipnxh.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:
 > One of the current annoyances with the bytes type in Python 3 is the
 > way the constructor handles integers:
 > 
 > >>> bytes(3)
 > b'\x00\x00\x00'
 > 
 > It would be far more consistent with the behaviour of other bytes
 > interfaces if the result of that call was instead b'\x03'.

+1

~ 13:49$ python3.3
Python 3.3.5 (default, Mar  9 2014, 08:10:50) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> str(3)
'3'
>>> list(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> tuple(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable

The behavior of str(3) is not exactly an argument *for* your claim,
but it's not an argument against since there's no reason to expect a
byte to have an ASCII interpretation in general.

 > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
 > not only ambiguous, but actually a genuine bug magnet (it's way too
 > easy to accidentally pass a large integer and try to allocate a
 > ridiculously large bytes object)

bytes.chr(accidental_large_integer) presumably raises.  I don't really
see a bigger problem with the current behavior than that (in the
accidentally large case).  Both are likely to distress users.

 > Anyway, what do people think? Does anyone actually *like* the way the
 > bytes constructor in Python 3 currently handles integers and want to
 > keep it forever?

No.  bytearray I'd have to think about.

 > Does the above proposal sound like a reasonable suggestion for
 > improvement in 3.5?

Yes.

 > Does this hit PEP territory, since it's changing the signature and
 > API of a builtin?

No opinion.

Steve

From solipsis at pitrou.net  Fri Mar 28 12:22:51 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 28 Mar 2014 12:22:51 +0100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <20140328122251.1933e4e4@fsol>

On Fri, 28 Mar 2014 20:27:33 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> One of the current annoyances with the bytes type in Python 3 is the
> way the constructor handles integers:
> 
> >>> bytes(3)
> b'\x00\x00\x00'
> 
> It would be far more consistent with the behaviour of other bytes
> interfaces if the result of that call was instead b'\x03'.

Which other bytes interfaces are you talking about?

> However, during a conversation today, a possible solution occurred to
> me: a "bytes.chr" class method, that served as an alternate
> constructor. That idea results in the following 3 part proposal:
> 
> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
> 361 defined "b'%c' % x"

You mean bytes.chr(x) is equivalent to bytes([x]). The intent is
slightly more obvious indeed, so I'd inclined to be +0, but Python
isn't really a worse language if it doesn't have that alternative
spelling.

> Anyway, what do people think? Does anyone actually *like* the way the
> bytes constructor in Python 3 currently handles integers and want to
> keep it forever?

I don't like it, but I also don't think it's enough of a nuisance to be
deprecated.
(the indexing behaviour of bytes objects is far more annoying)

Regards

Antoine.



From cory at lukasa.co.uk  Fri Mar 28 12:28:57 2014
From: cory at lukasa.co.uk (Cory Benfield)
Date: Fri, 28 Mar 2014 11:28:57 +0000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <CAH_hAJEu5m74QWYavwxQcmiMc3M1ZMLjstDQYQMqZUXZzid8uA@mail.gmail.com>

On 28 March 2014 10:27, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Anyway, what do people think? Does anyone actually *like* the way the
> bytes constructor in Python 3 currently handles integers and want to
> keep it forever?

I hate it, I'd love to see a change. It bit me when I wrote my first
Python-3-only project, and while it wasn't hard to debug it was
utterly baffling for quite a while. Writing bytes([integer]) is pretty
ugly, and it makes me sad each time I see it. I've even commented
occurrences of it to remind myself that this is really the way I meant
to do it.


> Does the above proposal sound like a reasonable
> suggestion for improvement in 3.5? Does this hit PEP territory, since
> it's changing the signature and API of a builtin?

I'd like to see it improved, even if I can't take advantage of that
improvement for quite some time. Sounds like a PEP to me.

From ncoghlan at gmail.com  Fri Mar 28 12:59:33 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Mar 2014 21:59:33 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328122251.1933e4e4@fsol>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
Message-ID: <CADiSq7ej4ENX52HK-n716cvK8Sg7Be51KMAnq-dZqp7NciTvXw@mail.gmail.com>

On 28 March 2014 21:22, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 28 Mar 2014 20:27:33 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> One of the current annoyances with the bytes type in Python 3 is the
>> way the constructor handles integers:
>>
>> >>> bytes(3)
>> b'\x00\x00\x00'
>>
>> It would be far more consistent with the behaviour of other bytes
>> interfaces if the result of that call was instead b'\x03'.
>
> Which other bytes interfaces are you talking about?

The ones where a length 1 bytes object and the corresponding integer
are interchangeable. For example, containment testing:

>>> b"a" in b"abc"
True
>>> 97 in b"abc"
True

Compare:

>>> len(bytes(b"a"))
1
>>> len(bytes(97))
97

That's the inconsistency that elevates the current constructor
behaviour from weird to actively wrong for me - it doesn't match the
way other bytes interfaces have evolved over the course of the Python
3 series.

>> However, during a conversation today, a possible solution occurred to
>> me: a "bytes.chr" class method, that served as an alternate
>> constructor. That idea results in the following 3 part proposal:
>>
>> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
>> 361 defined "b'%c' % x"
>
> You mean bytes.chr(x) is equivalent to bytes([x]). The intent is
> slightly more obvious indeed, so I'd inclined to be +0, but Python
> isn't really a worse language if it doesn't have that alternative
> spelling.
>
>> Anyway, what do people think? Does anyone actually *like* the way the
>> bytes constructor in Python 3 currently handles integers and want to
>> keep it forever?
>
> I don't like it, but I also don't think it's enough of a nuisance to be
> deprecated.
> (the indexing behaviour of bytes objects is far more annoying)

Oops, I forgot to explain the context where this idea came up: I was
trying to figure out how to iterate over a bytes object or wrap an
indexing operation to get a length 1 byte sequence rather than an
integer.

Currently: probably muck about with lambda or a comprehension

With this change (using Steven D'Aprano's suggested name):

    for x in map(bytes.byte, data):
        # x is a length 1 bytes object, not an int

    x = bytes.byte(data[0]) # ditto

bytes.byte could actually apply the same policy as some other APIs and
also accept ASCII text code points in addition to length 1 bytes
objects and integers below 256.

Since changing the iteration and indexing behaviour of bytes and
bytearray within the Python 3 series isn't feasible, this idea is
about making the current behaviour easier to deal with.

And yes, this is definitely going to need a PEP :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From python at 2sn.net  Fri Mar 28 13:06:28 2014
From: python at 2sn.net (Alexander Heger)
Date: Fri, 28 Mar 2014 23:06:28 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
 <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
Message-ID: <CAN3CYHz=e1DXR+izFn33BxcRNGJzoHLR6HY8Ukg4Uix7OgmNEw@mail.gmail.com>

>> +1 on bytearray.allnull, with a mild preference for spelling it "zeroes"
>> instead.
>
> If I could consistently remember whether to include the "e" or not,
> "zeroes" would be my preference as well, but almost every time I go to
> type it, I have to pause...

in Numpy a similar routine is

In [1]: np.zeros_like
Out[1]: <function numpy.core.numeric.zeros_like>

so keeping the spelling consistent with other key libraries,
bytes.zeros() and bytearray.zeros() would be better.

-Alexander

From greg.ewing at canterbury.ac.nz  Fri Mar 28 13:16:16 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 29 Mar 2014 01:16:16 +1300
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
 <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
Message-ID: <53356810.5060602@canterbury.ac.nz>

Nick Coghlan wrote:
> If I could consistently remember whether to include the "e" or not,
> "zeroes" would be my preference as well, but almost every time I go to
> type it, I have to pause...

Whichever you choose, please make sure it's the same as
whatever numpy uses!

-- 
Greg

From ron3200 at gmail.com  Fri Mar 28 14:41:02 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 28 Mar 2014 09:41:02 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <lh3u4v$44v$1@ger.gmane.org>



On 03/28/2014 06:27 AM, Nick Coghlan wrote:
> However, during a conversation today, a possible solution occurred to
> me: a "bytes.chr" class method, that served as an alternate
> constructor. That idea results in the following 3 part proposal:
>
> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
> 361 defined "b'%c' % x"

I'd expect to use it this way...

     bytes.chr('3')

Which wouldn't be correct.



We have this...

 >>> (3).to_bytes(1,"little")
b'\x03'


OR...

 >>> def hexclean(h):
...     if h.startswith('0x'):
...         h = h[2:]
...     if not len(h)//2:
...         h = '0' + h
...     return h
...
 >>> bytes.fromhex(hexclean(hex(3)))
b'\x03'


Both of those seem like way too much work to me.



Bytes is a container object, so it makes sense to do...

    bytes([3])



The other interface you're suggesting is more along the lines of converting 
an object to bytes.  That would be a nice feature if it can be made to work 
with more objects, and then covert back again.  (even if only a few types 
was supported)

     b_three = bytes.from_obj(3)

     three = b_three.to_obj(int)


That wouldn't be limited to values between 0 and 255 like the hex version 
above.  (it doesn't need a length, or byteorder args.)



> 2. Add "bytearray.allnull" and "bytes.allnull" such that
> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
> handling

Numpy uses zeros and shape...

 >>> y = np.zeros((2, 3, 4))
 >>> y.shape
(2, 3, 4)

(Without the e.)  I'm not sure if it's a good idea to use zeros with a 
different signature.

There has been mentioned, that adding multidimensional slicing to python 
may be done.  So it may be a good idea to follow numpy's usage.




> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
> not only ambiguous, but actually a genuine bug magnet (it's way too
> easy to accidentally pass a large integer and try to allocate a
> ridiculously large bytes object)

Agree


Cheers,
    Ron


From barry at python.org  Fri Mar 28 15:28:17 2014
From: barry at python.org (Barry Warsaw)
Date: Fri, 28 Mar 2014 10:28:17 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <20140328102817.76d2b7f9@anarchist.wooz.org>

Great idea Nick.  If I may dip my brush in some paint buckets.

On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote:

>However, during a conversation today, a possible solution occurred to
>me: a "bytes.chr" class method, that served as an alternate
>constructor. That idea results in the following 3 part proposal:
>
>1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
>361 defined "b'%c' % x"

I agree with Steven that bytes.byte() is a better spelling.

>2. Add "bytearray.allnull" and "bytes.allnull" such that
>"bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
>handling

I like bytearray.fill() for this.  The first argument would be the fill count,
but it could take an optional second argument for the byte value to fill it
with, which would of course default to zero.  E.g.

>>> bytearray.fill(5)
bytearray(b'\x00\x00\x00\x00\x00')
>>> bytearray.fill(5, 97)
bytearray(b'aaaaa')

>3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
>not only ambiguous, but actually a genuine bug magnet (it's way too
>easy to accidentally pass a large integer and try to allocate a
>ridiculously large bytes object)

+1

>Does the above proposal sound like a reasonable suggestion for improvement in
>3.5?

Very much so.

>Does this hit PEP territory, since it's changing the signature and API
>of a builtin?

I don't much care either way.  A PEP is not *wrong* (especially if we all
start painting), but I think a tracker issue would be fine too.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/03763789/attachment.sig>

From breamoreboy at yahoo.co.uk  Fri Mar 28 15:48:01 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 28 Mar 2014 14:48:01 +0000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <lh422u$ngs$1@ger.gmane.org>

On 28/03/2014 14:28, Barry Warsaw wrote:
> Great idea Nick.  If I may dip my brush in some paint buckets.
>
> On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote:
>
>> However, during a conversation today, a possible solution occurred to
>> me: a "bytes.chr" class method, that served as an alternate
>> constructor. That idea results in the following 3 part proposal:
>>
>> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
>> 361 defined "b'%c' % x"
>
> I agree with Steven that bytes.byte() is a better spelling.
>
>> 2. Add "bytearray.allnull" and "bytes.allnull" such that
>> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
>> handling
>
> I like bytearray.fill() for this.  The first argument would be the fill count,
> but it could take an optional second argument for the byte value to fill it
> with, which would of course default to zero.  E.g.
>
>>>> bytearray.fill(5)
> bytearray(b'\x00\x00\x00\x00\x00')
>>>> bytearray.fill(5, 97)
> bytearray(b'aaaaa')
>
>> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
>> not only ambiguous, but actually a genuine bug magnet (it's way too
>> easy to accidentally pass a large integer and try to allocate a
>> ridiculously large bytes object)
>
> +1
>
>> Does the above proposal sound like a reasonable suggestion for improvement in
>> 3.5?
>
> Very much so.
>
>> Does this hit PEP territory, since it's changing the signature and API
>> of a builtin?
>
> I don't much care either way.  A PEP is not *wrong* (especially if we all
> start painting), but I think a tracker issue would be fine too.
>
> Cheers,
> -Barry
>

I was under the impression that Ethan Furman had raised an issue, or at 
least commented on one, but I couldn't find such a thing, am I simply 
mistaken?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From ethan at stoneleaf.us  Fri Mar 28 15:31:01 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 28 Mar 2014 07:31:01 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <533587A5.9010701@stoneleaf.us>

On 03/28/2014 07:28 AM, Barry Warsaw wrote:
> Great idea Nick.  If I may dip my brush in some paint buckets.
>
> On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote:
>
>> However, during a conversation today, a possible solution occurred to
>> me: a "bytes.chr" class method, that served as an alternate
>> constructor. That idea results in the following 3 part proposal:
>>
>> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
>> 361 defined "b'%c' % x"
>
> I agree with Steven that bytes.byte() is a better spelling.

+1


>> 2. Add "bytearray.allnull" and "bytes.allnull" such that
>> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
>> handling
>
> I like bytearray.fill() for this.  The first argument would be the fill count,
> but it could take an optional second argument for the byte value to fill it
> with, which would of course default to zero.  E.g.

+1


>> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
>> not only ambiguous, but actually a genuine bug magnet (it's way too
>> easy to accidentally pass a large integer and try to allocate a
>> ridiculously large bytes object)
>
> +1

+1


>> Does the above proposal sound like a reasonable suggestion for improvement in
>> 3.5?

+1

>> Does this hit PEP territory, since it's changing the signature and API
>> of a builtin?
>
> I don't much care either way.  A PEP is not *wrong* (especially if we all
> start painting), but I think a tracker issue would be fine too.

You mean like http://bugs.python.org/issue20895 ?  :)

--
~Ethan~

From barry at python.org  Fri Mar 28 16:13:47 2014
From: barry at python.org (Barry Warsaw)
Date: Fri, 28 Mar 2014 11:13:47 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
 <533587A5.9010701@stoneleaf.us>
Message-ID: <20140328111347.20495c04@anarchist.wooz.org>

On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote:

>You mean like http://bugs.python.org/issue20895 ?  :)

Step *away* from the time machine.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/53c52266/attachment-0001.sig>

From nas-python at arctrix.com  Fri Mar 28 16:21:15 2014
From: nas-python at arctrix.com (Neil Schemenauer)
Date: Fri, 28 Mar 2014 09:21:15 -0600
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328105027.GH16526@ando>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
Message-ID: <20140328152115.GA8168@python.ca>

On 2014-03-28, Steven D'Aprano wrote:
> On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote:
> +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" 
> instead.

I think numpy uses 'zeros' so we should use that.

> > Anyway, what do people think? Does anyone actually *like* the way the
> > bytes constructor in Python 3 currently handles integers and want to
> > keep it forever? 
> 
> Not me!

It's odd, IMHO.  When looking to implement %-interpolation for bytes
I did some searching to see how widely it is used.  There are a few
uses in the standard library but it really should have been a
special constructor.   Accidental uses probably outnumber
legitimate ones, quite a bad API design.

  Neil

From alexander.belopolsky at gmail.com  Fri Mar 28 16:28:39 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 28 Mar 2014 11:28:39 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <lh3u4v$44v$1@ger.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <lh3u4v$44v$1@ger.gmane.org>
Message-ID: <CAP7h-xY8fit8s-b2QcTYq=nktO4dTDTRqrCxwUgo3ZcXZ-hEjA@mail.gmail.com>

On Fri, Mar 28, 2014 at 9:41 AM, Ron Adam <ron3200 at gmail.com> wrote:

> >>> y = np.zeros((2, 3, 4))
> >>> y.shape
> (2, 3, 4)
>
> (Without the e.)  I'm not sure if it's a good idea to use zeros with a
> different signature.
>

np.zeros() can be called with an integer (length) argument as well:

>>> np.zeros(3)
array([ 0.,  0.,  0.])

To be completely analogous to the proposed bytes constructor, you would
have to specify the data type, though:

>>> np.zeros(3, 'B')
array([0, 0, 0], dtype=uint8)

+1 for bytes.zeros(n)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/7d23c9e3/attachment.html>

From alexander.belopolsky at gmail.com  Fri Mar 28 16:36:40 2014
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 28 Mar 2014 11:36:40 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <CAP7h-xZOhPxWV2rB7MVQ_ny1kr=eRifz05=tdMM7tRokYxjGmQ@mail.gmail.com>

On Fri, Mar 28, 2014 at 10:28 AM, Barry Warsaw <barry at python.org> wrote:

> I like bytearray.fill() for this.  The first argument would be the fill
> count,
> but it could take an optional second argument for the byte value to fill it
> with, which would of course default to zero.  E.g.
>
> >>> bytearray.fill(5)
> bytearray(b'\x00\x00\x00\x00\x00')
> >>> bytearray.fill(5, 97)
> bytearray(b'aaaaa')
>

In numpy this constructor is spelled "full":

>>> np.full(3, 42)
array([ 42.,  42.,  42.])

(Which makes more sense in the full context: np.zeros, np.ones, np.empty,
np.full.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/e197e8ed/attachment.html>

From solipsis at pitrou.net  Fri Mar 28 17:29:27 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 28 Mar 2014 17:29:27 +0100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <20140328172927.42983ab6@fsol>

On Fri, 28 Mar 2014 10:28:17 -0400
Barry Warsaw <barry at python.org> wrote:
> 
> I like bytearray.fill() for this.  The first argument would be the fill count,
> but it could take an optional second argument for the byte value to fill it
> with, which would of course default to zero.  E.g.
> 
> >>> bytearray.fill(5)
> bytearray(b'\x00\x00\x00\x00\x00')
> >>> bytearray.fill(5, 97)
> bytearray(b'aaaaa')

Do we need another spelling for a very uncommon use case?

>>> bytearray([0]) * 5
bytearray(b'\x00\x00\x00\x00\x00')
>>> bytearray([97]) * 5
bytearray(b'aaaaa')

Let's keep it simple.

Regards

Antoine.



From ericsnowcurrently at gmail.com  Fri Mar 28 17:34:20 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 28 Mar 2014 10:34:20 -0600
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
 <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
Message-ID: <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com>

On Mar 28, 2014 5:01 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
> If I could consistently remember whether to include the "e" or not,
> "zeroes" would be my preference as well, but almost every time I go to
> type it, I have to pause...

However, it reads better, which trumps the possible minor spelling
inconvenience. (all else equal, readability trumps write-ability, etc.)

> It's actually in the proposal solely because "bytes(x)" already works
> that way, and "this is deprecated, use bytes.allnull instead" is a
> much easier deprecation to sell.

Fair point, though I also agree with Steven here.  Could the replacement
(b'\0'*x) be a doc note instead of a new method?

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/3ca290ec/attachment.html>

From ericsnowcurrently at gmail.com  Fri Mar 28 17:37:03 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 28 Mar 2014 10:37:03 -0600
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328122251.1933e4e4@fsol>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
Message-ID: <CALFfu7DBiMXOqctxwqTb6HB05334CaYEaWZfcbtf6y=v435C4Q@mail.gmail.com>

On Mar 28, 2014 5:24 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
> (the indexing behaviour of bytes objects is far more annoying)

+1

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/e56e4291/attachment.html>

From jeanpierreda at gmail.com  Fri Mar 28 17:40:00 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 28 Mar 2014 09:40:00 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando>
 <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com>
 <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com>
Message-ID: <CABicbJL2LigSbOAZpyodWbh77FK-wfMdfS_n77qroF5gzey6sg@mail.gmail.com>

On Fri, Mar 28, 2014 at 9:34 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>
> On Mar 28, 2014 5:01 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>> If I could consistently remember whether to include the "e" or not,
>> "zeroes" would be my preference as well, but almost every time I go to
>> type it, I have to pause...
>
> However, it reads better, which trumps the possible minor spelling
> inconvenience. (all else equal, readability trumps write-ability, etc.)

If I have to pause while reading to figure out if the code has a typo,
that isn't better.

-- Devin

From steve at pearwood.info  Fri Mar 28 18:02:23 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Mar 2014 04:02:23 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328152115.GA8168@python.ca>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328105027.GH16526@ando> <20140328152115.GA8168@python.ca>
Message-ID: <20140328170222.GK16526@ando>

On Fri, Mar 28, 2014 at 09:21:15AM -0600, Neil Schemenauer wrote:
> On 2014-03-28, Steven D'Aprano wrote:
> > On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote:
> > +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" 
> > instead.
> 
> I think numpy uses 'zeros' so we should use that.

We would we want to duplicate their spelling error? *wink*

I think I'd rather Nick's original suggestion allnull than "zeros". 
"Zeros" sounds like it ought to be the name of a Greek singer :-)

I'm aware that zeros/zeroes are both considered acceptable variant 
spellings. Not acceptable to me *wink*


-- 
Steven

From benjamin at python.org  Fri Mar 28 18:06:24 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 28 Mar 2014 17:06:24 +0000 (UTC)
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
Message-ID: <loom.20140328T180549-288@post.gmane.org>

Antoine Pitrou <solipsis at ...> writes:
> I don't like it, but I also don't think it's enough of a nuisance to be
> deprecated.
> (the indexing behaviour of bytes objects is far more annoying)

I agree. We would add a method that iterates over single bytes as bytes
objects instead of integers.


From guido at python.org  Fri Mar 28 18:17:34 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 28 Mar 2014 10:17:34 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <loom.20140328T180549-288@post.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
Message-ID: <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>

On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson <benjamin at python.org>wrote:

> Antoine Pitrou <solipsis at ...> writes:
> > I don't like it, but I also don't think it's enough of a nuisance to be
> > deprecated.
> > (the indexing behaviour of bytes objects is far more annoying)
>
> I agree. We would add a method that iterates over single bytes as bytes
> objects instead of integers.
>

Now that's making sense.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/89438e19/attachment.html>

From benjamin at python.org  Fri Mar 28 18:23:09 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 28 Mar 2014 17:23:09 +0000 (UTC)
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
Message-ID: <loom.20140328T182140-300@post.gmane.org>

Guido van Rossum <guido at ...> writes:

> 
> 
> 
> On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson
<benjamin at python.org> wrote:
> Antoine Pitrou <solipsis <at> ...> writes:
> > I don't like it, but I also don't think it's enough of a nuisance to be
> > deprecated.
> > (the indexing behaviour of bytes objects is far more annoying)
> I agree. We would add a method that iterates over single bytes as bytes
> objects instead of integers.
> 
> 
> 
> 
> Now that's making sense.

Is this sarcasm about my poor writing skills ("would" should be "could") or
actual support for the idea?




From solipsis at pitrou.net  Fri Mar 28 18:25:44 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 28 Mar 2014 18:25:44 +0100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
 <loom.20140328T180549-288@post.gmane.org>
Message-ID: <20140328182544.62ce91ad@fsol>

On Fri, 28 Mar 2014 17:06:24 +0000 (UTC)
Benjamin Peterson <benjamin at python.org>
wrote:
> Antoine Pitrou <solipsis at ...> writes:
> > I don't like it, but I also don't think it's enough of a nuisance to be
> > deprecated.
> > (the indexing behaviour of bytes objects is far more annoying)
> 
> I agree. We would add a method that iterates over single bytes as bytes
> objects instead of integers.

In my experience, indexing is far more common than iterating over bytes.

Regards

Antoine.



From flying-sheep at web.de  Fri Mar 28 18:31:58 2014
From: flying-sheep at web.de (Philipp A.)
Date: Fri, 28 Mar 2014 18:31:58 +0100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
Message-ID: <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com>

2014-03-28 18:17 GMT+01:00 Guido van Rossum <guido at python.org>:


> On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson <benjamin at python.org>wrote:
>
>> I agree. We would add a method that iterates over single bytes as bytes
>> objects instead of integers.
>>
>
> Now that's making sense.
>
even if some people don?t like it, the iterating over bytes objects makes
perfect sense.

in every statically typed language, a byte is most commonly repesented as
an integer in the range [0,255].

the annoyance has two sources:

   1.

   python 2 byte strings (aka ?native strings?) behaved like that (and
   unicode strings behave like that because python has no char type)
    2.

   byte strings are represented like strings, including ascii-compatible
   parts as if they were ASCII text.

therefore, paople think single bytes would be something akin to chars
instead of simply integers from 0 to 255.

b'a'[0] == 97 looks strange, but let?s not forget that it?s actually b'\x61'[0]
== 0x61
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/69b70f27/attachment-0001.html>

From solipsis at pitrou.net  Fri Mar 28 18:42:41 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 28 Mar 2014 18:42:41 +0100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
 <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com>
Message-ID: <20140328184241.04693b7a@fsol>

On Fri, 28 Mar 2014 18:31:58 +0100
"Philipp A." <flying-sheep at web.de> wrote:
> even if some people don?t like it, the iterating over bytes objects makes
> perfect sense.
> 
> in every statically typed language, a byte is most commonly repesented as
> an integer in the range [0,255].
> 
> the annoyance has two sources:
> 
>    1.
> 
>    python 2 byte strings (aka ?native strings?) behaved like that (and
>    unicode strings behave like that because python has no char type)
>     2.
> 
>    byte strings are represented like strings, including ascii-compatible
>    parts as if they were ASCII text.

The actual source isn't merely cultural. It's that most of the time,
bytes objects are used as containers of arbitrary binary data, not
as arrays of integers (on which you would do element-wise arithmetic
calculations, for instance).

So, even if the current behaviour makes sense, it's not optimal for the
common uses of bytes objects.

Regards

Antoine.



From guido at python.org  Fri Mar 28 18:50:34 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 28 Mar 2014 10:50:34 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <loom.20140328T182140-300@post.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <loom.20140328T182140-300@post.gmane.org>
Message-ID: <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com>

Support.


On Fri, Mar 28, 2014 at 10:23 AM, Benjamin Peterson <benjamin at python.org>wrote:

> Guido van Rossum <guido at ...> writes:
>
> >
> >
> >
> > On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson
> <benjamin at python.org> wrote:
> > Antoine Pitrou <solipsis <at> ...> writes:
> > > I don't like it, but I also don't think it's enough of a nuisance to be
> > > deprecated.
> > > (the indexing behaviour of bytes objects is far more annoying)
> > I agree. We would add a method that iterates over single bytes as bytes
> > objects instead of integers.
> >
> >
> >
> >
> > Now that's making sense.
>
> Is this sarcasm about my poor writing skills ("would" should be "could") or
> actual support for the idea?
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/6c33b70b/attachment.html>

From benjamin at python.org  Fri Mar 28 18:54:04 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 28 Mar 2014 17:54:04 +0000 (UTC)
Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing the
	Python 3 bytes constructor)
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <loom.20140328T182140-300@post.gmane.org>
 <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com>
Message-ID: <loom.20140328T185247-589@post.gmane.org>

Guido van Rossum <guido at ...> writes:

> 
> 
> Support.

Excellent. I think the means the only thing left to do is bikeshed the name.
iterbytes() maybe? That's what it's called in six.





From ryan at ryanhiebert.com  Fri Mar 28 18:52:37 2014
From: ryan at ryanhiebert.com (Ryan Hiebert)
Date: Fri, 28 Mar 2014 12:52:37 -0500
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328184241.04693b7a@fsol>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol>
 <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com>
 <20140328184241.04693b7a@fsol>
Message-ID: <CABpHFHRcq7k8uApDcU+VFJjL8mpgOPVBNvz2SK6FAcBy9293hQ@mail.gmail.com>

On Fri, Mar 28, 2014 at 12:42 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> On Fri, 28 Mar 2014 18:31:58 +0100
> "Philipp A." <flying-sheep at web.de> wrote:
> > even if some people don't like it, the iterating over bytes objects makes
> > perfect sense.
> >
> > in every statically typed language, a byte is most commonly repesented as
> > an integer in the range [0,255].
> >
> > the annoyance has two sources:
> >
> >    1.
> >
> >    python 2 byte strings (aka "native strings") behaved like that (and
> >    unicode strings behave like that because python has no char type)
> >     2.
> >
> >    byte strings are represented like strings, including ascii-compatible
> >    parts as if they were ASCII text.
>
> The actual source isn't merely cultural. It's that most of the time,
> bytes objects are used as containers of arbitrary binary data, not
> as arrays of integers (on which you would do element-wise arithmetic
> calculations, for instance).
>
> So, even if the current behaviour makes sense, it's not optimal for the
> common uses of bytes objects.


I don't see bytes as integers, but as representations of integers, that
typically come in a stream or array. Using an integer to represent
that byte sometimes makes sense, but sometimes you wish to refer
to the representation of the integer (the byte), not the integer itself.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/d2db8b2e/attachment.html>

From random832 at fastmail.us  Fri Mar 28 19:31:01 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 28 Mar 2014 14:31:01 -0400
Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing
 the Python 3 bytes constructor)
In-Reply-To: <loom.20140328T185247-589@post.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <loom.20140328T182140-300@post.gmane.org>
 <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com>
 <loom.20140328T185247-589@post.gmane.org>
Message-ID: <1396031461.31107.100141201.4542F733@webmail.messagingengine.com>

On Fri, Mar 28, 2014, at 13:54, Benjamin Peterson wrote:
> Guido van Rossum <guido at ...> writes:
> 
> > Support.
> 
> Excellent. I think the means the only thing left to do is bikeshed the
> name.
> iterbytes() maybe? That's what it's called in six.

Should this be purely an iteration function, or something that can also
index and slice [into objects that can subsequently be iterated or
indexed]? What does six do about that?

From random832 at fastmail.us  Fri Mar 28 19:38:06 2014
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 28 Mar 2014 14:38:06 -0400
Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing
 the Python 3 bytes constructor)
In-Reply-To: <loom.20140328T185247-589@post.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
 <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com>
 <loom.20140328T182140-300@post.gmane.org>
 <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com>
 <loom.20140328T185247-589@post.gmane.org>
Message-ID: <1396031886.356.100142697.5C37CC66@webmail.messagingengine.com>

On Fri, Mar 28, 2014, at 13:54, Benjamin Peterson wrote:
> Excellent. I think the means the only thing left to do is bikeshed the
> name.
> iterbytes() maybe? That's what it's called in six.

The thing called that in six is actually exactly the opposite of this.
It doesn't seem to have functions for this.

From storchaka at gmail.com  Fri Mar 28 22:09:15 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 28 Mar 2014 23:09:15 +0200
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <lh4oc2$ono$1@ger.gmane.org>

28.03.14 12:27, Nick Coghlan ???????(??):
> The other consequence of this is that's currently no neat way to
> convert the integers produced by various bytes APIs back to a length
> one bytes object - we have no binary equivalent of "chr" to convert an
> integer in the range 0-255 inclusive to its bytes counterpart. The
> acceptance of PEP 361 means we'll get another option (b"%c".__mod__)
> but that's hardly what anyone would call obvious.

AFAIK currently the fastest way is:

packB = struct.Struct('B').pack

But lambda x: bytes([x]) looks most obvious.

> However, during a conversation today, a possible solution occurred to
> me: a "bytes.chr" class method, that served as an alternate
> constructor. That idea results in the following 3 part proposal:
>
> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
> 361 defined "b'%c' % x"

-0. This is not very needed method and is not worth the language 
complication. bytes([x]) works pretty good in most cases.

> 2. Add "bytearray.allnull" and "bytes.allnull" such that
> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
> handling

-1. b'\0' * x does this.

> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
> not only ambiguous, but actually a genuine bug magnet (it's way too
> easy to accidentally pass a large integer and try to allocate a
> ridiculously large bytes object)

+0.



From storchaka at gmail.com  Fri Mar 28 22:15:35 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 28 Mar 2014 23:15:35 +0200
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <loom.20140328T180549-288@post.gmane.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org>
Message-ID: <lh4ons$tdr$1@ger.gmane.org>

28.03.14 19:06, Benjamin Peterson ???????(??):
> Antoine Pitrou <solipsis at ...> writes:
>> I don't like it, but I also don't think it's enough of a nuisance to be
>> deprecated.
>> (the indexing behaviour of bytes objects is far more annoying)
>
> I agree. We would add a method that iterates over single bytes as bytes
> objects instead of integers.

More general function can be used instead.

def chunks(seq, size):
     for i in range(0, len(seq), size):
         yield seq[i: i + size]


 >>> list(chunks(b'abc', 1))
[b'a', b'b', b'c']



From cs at zip.com.au  Fri Mar 28 22:59:26 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 29 Mar 2014 08:59:26 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <53356810.5060602@canterbury.ac.nz>
References: <53356810.5060602@canterbury.ac.nz>
Message-ID: <20140328215926.GA60776@cskk.homeip.net>

On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
> >If I could consistently remember whether to include the "e" or not,
> >"zeroes" would be my preference as well, but almost every time I go to
> >type it, I have to pause...
> 
> Whichever you choose, please make sure it's the same as
> whatever numpy uses!

Please get this right. Zeroes. Two 'e's.

Otherwise I will feel pain every time I encounter zeros. Looks Greek.
-- 
Cameron Simpson <cs at zip.com.au>

Yes, we plot no less than the destruction of the West. Just the other day a
friend and I came up with the most pernicious academic scheme to date for
toppling the West: He will kneel behind the West on all fours. I will push it
backwards over him.     - Michael Berube

From ethan at stoneleaf.us  Fri Mar 28 23:33:55 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 28 Mar 2014 15:33:55 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328111347.20495c04@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org> <533587A5.9010701@stoneleaf.us>
 <20140328111347.20495c04@anarchist.wooz.org>
Message-ID: <5335F8D3.5090101@stoneleaf.us>

On 03/28/2014 08:13 AM, Barry Warsaw wrote:
> On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote:
>
>> You mean like http://bugs.python.org/issue20895 ?  :)
>
> Step *away* from the time machine.
>

Time machine?  What time machine?

*whistles innocently*

--
~Ethan~

From ncoghlan at gmail.com  Sat Mar 29 00:39:33 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 29 Mar 2014 09:39:33 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328111347.20495c04@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
 <533587A5.9010701@stoneleaf.us>
 <20140328111347.20495c04@anarchist.wooz.org>
Message-ID: <CADiSq7cXP6pw48Cyxbtmg8NiSwUHPxGKnmtKk_eh6dEMh4jYHg@mail.gmail.com>

On 29 March 2014 01:13, Barry Warsaw <barry at python.org> wrote:
> On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote:
>
>>You mean like http://bugs.python.org/issue20895 ?  :)
>
> Step *away* from the time machine.

Hah, I *thought* there was already an issue for this, but my search-fu
failed me :)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg at krypto.org  Sat Mar 29 08:52:42 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Mar 2014 00:52:42 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328215926.GA60776@cskk.homeip.net>
References: <53356810.5060602@canterbury.ac.nz>
 <20140328215926.GA60776@cskk.homeip.net>
Message-ID: <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com>

On Fri, Mar 28, 2014 at 2:59 PM, Cameron Simpson <cs at zip.com.au> wrote:

> On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > Nick Coghlan wrote:
> > >If I could consistently remember whether to include the "e" or not,
> > >"zeroes" would be my preference as well, but almost every time I go to
> > >type it, I have to pause...
> >
> > Whichever you choose, please make sure it's the same as
> > whatever numpy uses!
>
> Please get this right. Zeroes. Two 'e's.
>
> Otherwise I will feel pain every time I encounter zeros. Looks Greek.
>

zeros is way more popular than zeroes:

https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/965cb268/attachment.html>

From ncoghlan at gmail.com  Sat Mar 29 09:37:14 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 29 Mar 2014 18:37:14 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com>
References: <53356810.5060602@canterbury.ac.nz>
 <20140328215926.GA60776@cskk.homeip.net>
 <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com>
Message-ID: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com>

On 29 March 2014 17:52, Gregory P. Smith <greg at krypto.org> wrote:
> On Fri, Mar 28, 2014 at 2:59 PM, Cameron Simpson <cs at zip.com.au> wrote:
>> On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> > Nick Coghlan wrote:
>> > >If I could consistently remember whether to include the "e" or not,
>> > >"zeroes" would be my preference as well, but almost every time I go to
>> > >type it, I have to pause...
>> >
>> > Whichever you choose, please make sure it's the same as
>> > whatever numpy uses!
>>
>> Please get this right. Zeroes. Two 'e's.
>>
>> Otherwise I will feel pain every time I encounter zeros. Looks Greek.
>
> zeros is way more popular than zeroes:
>
> https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0

This subthread is confirming my original instinct to just avoid the
problem entirely because *either* choice is inevitably confusing for a
non-trivial subset of users :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ron3200 at gmail.com  Sat Mar 29 12:59:22 2014
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 29 Mar 2014 07:59:22 -0400
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com>
References: <53356810.5060602@canterbury.ac.nz>
 <20140328215926.GA60776@cskk.homeip.net>
 <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com>
 <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com>
Message-ID: <lh6cib$1h7$1@ger.gmane.org>



On 03/29/2014 04:37 AM, Nick Coghlan wrote:
>>> >>Please get this right. Zeroes. Two 'e's.
>>> >>
>>> >>Otherwise I will feel pain every time I encounter zeros. Looks Greek.
>> >
>> >zeros is way more popular than zeroes:
>> >
>> >https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0
> This subthread is confirming my original instinct to just avoid the
> problem entirely because*either*  choice is inevitably confusing for a
> non-trivial subset of users:)

Yes, and we don't have nones for lists and tuples, or spaces for strings.

;-)

Ron


From bcannon at gmail.com  Sat Mar 29 14:49:14 2014
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 29 Mar 2014 13:49:14 +0000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com>

On Fri Mar 28 2014 at 10:29:36 AM, Barry Warsaw <barry at python.org> wrote:

> Great idea Nick.  If I may dip my brush in some paint buckets.
>
> On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote:
>
> >However, during a conversation today, a possible solution occurred to
> >me: a "bytes.chr" class method, that served as an alternate
> >constructor. That idea results in the following 3 part proposal:
> >
> >1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP
> >361 defined "b'%c' % x"
>
> I agree with Steven that bytes.byte() is a better spelling.
>

If we are going to add a classmethod, then yes, otherwise just stick with
byte([x]).


>
> >2. Add "bytearray.allnull" and "bytes.allnull" such that
> >"bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int
> >handling
>
> I like bytearray.fill() for this.  The first argument would be the fill
> count,
> but it could take an optional second argument for the byte value to fill it
> with, which would of course default to zero.  E.g.
>
> >>> bytearray.fill(5)
> bytearray(b'\x00\x00\x00\x00\x00')
> >>> bytearray.fill(5, 97)
> bytearray(b'aaaaa')
>

I agree that if we are going to keep a classmethod to do this it should be
like str.zfill(). Otherwise just rely on the multiply operator.


>
> >3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
> >not only ambiguous, but actually a genuine bug magnet (it's way too
> >easy to accidentally pass a large integer and try to allocate a
> >ridiculously large bytes object)
>
> +1
>

+1


>
> >Does the above proposal sound like a reasonable suggestion for
> improvement in
> >3.5?
>
> Very much so.
>
> >Does this hit PEP territory, since it's changing the signature and API
> >of a builtin?
>
> I don't much care either way.  A PEP is not *wrong* (especially if we all
> start painting), but I think a tracker issue would be fine too.
>

PEP wouldn't hurt since it's a core type, but I'm not going to scream for
it if Guido just says "fine" at the language summit or something (should we
discuss this there?).


Overall I think I'm +1 on the "simplify the API" by dropping the
constructor issue and relying on the multiply operator instead of coming up
with a new method. Same with bytes([x]) if you need to convert a single int.

I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just
leaves the inability to index into bytes and get back a single as the
biggest annoyance in the bytes API.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/ba76b561/attachment.html>

From richard.prosser at mail.com  Sat Mar 29 16:29:53 2014
From: richard.prosser at mail.com (Richard Prosser)
Date: Sat, 29 Mar 2014 15:29:53 +0000
Subject: [Python-ideas] Browser for mailing lists
Message-ID: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>

I wanted to search this list (and other Python ones) but it is practically
impossible to do so.

So why not add a tool like http://markmail.org/search/?q=python to
https://mail.python.org/mailman/listinfo/python-ideas or the like? I got
this from http://openmeetings.apache.org/mail-lists.html


Cheers,

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/eb6cba7d/attachment.html>

From richard.prosser at mail.com  Sat Mar 29 16:36:06 2014
From: richard.prosser at mail.com (Richard Prosser)
Date: Sat, 29 Mar 2014 15:36:06 +0000
Subject: [Python-ideas] Nested dictionaries
Message-ID: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>

Not the first time I found myself recently needing to create
"multi-dimensional" dictionaries, so that I could write something like:

my_dict = ['a']['b']['b']['a'] = 'fred'

and once again I wonder why Python does not support that idiom directly, as
it must be a fairly common requirement. I know that there are good
solutions like the one at
http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by
using defaultdict but I feel that this really should be part of the
language.

So, should I propose a PEP?


Thanks ...

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/4e4b24a4/attachment.html>

From rosuav at gmail.com  Sat Mar 29 16:46:30 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 30 Mar 2014 02:46:30 +1100
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
Message-ID: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com>

On Sun, Mar 30, 2014 at 2:36 AM, Richard Prosser
<richard.prosser at mail.com> wrote:
> Not the first time I found myself recently needing to create
> "multi-dimensional" dictionaries, so that I could write something like:
>
> my_dict = ['a']['b']['b']['a'] = 'fred'
>
> and once again I wonder why Python does not support that idiom directly, as
> it must be a fairly common requirement. I know that there are good solutions
> like the one at
> http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by
> using defaultdict but I feel that this really should be part of the
> language.

The NestedDict at the end of that blog post looks like a nice clean
solution; and it's so short that it probably doesn't need to be in the
standard library. But maybe that would make a handy example, if
there's a place in the docs where you'd go looking for that sort of
thing.

Alternatively, you may find it simpler and easier to use a non-nested
dictionary. I store persistent data for my MUD client in a structure
like this:

persist["window/downarr"] = blah blah

Of course, this assumes your keys are simple strings, and it's best
for literals; you can't easily iterate over "everything under
window/", for instance. But if that covers your requirements, it's
probably going to be easier to read and comprehend; and it'd probably
be faster, too, although that shouldn't be a primary consideration.

ChrisA

From richard.prosser at mail.com  Sat Mar 29 16:51:20 2014
From: richard.prosser at mail.com (Richard Prosser)
Date: Sat, 29 Mar 2014 15:51:20 +0000
Subject: [Python-ideas] "Read Only" namespaces
Message-ID: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com>

This is probably far-fetched where Python is concerned but I still think
that it could be useful. It is based on an idea that I had 36 years ago!

Quite simply, the principle is to allow changes to variables within a
function body (for example) but not outside of it, rather like the
Functional Programming paradigm. That of course excludes the use of global
variables and requires functions to return whatever is needed to the outer
scope. In my experience, globals are only needed to implement persistent
(inter-call) states anyway, so assuming that such a feature is available in
the language I believe that the "Read Only" proposal is viable.

This is highly unlikely to be implemented in Python any time soon (even
with an Interpreter/compiler option) and would imply some performance
overhead but I still think that it is a nice idea.

Any comments? Would a "Persistent Variables" PEP be interesting in its own
right?


Thanks ...

Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/e045523b/attachment.html>

From flying-sheep at web.de  Sat Mar 29 17:43:31 2014
From: flying-sheep at web.de (Philipp A.)
Date: Sat, 29 Mar 2014 17:43:31 +0100
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
 <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com>
Message-ID: <CAN8d9g=PQE4xG4NjtWi_eGcbSsK9wQ9+2HfiPSpEreDEr=9Vgg@mail.gmail.com>

2014-03-29 16:46 GMT+01:00 Chris Angelico <rosuav at gmail.com>:

Alternatively, you may find it simpler and easier to use a non-nested
> dictionary. I store persistent data for my MUD client in a structure
> like this:
>
> persist["window/downarr"] = blah blah
>
> Of course, this assumes your keys are simple strings, and it's best
> for literals; you can't easily iterate over "everything under
> window/", for instance. But if that covers your requirements, it's
> probably going to be easier to read and comprehend; and it'd probably
> be faster, too, although that shouldn't be a primary consideration.
>
> ChrisA
>
another possibility: use tuples as keys.

persist['window', 'downarr'] = blah blah

this works with all hashable types.

PS: to elaborate for non-gurus: tuples in python are created by the comma,
not the braces, so the above is another way to write persist[ ('window',
'downarr') ]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/39b1ba4e/attachment.html>

From steve at pearwood.info  Sat Mar 29 18:29:12 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 30 Mar 2014 04:29:12 +1100
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
Message-ID: <20140329172912.GL16526@ando>

On Sat, Mar 29, 2014 at 03:36:06PM +0000, Richard Prosser wrote:
> Not the first time I found myself recently needing to create
> "multi-dimensional" dictionaries, so that I could write something like:
> 
> my_dict = ['a']['b']['b']['a'] = 'fred'

Could you please explain what that is supposed to mean? I can't tell if 
that's supposed to be new syntax, or a typo. Did you mean this instead?

mydict = {'a': {'b': {'b': {}}}}
my_dict['a']['b']['b']['a'] = 'fred'


> and once again I wonder why Python does not support that idiom directly,

It does. There are no special rules to learn, no special syntax to 
memorise, just the same dict syntax as you already know. You create a 
dict with {key: value}, and if the value itself is a dict, you just 
repeat the process with value --> {key: value}, as deeply as you need.

Not every special case needs special syntax.


> as it must be a fairly common requirement.

I doubt it. As the Zen of Python states, "Flat is better than nested". I 
think most people would try to avoid such deeply nested dicts if they 
can. I know I've never needed them.


> I know that there are good
> solutions like the one at
> http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by
> using defaultdict but I feel that this really should be part of the
> language.

I don't think something like this should be encouraged, and besides it's 
a four-line solution:

py> class MyDict(dict):
...     def __missing__(self, key):
...             t = self[key] = MyDict()
...             return t
...
py> x = MyDict()
py> x[1]['a'][None][True][23.0] = "Surprise!"
py> x
{1: {'a': {None: {True: {23.0: 'Surprise!'}}}}}


> So, should I propose a PEP?

No. Not every four-line utility class needs to be a built-in.


-- 
Steven

From ethan at stoneleaf.us  Sat Mar 29 17:49:42 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 29 Mar 2014 09:49:42 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
Message-ID: <5336F9A6.2030006@stoneleaf.us>

On 03/28/2014 03:27 AM, Nick Coghlan wrote:
>
> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
> not only ambiguous, but actually a genuine bug magnet (it's way too
> easy to accidentally pass a large integer and try to allocate a
> ridiculously large bytes object)

Why do 'bytes' and 'bytearray' have to have exactly the same API?  Having 'bytearray(10)' return a zero filled array is 
fine, as bytearrays are mutable; it just doesn't make sense for 'bytes' which are immutable.

--
~Ethan~

From ericsnowcurrently at gmail.com  Sat Mar 29 18:46:37 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 29 Mar 2014 11:46:37 -0600
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
Message-ID: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com>

On Sat, Mar 29, 2014 at 9:29 AM, Richard Prosser
<richard.prosser at mail.com> wrote:
> I wanted to search this list (and other Python ones) but it is practically
> impossible to do so.

Google lets you search a specific URL (and its subpages):

site:https://mail.python.org/mailman/listinfo/python-ideas ...

That's been the most effective tool I've used.

-eric

From steve at pearwood.info  Sat Mar 29 18:46:44 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 30 Mar 2014 04:46:44 +1100
Subject: [Python-ideas] "Read Only" namespaces
In-Reply-To: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com>
References: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com>
Message-ID: <20140329174644.GM16526@ando>

On Sat, Mar 29, 2014 at 03:51:20PM +0000, Richard Prosser wrote:
> This is probably far-fetched where Python is concerned but I still think
> that it could be useful. It is based on an idea that I had 36 years ago!
> 
> Quite simply, the principle is to allow changes to variables within a
> function body (for example) but not outside of it, rather like the
> Functional Programming paradigm.

I'm afraid your description leaves me completely in the dark.

(1) What do you mean, "changes to variables? Do you mean rebinding, or 
mutation, or both?

e.g. "x = 1" is a binding; "x.append(1)" is a mutation.

(2) What is the scope of these variables? If they are local 
variables, by definition they aren't visible outside of the function 
body, so they cannot be rebound or mutated by anything outside. If 
they're global variables, then by definition they're supposed to be 
available outside of any function for rebinding and/or mutation.

If they're meant to be *constants*, not variables, then you should call 
them constants.

(3) Under what circumstances would you use this feature?

(4) What you describe doesn't sound like anything from Functional 
Programming that I know of. What is the connection that I am missing?

(5) I'm not seeing the connection between your description and "read 
only namespaces", the subject of your post. Can you explain?

(6) Later you describe this as "Persistent Variables". Persistent across 
what?

[...] 
> Any comments? Would a "Persistent Variables" PEP be interesting in its own
> right?

PEP stands for Python Enhancement Proposal. From your own description, 
you don't seem to be proposing this as an enhancement to the language, 
more like just an interesting idea for discussion, so a PEP would be 
completely inappropriate.

Do you have a blog? Perhaps this might be better started as a blog post 
or three. You can iron out any kinks in the idea and then reconsider 
whether or not it makes a serious proposal.


-- 
Steven

From greg at krypto.org  Sat Mar 29 19:03:24 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Mar 2014 11:03:24 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <5336F9A6.2030006@stoneleaf.us>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
Message-ID: <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>

On Sat, Mar 29, 2014 at 9:49 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 03/28/2014 03:27 AM, Nick Coghlan wrote:
>
>>
>> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
>> not only ambiguous, but actually a genuine bug magnet (it's way too
>> easy to accidentally pass a large integer and try to allocate a
>> ridiculously large bytes object)
>>
>
> Why do 'bytes' and 'bytearray' have to have exactly the same API?  Having
> 'bytearray(10)' return a zero filled array is fine, as bytearrays are
> mutable; it just doesn't make sense for 'bytes' which are immutable.
>
>
I agree they didn't need to be. But that's already happened. Consistency
*is* convenient when reading and re-factoring code.

Regardless, the real issue is that it is easy to inadvertently pass an
integer to the bytes() constructor [and thus the bytearray() constructor]
when you really intended to pass a single element sequence.

+1 Deprecating the single integer argument to the constructor in 3.5
(DeprecationWarning) to be considered for removal in 3.6 or 3.7 sounds like
a good idea.

+1 At the same time, adding class methods: bytes*.byte*(97) and bytearray
*.byte*(0x65) sound great. Those are *always explicit* about the
programmer's intent rather than being an overloaded constructor that does
different things based on the type passed in which is more prone to bugs.

+1 that *bytearray*() does need a constructor that pre-allocates a bunch of
empty (zero) space as that is a common need for a mutable type.  That
should be .*zfill*(n) for consistency. Filling with other values is way
less common and doesn't deserve a .fill(n, value) method with potentially
ambiguous parameters (which one is the count and which one is the value
again? that'll be a continual question just as it is for C's memset and
similar functions).

-0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n)
class methods for the *bytes*() type. That is fine written as bytes.byte(0)
* n as it is expected to be an uncommon operation.  But if you want to add
it for consistency, fine by me, change the sign of my preference. :)

I don't think this is worthy of a PEP but won't object if you go that route.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/3aa42880/attachment.html>

From python at 2sn.net  Sat Mar 29 19:31:33 2014
From: python at 2sn.net (Alexander Heger)
Date: Sun, 30 Mar 2014 05:31:33 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
Message-ID: <CAN3CYHy006s7nLg2kKmoBJEdgo3b1jmz5iTneSAeVPxcrVad-g@mail.gmail.com>

>>>> bytearray.fill(5, 97)

why would you not want to write this as

bytearray(5).fill(97) ?

>>3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as
>>not only ambiguous, but actually a genuine bug magnet (it's way too
>>easy to accidentally pass a large integer and try to allocate a
>>ridiculously large bytes object)

I think these are just fine.

From ericsnowcurrently at gmail.com  Sat Mar 29 19:39:31 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 29 Mar 2014 12:39:31 -0600
Subject: [Python-ideas] Persisting private function state between calls
	(was: "Read Only" namespaces)
Message-ID: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com>

On Sat, Mar 29, 2014 at 9:51 AM, Richard Prosser
<richard.prosser at mail.com> wrote:
> Quite simply, the principle is to allow changes to variables within a
> function body (for example) but not outside of it, rather like the
> Functional Programming paradigm. That of course excludes the use of global
> variables and requires functions to return whatever is needed to the outer
> scope. In my experience, globals are only needed to implement persistent
> (inter-call) states anyway, so assuming that such a feature is available in
> the language I believe that the "Read Only" proposal is viable.

To restate, you want a way to have some variables in your function
body that persist between calls without using global variables.  There
are already several ways to do this, though none is particularly
obvious.

Let's use a simple example using a global variable followed by the
alternative solutions.

current = 0

def adder(x):
    global current
    current += x
    return current

1. Still use a global variable, but make it "private" (just use the
convention of a leading underscore on the name):

_current = 0

def adder(x):
    global _current
    _current += x
    return _current

2. Use the "default argument hack":

def adder(x, _ns={'current': 0}):
    _ns['current'] += x
    return _ns['current']

or

class IntWrapper:
    def __init__(self, value=0):
        self.value = value

def adder(x, _current=IntWrapper()):
    _current.value += x
    return _current.value

3. Use a closure (making use of the "nonlocal" keyword):

def get_adder(start=0):
    current = start  # Done for clarity.
    def adder(x):
        nonlocal current
        current += x
        return current
    return adder

adder = get_adder()

Each of the four approaches has its pros and cons.  The closure
approach is probably the clearest if you really want to avoid using a
global.  If there were agreement on a best approach, the least we
could do it make it clearer in the docs, assuming you looked and
didn't find anything helpful in the obvious places. :)

A more explicit mechanism could be nice, but would have to pass a
pretty high bar for inclusion into the language.  Here are some
approaches:

* A new keyword, e.g. "persistent", to accompany global and nonlocal.
* A special decorator that identifies persistent local variables in
the function.

So that you're aware, this idea came up a few years ago and involved a
long discussion that eventually lost steam (don't remember why)
[1][2][3].  There are also a couple existing PEPs [4][5] that would be
applicable (my favorite is the given statement).

-eric

[1] https://mail.python.org/pipermail/python-ideas/2011-September/011805.html
[2] https://mail.python.org/pipermail/python-ideas/2011-June/010479.html
[3] https://mail.python.org/pipermail/python-ideas/2011-September/011750.html
[4] "statement local namespaces" (given statement)
http://www.python.org/dev/peps/pep-3150/
[5] "General purpose decorator clause" http://www.python.org/dev/peps/pep-0403/

From ericsnowcurrently at gmail.com  Sat Mar 29 19:41:12 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 29 Mar 2014 12:41:12 -0600
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com>
Message-ID: <CALFfu7A2AoqbX4gYEF7Rjhai--PfBt1FGwy3i-=kjhfg02-hpQ@mail.gmail.com>

On Sat, Mar 29, 2014 at 11:46 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> site:https://mail.python.org/mailman/listinfo/python-ideas ...

Correction:

site:https://mail.python.org/pipermail/python-ideas/ ...

-eric

From masklinn at masklinn.net  Sat Mar 29 19:50:29 2014
From: masklinn at masklinn.net (Masklinn)
Date: Sat, 29 Mar 2014 19:50:29 +0100
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <20140329172912.GL16526@ando>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
 <20140329172912.GL16526@ando>
Message-ID: <CF52F35F-4BCF-4B08-B5D4-1B41E1FD496E@masklinn.net>

On 2014-03-29, at 18:29 , Steven D'Aprano <steve at pearwood.info> wrote:
>> I know that there are good
>> solutions like the one at
>> http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by
>> using defaultdict but I feel that this really should be part of the
>> language.
> 
> I don't think something like this should be encouraged, and besides it's 
> a four-line solution:
> 
> py> class MyDict(dict):
> ...     def __missing__(self, key):
> ...             t = self[key] = MyDict()
> ...             return t
> ...
> py> x = MyDict()
> py> x[1]['a'][None][True][23.0] = "Surprise!"
> py> x
> {1: {'a': {None: {True: {23.0: 'Surprise!'}}}}}
> 

In most case, a single line probably suffices:

    MyDict = lambda: collections.defaultdict(MyDict)

From steve at pearwood.info  Sat Mar 29 19:58:25 2014
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 30 Mar 2014 05:58:25 +1100
Subject: [Python-ideas] Persisting private function state between calls
	(was: "Read Only" namespaces)
In-Reply-To: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com>
References: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com>
Message-ID: <20140329185824.GN16526@ando>

On Sat, Mar 29, 2014 at 12:39:31PM -0600, Eric Snow wrote:
> On Sat, Mar 29, 2014 at 9:51 AM, Richard Prosser
> <richard.prosser at mail.com> wrote:
> > Quite simply, the principle is to allow changes to variables within a
> > function body (for example) but not outside of it, rather like the
> > Functional Programming paradigm. That of course excludes the use of global
> > variables and requires functions to return whatever is needed to the outer
> > scope. In my experience, globals are only needed to implement persistent
> > (inter-call) states anyway, so assuming that such a feature is available in
> > the language I believe that the "Read Only" proposal is viable.
> 
> To restate, you want a way to have some variables in your function
> body that persist between calls without using global variables. 

Ah-ha! That makes sense now. Thank you!


> There are already several ways to do this, though none is particularly
> obvious.

I disagree -- I think your example of an adder() function with a global 
is extremely obvious. It seems to be the usual solution that most 
beginners go for. It's not a *good* solution, but it is simple and 
obvious.

A much better solution if you need persistent state is to encapsulate 
the function and state into an object:

class Adder:
    def __init__(self):
        self.current = 0
    def add(self, x):
        self.current += x
        return self.current


Then you can have as many Adder instances as you want, without them all 
sharing state.

adder1 = Adder().add
adder2 = Adder().add



[...]
> A more explicit mechanism could be nice, but would have to pass a
> pretty high bar for inclusion into the language.  Here are some
> approaches:
> 
> * A new keyword, e.g. "persistent", to accompany global and nonlocal.
> * A special decorator that identifies persistent local variables in
> the function.

-1 to both of those. Not every idiom needs a keyword.



-- 
Steven

From ericsnowcurrently at gmail.com  Sat Mar 29 20:14:22 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 29 Mar 2014 13:14:22 -0600
Subject: [Python-ideas] Persisting private function state between calls
 (was: "Read Only" namespaces)
In-Reply-To: <20140329185824.GN16526@ando>
References: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com>
 <20140329185824.GN16526@ando>
Message-ID: <CALFfu7Ddk0Yi5R7-gp1Txa5dSu=UcmJ3+V7mP86_7esTti3fUQ@mail.gmail.com>

On Sat, Mar 29, 2014 at 12:58 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, Mar 29, 2014 at 12:39:31PM -0600, Eric Snow wrote:
>> There are already several ways to do this, though none is particularly
>> obvious.
>
> I disagree -- I think your example of an adder() function with a global
> is extremely obvious. It seems to be the usual solution that most
> beginners go for. It's not a *good* solution, but it is simple and
> obvious.

Agreed.  It was the example that mapped most closely to how the OP
described his problem. :)

[...]

>> A more explicit mechanism could be nice, but would have to pass a
>> pretty high bar for inclusion into the language.  Here are some
>> approaches:
>>
>> * A new keyword, e.g. "persistent", to accompany global and nonlocal.
>> * A special decorator that identifies persistent local variables in
>> the function.
>
> -1 to both of those. Not every idiom needs a keyword.

Oh, I agree*.  I should have made it clear. :)  I would advocate
instead for a more general solution such as that provided by statement
local namespaces (e.g. PEP 3150/403).

-eric

* I do think it may be worth exploring the idea of special decorators
that set code/function object flags that influence special-case
behaviors.  However, the performance implications of handling such
flags at call-time *may* render the idea impractical.

From masklinn at masklinn.net  Sat Mar 29 21:23:12 2014
From: masklinn at masklinn.net (Masklinn)
Date: Sat, 29 Mar 2014 21:23:12 +0100
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAJvrtnuHTW7sO8Kr1xy0kUyLGm3W5zWXZct0jPzZZMZYH9qA4Q@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
 <20140329172912.GL16526@ando>
 <CF52F35F-4BCF-4B08-B5D4-1B41E1FD496E@masklinn.net>
 <CAJvrtnuHTW7sO8Kr1xy0kUyLGm3W5zWXZct0jPzZZMZYH9qA4Q@mail.gmail.com>
Message-ID: <A6CCAE20-621E-494C-9D97-D58AB1D106BE@masklinn.net>

On 2014-03-29, at 20:43 , Richard Prosser <richard.prosser at mail.com> wrote:

> Maybe so but it is not straightforward and therefore IMO not "Pythonic".

I don't believe autovivification to be pythonic in the first place[0].
Python making you roll your own thus feels sensible.

> I really don't understand the objections that are being raised. What is wrong with being able to use a simple expression?
> 
> 
> Richard
> PS Thanks for the responses everyone but it would nice if someone agreed with my proposal!

[0] Steven already mentioned "flat is better than nested", but "explicit
    is better than implicit" also applies. To the surprise of no one,
    using autovivification is idiomatic perl.

From ben+python at benfinney.id.au  Sat Mar 29 21:54:38 2014
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 30 Mar 2014 07:54:38 +1100
Subject: [Python-ideas] Browser for mailing lists
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
Message-ID: <85d2h4ohdd.fsf@benfinney.id.au>

Richard Prosser <richard.prosser at mail.com> writes:

> I wanted to search this list (and other Python ones) but it is practically
> impossible to do so.

The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas>
has all its messages archived online for web browsing at
<URL:http://mail.python.org/pipermail/python-ideas/>.

So you can use any web search tool to search it. A good search engine
like DuckDuckGo will let you constrain the search to a particular subset
of the web:

    site:http://mail.python.org/pipermail/python-ideas foo bar baz

> So why not add a tool like http://markmail.org/search/?q=python to
> https://mail.python.org/mailman/listinfo/python-ideas or the like?

We have generally-appicable services that index the web and make much
better search tools. I don't often find a site's own custom search tool
to be sufficiently better to use it, when DuckDuckGo is available.

-- 
 \     ?For every complex problem, there is a solution that is simple, |
  `\                               neat, and wrong.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From python at 2sn.net  Sat Mar 29 21:59:13 2014
From: python at 2sn.net (Alexander Heger)
Date: Sun, 30 Mar 2014 07:59:13 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
 <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com>
Message-ID: <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com>

> I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just
> leaves the inability to index into bytes and get back a single as the
> biggest annoyance in the bytes API.

The issue remains that there is no single byte type, but what is
returned is bytes type over which, again, you can iterate.  This is
different from a tuple of integers, when you iterate over that, you
get integers not length-one tuples of integers.  The issue, I think,
originates from the string concept where iteration also return a
string not a character - which python does not have.  A logical
extension would be to have to have character and byte types that are
returned when iterating of strings and bytes.  This is like taking a
giraffe out of a zoo and still calling the result a (small) zoo.  Or
calling a can of tuna a grocery store.

From greg.ewing at canterbury.ac.nz  Sat Mar 29 22:06:29 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 30 Mar 2014 10:06:29 +1300
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <20140328215926.GA60776@cskk.homeip.net>
References: <53356810.5060602@canterbury.ac.nz>
 <20140328215926.GA60776@cskk.homeip.net>
Message-ID: <533735D5.1000901@canterbury.ac.nz>

Cameron Simpson wrote:
> Please get this right. Zeroes. Two 'e's.
> 
> Otherwise I will feel pain every time I encounter zeros. Looks Greek.

I sympathi{s,z}e, but I feel that having it inconsistent
with with numpy would cause more pain overall.

Ah, I know -- Python should treat identifiers ending in
"-os" and "-oes" as equivalent! That would solve all
problems of this kind once and for all.

"import oes"-ly y'rs,
Greg

From ncoghlan at gmail.com  Sat Mar 29 22:07:45 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 07:07:45 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
Message-ID: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>

On 30 March 2014 04:03, Gregory P. Smith <greg at krypto.org> wrote:
>
> +1 that bytearray() does need a constructor that pre-allocates a bunch of
> empty (zero) space as that is a common need for a mutable type.  That should
> be .zfill(n) for consistency. Filling with other values is way less common
> and doesn't deserve a .fill(n, value) method with potentially ambiguous
> parameters (which one is the count and which one is the value again? that'll
> be a continual question just as it is for C's memset and similar functions).

We can't use .zfill(), as that is already used for the same purposes
that it is used for with str and bytes objects (i.e. an ASCII
zero-fill).

I'm currently leaning towards the more explicit "from_len()" (with the
fill value being optional, and defaulting to zero).

> -0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n)
> class methods for the bytes() type. That is fine written as bytes.byte(0) *
> n as it is expected to be an uncommon operation.  But if you want to add it
> for consistency, fine by me, change the sign of my preference. :)
>
> I don't think this is worthy of a PEP but won't object if you go that route.

I already have a draft PEP written that covers the constructor issue,
iteration and adding acceptance of integer inputs to the remaining
methods that don't currently handle them. There was some background
explanation of the text/binary domain split in the Python 2->3
transition that I wanted Guido's feedback on before posting, but I
just realised I can cut that out for now, and then add it back after
Guido has had a chance to review it.

So I'll tidy that up and get the draft posted later today.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ethan at stoneleaf.us  Sat Mar 29 22:09:36 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 29 Mar 2014 14:09:36 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
 <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com>
 <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com>
Message-ID: <53373690.1060209@stoneleaf.us>

On 03/29/2014 01:59 PM, Alexander Heger wrote:
>>
>> I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just
>> leaves the inability to index into bytes and get back a single as the
>> biggest annoyance in the bytes API.
>
> The issue remains that there is no single byte type, but what is
> returned is bytes type over which, again, you can iterate.  This is
> different from a tuple of integers, when you iterate over that, you
> get integers not length-one tuples of integers.  The issue, I think,
> originates from the string concept where iteration also return a
> string not a character - which python does not have.  A logical
> extension would be to have to have character and byte types that are
> returned when iterating of strings and bytes.  This is like taking a
> giraffe out of a zoo and still calling the result a (small) zoo.  Or
> calling a can of tuna a grocery store.

Nice examples.  The problem is that one *can* say:

    b'a'

and get a 'bytes' object of length one, and then quite naturally one would then try something like

    b'abc'[0] == b'a'

and that will fail, even though it *looks* correct.

--
~Ethan~

From greg.ewing at canterbury.ac.nz  Sat Mar 29 22:31:35 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 30 Mar 2014 10:31:35 +1300
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com>
References: <53356810.5060602@canterbury.ac.nz>
 <20140328215926.GA60776@cskk.homeip.net>
 <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com>
 <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com>
Message-ID: <53373BB7.2050208@canterbury.ac.nz>

Nick Coghlan wrote:
> This subthread is confirming my original instinct to just avoid the
> problem entirely because *either* choice is inevitably confusing for a
> non-trivial subset of users :)

How about just calling it "zero"? There's really no need
for it to be plural.

After all, we refer to int(0) as just "zero", even though
there may be more than one zero bit in its representation.

I really don't like "allnull", it just sounds wrong, too
ascii-centric.

-- 
Greg

From ncoghlan at gmail.com  Sat Mar 29 22:34:44 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 07:34:44 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <53373690.1060209@stoneleaf.us>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <20140328102817.76d2b7f9@anarchist.wooz.org>
 <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com>
 <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com>
 <53373690.1060209@stoneleaf.us>
Message-ID: <CADiSq7fD7Ax8B4d=VKKPpiRu2r2Wa5yqk_qW9_fAkg9YxhR0jw@mail.gmail.com>

On 30 March 2014 07:09, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 03/29/2014 01:59 PM, Alexander Heger wrote:
>>>
>>>
>>> I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That
>>> just
>>> leaves the inability to index into bytes and get back a single as the
>>> biggest annoyance in the bytes API.
>>
>>
>> The issue remains that there is no single byte type, but what is
>> returned is bytes type over which, again, you can iterate.  This is
>> different from a tuple of integers, when you iterate over that, you
>> get integers not length-one tuples of integers.  The issue, I think,
>> originates from the string concept where iteration also return a
>> string not a character - which python does not have.  A logical
>> extension would be to have to have character and byte types that are
>> returned when iterating of strings and bytes.  This is like taking a
>> giraffe out of a zoo and still calling the result a (small) zoo.  Or
>> calling a can of tuna a grocery store.
>
>
> Nice examples.  The problem is that one *can* say:
>
>    b'a'
>
> and get a 'bytes' object of length one, and then quite naturally one would
> then try something like
>
>    b'abc'[0] == b'a'
>
> and that will fail, even though it *looks* correct.

FWIW, this is part of why Python 3 *didn't* originally have a bytes
literal - the affordances of the str-like literal syntax are wrong for
a tuple-of-integers type. We only added it back (pre 3.0) because
people wanted it for working with wire protocols that contain ASCII
segments (and I think that was a reasonable decision, even though it
has contributed directly to people several years later still expecting
them to behave more like Python 2 strings)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Sat Mar 29 22:37:43 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 30 Mar 2014 10:37:43 +1300
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
 <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com>
Message-ID: <53373D27.2020700@canterbury.ac.nz>

Chris Angelico wrote:
> Alternatively, you may find it simpler and easier to use a non-nested
> dictionary.
>
> persist["window/downarr"] = blah blah

Also keep in mind that you can use tuples as keys:

    mydict['a', 'b', 'b', 'a'] = fred

-- 
Greg

From ncoghlan at gmail.com  Sat Mar 29 23:05:45 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 08:05:45 +1000
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <85d2h4ohdd.fsf@benfinney.id.au>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au>
Message-ID: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>

On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote:
> Richard Prosser <richard.prosser at mail.com> writes:
>
> We have generally-appicable services that index the web and make much
> better search tools. I don't often find a site's own custom search tool
> to be sufficiently better to use it, when DuckDuckGo is available.

That said, migrating to Mailman3 + HyperKitty is definitely in the
longer term plans for the python.org mailing list infrastructure, and
that includes integrated search on the HyperKitty side of things.

While Mailman3 has been an ongoing project for quite some time (I
believe the remaining blockers mostly relate to handling migrations of
existing Mailman 2 installations), the HyperKitty work is mostly being
driven by some Fedora folks in order to upgrade Fedora's own
infrastructure. You can see the current state of the prototype here:
https://lists.stg.fedoraproject.org/archives/

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Sat Mar 29 23:08:55 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 29 Mar 2014 18:08:55 -0400
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <85d2h4ohdd.fsf@benfinney.id.au>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au>
Message-ID: <lh7ga5$c77$2@ger.gmane.org>

On 3/29/2014 4:54 PM, Ben Finney wrote:
> Richard Prosser <richard.prosser at mail.com> writes:
>
>> I wanted to search this list (and other Python ones) but it is practically
>> impossible to do so.
>
> The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas>
> has all its messages archived online for web browsing at
> <URL:http://mail.python.org/pipermail/python-ideas/>.
>
> So you can use any web search tool to search it. A good search engine
> like DuckDuckGo will let you constrain the search to a particular subset
> of the web:
>
>      site:http://mail.python.org/pipermail/python-ideas foo bar baz
>
>> So why not add a tool like http://markmail.org/search/?q=python to
>> https://mail.python.org/mailman/listinfo/python-ideas or the like?
>
> We have generally-appicable services that index the web and make much
> better search tools. I don't often find a site's own custom search tool
> to be sufficiently better to use it, when DuckDuckGo is available.

This list and many others are also available through, archived at, and 
searchable at news.gmane.org.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sat Mar 29 23:20:16 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 29 Mar 2014 18:20:16 -0400
Subject: [Python-ideas] Nested dictionaries
In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com>
Message-ID: <lh7gvf$le5$1@ger.gmane.org>

On 3/29/2014 11:36 AM, Richard Prosser wrote:
> I know that there are good solutions like the one at
> http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html

You can file a request on the tracker asking that this simple 4-line 
example be added to the doc.

 >>> class NestedDict(dict):
...     def __getitem__(self, key):
...         if key in self: return self.get(key)
...         return self.setdefault(key, NestedDict())

 > or by using defaultdict but I feel that this really
 > should be part of the language.

It is. You just have to copy and paste ;-).
Actually, I am being serious. The example is an application of 
dict.setdefault, which *is* in the language.

-- 
Terry Jan Reedy


From greg at krypto.org  Sat Mar 29 23:30:26 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Mar 2014 15:30:26 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
Message-ID: <CAGE7PNK9a9E3xs8KE6n7z0hihjakRwUabCXAGS+8YT2SowpUvA@mail.gmail.com>

On Sat, Mar 29, 2014 at 2:07 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 30 March 2014 04:03, Gregory P. Smith <greg at krypto.org> wrote:
> >
> > +1 that bytearray() does need a constructor that pre-allocates a bunch of
> > empty (zero) space as that is a common need for a mutable type.  That
> should
> > be .zfill(n) for consistency. Filling with other values is way less
> common
> > and doesn't deserve a .fill(n, value) method with potentially ambiguous
> > parameters (which one is the count and which one is the value again?
> that'll
> > be a continual question just as it is for C's memset and similar
> functions).
>
> We can't use .zfill(), as that is already used for the same purposes
> that it is used for with str and bytes objects (i.e. an ASCII
> zero-fill).
>
> I'm currently leaning towards the more explicit "from_len()" (with the
> fill value being optional, and defaulting to zero).
>

makes sense to me.


>
> > -0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n)
> > class methods for the bytes() type. That is fine written as
> bytes.byte(0) *
> > n as it is expected to be an uncommon operation.  But if you want to add
> it
> > for consistency, fine by me, change the sign of my preference. :)
> >
> > I don't think this is worthy of a PEP but won't object if you go that
> route.
>
> I already have a draft PEP written that covers the constructor issue,
> iteration and adding acceptance of integer inputs to the remaining
> methods that don't currently handle them. There was some background
> explanation of the text/binary domain split in the Python 2->3
> transition that I wanted Guido's feedback on before posting, but I
> just realised I can cut that out for now, and then add it back after
> Guido has had a chance to review it.
>
> So I'll tidy that up and get the draft posted later today.
>

Thanks Nick!


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/6c8a9854/attachment-0001.html>

From cs at zip.com.au  Sat Mar 29 23:47:16 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 30 Mar 2014 09:47:16 +1100
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <53373BB7.2050208@canterbury.ac.nz>
References: <53373BB7.2050208@canterbury.ac.nz>
Message-ID: <20140329224716.GA61813@cskk.homeip.net>

On 30Mar2014 10:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
> >This subthread is confirming my original instinct to just avoid the
> >problem entirely because *either* choice is inevitably confusing for a
> >non-trivial subset of users :)
> 
> How about just calling it "zero"? There's really no need
> for it to be plural.
> 
> After all, we refer to int(0) as just "zero", even though
> there may be more than one zero bit in its representation.

Unfortunately, "zero" reads like a verb. One might zero an array, for example.
As a noun it would imply exactly one.

It's a good idea though; I guess I'm -0 purely for verb/noun reasons.

If numpy already has zeros() with a very similar meaning I would
live with the misspelling for the sake of consistency. If the numpy
zeros does something different then it would carry no weight with
me.

Hoping for something more betterer,
-- 
Cameron Simpson <cs at zip.com.au>

For those who understand, NO explanation is needed,
for those who don't understand, NO explanation will be given!
        - Davey D <decoster at vnet.ibm.com>

From ncoghlan at gmail.com  Sun Mar 30 04:17:08 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 12:17:08 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
Message-ID: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>

On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I already have a draft PEP written that covers the constructor issue,
> iteration and adding acceptance of integer inputs to the remaining
> methods that don't currently handle them. There was some background
> explanation of the text/binary domain split in the Python 2->3
> transition that I wanted Guido's feedback on before posting, but I
> just realised I can cut that out for now, and then add it back after
> Guido has had a chance to review it.
>
> So I'll tidy that up and get the draft posted later today.

Guido pointed out most of the stuff I had asked him to look at wasn't
actually relevant to the PEP, so I just cut most of it entirely.
Suffice to say, after stepping back and reviewing them systematically
for the first time in years, I believe the APIs for the core binary
data types in Python 3 could do with a little sprucing up :)

Web version: http://www.python.org/dev/peps/pep-0467/

======================================
PEP: 467
Title: Improved API consistency for bytes and bytearray
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-03-30
Python-Version: 3.5
Post-History: 2014-03-30


Abstract
========

During the initial development of the Python 3 language specification, the
core ``bytes`` type for arbitrary binary data started as the mutable type
that is now referred to as ``bytearray``. Other aspects of operating in
the binary domain in Python have also evolved over the course of the Python
3 series.

This PEP proposes a number of small adjustments to the APIs of the ``bytes``
and ``bytearray`` types to make their behaviour more internally consistent
and to make it easier to operate entirely in the binary domain for use cases
that actually involve manipulating binary data directly, rather than
converting it to a more structured form with additional modelling
semantics (such as ``str``) and then converting back to binary format after
processing.


Background
==========

Over the course of Python 3's evolution, a number of adjustments have been
made to the core ``bytes`` and ``bytearray`` types as additional practical
experience was gained with using them in code beyond the Python 3 standard
library and test suite. However, to date, these changes have been made
on a relatively ad hoc tactical basis as specific issues were identified,
rather than as part of a systematic review of the APIs of these types. This
approach has allowed inconsistencies to creep into the API design as to which
input types are accepted by different methods. Additional inconsistencies
linger from an earlier pre-release design where there was *no* separate
``bytearray`` type, and instead the core ``bytes`` type was mutable (with
no immutable counterpart), as well as from the origins of these types in
the text-like behaviour of the Python 2 ``str`` type.

This PEP aims to provide the missing systematic review, with the goal of
ensuring that wherever feasible (given backwards compatibility constraints)
these current inconsistencies are addressed for the Python 3.5 release.


Proposals
=========

As a "consistency improvement" proposal, this PEP is actually about a number
of smaller micro-proposals, each aimed at improving the self-consistency of
the binary data model in Python 3. Proposals are motivated by one of three
factors:

* removing remnants of the original design of ``bytes`` as a mutable type
* more consistently accepting length 1 ``bytes`` objects as input where an
  integer between ``0`` and ``255`` inclusive is expected, and vice-versa
* allowing users to easily convert integer output to a length 1 ``bytes``
  object


Alternate Constructors
----------------------

The ``bytes`` and ``bytearray`` constructors currently accept an integer
argument, but interpret it to mean a zero-filled object of the given length.
This is a legacy of the original design of ``bytes`` as a mutable type,
rather than a particularly intuitive behaviour for users. It has become
especially confusing now that other ``bytes`` interfaces treat integers
and the corresponding length 1 bytes instances as equivalent input.
Compare::

    >>> b"\x03" in bytes([1, 2, 3])
    True
    >>> 3 in bytes([1, 2, 3])
    True

    >>> bytes(b"\x03")
    b'\x03'
    >>> bytes(3)
    b'\x00\x00\x00'

This PEP proposes that the current handling of integers in the bytes and
bytearray constructors by deprecated in Python 3.5 and removed in Python
3.6, being replaced by two more type appropriate alternate constructors
provided as class methods. The initial python-ideas thread [ideas-thread1]_
that spawned this PEP was specifically aimed at deprecating this constructor
behaviour.

For ``bytes``, a ``byte`` constructor is proposed that converts integers
(as indicated by ``operator.index``) in the appropriate range to a ``bytes``
object, converts objects that support the buffer API to bytes, and also
passes through length 1 byte strings unchanged::

    >>> bytes.byte(3)
    b'\x03'
    >>> bytes.byte(bytearray(bytes([3])))
    b'\x03'
    >>> bytes.byte(memoryview(bytes([3])))
    b'\x03'
    >>> bytes.byte(bytes([3]))
    b'\x03'
    >>> bytes.byte(512)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: bytes must be in range(0, 256)
    >>> bytes.byte(b"ab")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: bytes.byte() expected a byte, but buffer of length 2 found

One specific use case for this alternate constructor is to easily convert
the result of indexing operations on ``bytes`` and other binary sequences
from an integer to a ``bytes`` object. The documentation for this API
should note that its counterpart for the reverse conversion is ``ord()``.

For ``bytearray``, a ``from_len`` constructor is proposed that preallocates
the buffer filled with a particular value (default to ``0``) as a direct
replacement for the current constructor behaviour, rather than having to use
sequence repetition to achieve the same effect in a less intuitive way::

    >>> bytearray.from_len(3)
    bytearray(b'\x00\x00\x00')
    >>> bytearray.from_len(3, 6)
    bytearray(b'\x06\x06\x06')

This part of the proposal was covered by an existing issue
[empty-buffer-issue]_ and a variety of names have been proposed
(``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The
specific name currently proposed was chosen by analogy with
``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely
explicit that it is an alternate constructor rather than an in-place
mutation, as well as how it differs from the standard constructor.


Open questions
^^^^^^^^^^^^^^

* Should ``bytearray.byte()`` also be added? Or is
  ``bytearray(bytes.byte(x))`` sufficient for that case?
* Should ``bytes.from_len()`` also be added? Or is sequence repetition
  sufficient for that case?
* Should ``bytearray.from_len()`` use a different name?
* Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary
  sequences with more than one element? The ``TypeError`` currently proposed
  is copied (with slightly improved wording) from the behaviour of ``ord()``
  with sequences containing more than one code point, while ``ValueError``
  would be more consistent with the existing handling of out-of-range
  integer values.
* ``bytes.byte()`` is defined above as accepting length 1 binary sequences
  as individual bytes, but this is currently inconsistent with the main
  ``bytes`` constructor::

      >>> bytes([b"a", b"b", b"c"])
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      TypeError: 'bytes' object cannot be interpreted as an integer

  Should the ``bytes`` constructor be changed to accept iterables of length 1
  bytes objects in addition to iterables of integers? If so, should it
  allow a mixture of the two in a single iterable?


Iteration
---------

Iteration over ``bytes`` objects and other binary sequences produces
integers. Rather than proposing a new method that would need to be added
not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially
to third party types as well, this PEP proposes that iteration to produce
length 1 ``bytes`` objects instead be handled by combining ``map`` with
the new ``bytes.byte()`` alternate constructor proposed above::

    for x in map(bytes.byte, data):
        # x is a length 1 ``bytes`` object, rather than an integer
        # This works with *any* container of integers in the range
        # 0 to 255 inclusive


Consistent support for different input types
--------------------------------------------

In Python 3.3, the binary search operations (``in``, ``count()``,
``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to
accept integers in the range 0 to 255 (inclusive) as their first argument
(in addition to the existing support for binary sequences).

This PEP proposes extending that behaviour of accepting integers as being
equivalent to the corresponding length 1 binary sequence to several other
``bytes`` and ``bytearray`` methods that currently expect a ``bytes``
object for certain parameters. In essence, if a value is an acceptable
input to the new ``bytes.byte`` constructor defined above, then it would
be acceptable in the roles defined here (in addition to any other already
supported inputs):

* ``startswith()`` prefix(es)
* ``endswith()`` suffix(es)

* ``center()`` fill character
* ``ljust()`` fill character
* ``rjust()`` fill character

* ``strip()`` character to strip
* ``lstrip()`` character to strip
* ``rstrip()`` character to strip

* ``partition()`` separator argument
* ``rpartition()`` separator argument

* ``split()`` separator argument
* ``rsplit()`` separator argument

* ``replace()`` old value and new value

In addition to the consistency motive, this approach also makes it easier
to work with the indexing behaviour , as the result of an indexing operation
can more easily be fed back in to other methods.

For ``bytearray``, some additional changes are proposed to the current
integer based operations to ensure they remain consistent with the proposed
constructor changes::

* ``append()``: updated to be consistent with ``bytes.byte()``
* ``remove()``: updated to be consistent with ``bytes.byte()``
* ``+=``: updated to be consistent with ``bytes()`` changes (if any)
* ``extend()``: updated to be consistent with ``bytes()`` changes (if any)


Acknowledgement of surprising behaviour of some ``bytearray`` methods
---------------------------------------------------------------------

Several of the ``bytes`` and ``bytearray`` methods have their origins in the
Python 2 ``str`` API. As ``str`` is an immutable type, all of these
operations are defined as returning a *new* instance, rather than operating
in place. This contrasts with methods on other mutable types like ``list``,
where ``list.sort()`` and ``list.reverse()`` operate in-place and return
``None``, rather than creating a new object.

Backwards compatibility constraints make it impractical to change this
behaviour at this point, but it may be appropriate to explicitly call out
this quirk in the documentation for the ``bytearray`` type. It affects the
following methods that could reasonably be expected to operate in-place on
a mutable type:

* ``center()``
* ``ljust()``
* ``rjust()``
* ``strip()``
* ``lstrip()``
* ``rstrip()``
* ``replace()``
* ``lower()``
* ``upper()``
* ``swapcase()``
* ``title()``
* ``capitalize()``
* ``translate()``
* ``expandtabs()``
* ``zfill()``

Note that the following ``bytearray`` operations *do* operate in place, as
they're part of the mutable sequence API in ``bytearray``, rather than being
inspired by the immutable Python 2 ``str`` API:

* ``+=``
* ``append()``
* ``extend()``
* ``reverse()``
* ``remove()``
* ``pop()``


References
==========

.. [ideas-thread1]
https://mail.python.org/pipermail/python-ideas/2014-March/027295.html
.. [empty-buffer-issue] http://bugs.python.org/issue20895


Copyright
=========

This document has been placed in the public domain.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Sun Mar 30 06:02:33 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 29 Mar 2014 21:02:33 -0700 (PDT)
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <lh7ga5$c77$2@ger.gmane.org>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
Message-ID: <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>

From: Terry Reedy <tjreedy at udel.edu>

Sent: Saturday, March 29, 2014 3:08 PM


> On 3/29/2014 4:54 PM, Ben Finney wrote:
>>  Richard Prosser <richard.prosser at mail.com> writes:
>> 
>>>  I wanted to search this list (and other Python ones) but it is 
> practically
>>>  impossible to do so.
>> 
>>  The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas>
>>  has all its messages archived online for web browsing at
>>  <URL:http://mail.python.org/pipermail/python-ideas/>.
>> 
>>  So you can use any web search tool to search it. A good search engine
>>  like DuckDuckGo will let you constrain the search to a particular subset
>>  of the web:
>> 
>> ? ? ? site:http://mail.python.org/pipermail/python-ideas foo bar baz
>> 
>>>  So why not add a tool like http://markmail.org/search/?q=python to
>>>  https://mail.python.org/mailman/listinfo/python-ideas or the like?
>> 
>>  We have generally-appicable services that index the web and make much
>>  better search tools. I don't often find a site's own custom search 
> tool
>>  to be sufficiently better to use it, when DuckDuckGo is available.
> 
> This list and many others are also available through, archived at, and 
> searchable at news.gmane.org.


How hard would it be to add appropriate links?(or, better, custom search boxes, or, worse but still helpful, a paragraph explaining how to search each of those manually) to the mailing list archive pages?

For example, if you link to this:

http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas


? then people just need to fill in their search terms on the Advanced Search page that comes up.

From greg at krypto.org  Sun Mar 30 08:10:53 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Mar 2014 23:10:53 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
Message-ID: <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com>

On Sat, Mar 29, 2014 at 7:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > I already have a draft PEP written that covers the constructor issue,
> > iteration and adding acceptance of integer inputs to the remaining
> > methods that don't currently handle them. There was some background
> > explanation of the text/binary domain split in the Python 2->3
> > transition that I wanted Guido's feedback on before posting, but I
> > just realised I can cut that out for now, and then add it back after
> > Guido has had a chance to review it.
> >
> > So I'll tidy that up and get the draft posted later today.
>
> Guido pointed out most of the stuff I had asked him to look at wasn't
> actually relevant to the PEP, so I just cut most of it entirely.
> Suffice to say, after stepping back and reviewing them systematically
> for the first time in years, I believe the APIs for the core binary
> data types in Python 3 could do with a little sprucing up :)
>
> Web version: http://www.python.org/dev/peps/pep-0467/
>
> ======================================
> PEP: 467
> Title: Improved API consistency for bytes and bytearray
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nick Coghlan <ncoghlan at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 2014-03-30
> Python-Version: 3.5
> Post-History: 2014-03-30
>
>
> Abstract
> ========
>
> During the initial development of the Python 3 language specification, the
> core ``bytes`` type for arbitrary binary data started as the mutable type
> that is now referred to as ``bytearray``. Other aspects of operating in
> the binary domain in Python have also evolved over the course of the Python
> 3 series.
>
> This PEP proposes a number of small adjustments to the APIs of the
> ``bytes``
> and ``bytearray`` types to make their behaviour more internally consistent
> and to make it easier to operate entirely in the binary domain for use
> cases
> that actually involve manipulating binary data directly, rather than
> converting it to a more structured form with additional modelling
> semantics (such as ``str``) and then converting back to binary format after
> processing.
>
>
> Background
> ==========
>
> Over the course of Python 3's evolution, a number of adjustments have been
> made to the core ``bytes`` and ``bytearray`` types as additional practical
> experience was gained with using them in code beyond the Python 3 standard
> library and test suite. However, to date, these changes have been made
> on a relatively ad hoc tactical basis as specific issues were identified,
> rather than as part of a systematic review of the APIs of these types. This
> approach has allowed inconsistencies to creep into the API design as to
> which
> input types are accepted by different methods. Additional inconsistencies
> linger from an earlier pre-release design where there was *no* separate
> ``bytearray`` type, and instead the core ``bytes`` type was mutable (with
> no immutable counterpart), as well as from the origins of these types in
> the text-like behaviour of the Python 2 ``str`` type.
>
> This PEP aims to provide the missing systematic review, with the goal of
> ensuring that wherever feasible (given backwards compatibility constraints)
> these current inconsistencies are addressed for the Python 3.5 release.
>
>
> Proposals
> =========
>
> As a "consistency improvement" proposal, this PEP is actually about a
> number
> of smaller micro-proposals, each aimed at improving the self-consistency of
> the binary data model in Python 3. Proposals are motivated by one of three
> factors:
>
> * removing remnants of the original design of ``bytes`` as a mutable type
> * more consistently accepting length 1 ``bytes`` objects as input where an
>   integer between ``0`` and ``255`` inclusive is expected, and vice-versa
> * allowing users to easily convert integer output to a length 1 ``bytes``
>   object
>
>
> Alternate Constructors
> ----------------------
>
> The ``bytes`` and ``bytearray`` constructors currently accept an integer
> argument, but interpret it to mean a zero-filled object of the given
> length.
> This is a legacy of the original design of ``bytes`` as a mutable type,
> rather than a particularly intuitive behaviour for users. It has become
> especially confusing now that other ``bytes`` interfaces treat integers
> and the corresponding length 1 bytes instances as equivalent input.
> Compare::
>
>     >>> b"\x03" in bytes([1, 2, 3])
>     True
>     >>> 3 in bytes([1, 2, 3])
>     True
>
>     >>> bytes(b"\x03")
>     b'\x03'
>     >>> bytes(3)
>     b'\x00\x00\x00'
>
> This PEP proposes that the current handling of integers in the bytes and
> bytearray constructors by deprecated in Python 3.5 and removed in Python
> 3.6, being replaced by two more type appropriate alternate constructors
> provided as class methods. The initial python-ideas thread [ideas-thread1]_
> that spawned this PEP was specifically aimed at deprecating this
> constructor
> behaviour.
>
> For ``bytes``, a ``byte`` constructor is proposed that converts integers
> (as indicated by ``operator.index``) in the appropriate range to a
> ``bytes``
> object, converts objects that support the buffer API to bytes, and also
> passes through length 1 byte strings unchanged::
>
>     >>> bytes.byte(3)
>     b'\x03'
>     >>> bytes.byte(bytearray(bytes([3])))
>     b'\x03'
>     >>> bytes.byte(memoryview(bytes([3])))
>     b'\x03'
>     >>> bytes.byte(bytes([3]))
>     b'\x03'
>     >>> bytes.byte(512)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     ValueError: bytes must be in range(0, 256)
>     >>> bytes.byte(b"ab")
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: bytes.byte() expected a byte, but buffer of length 2 found
>
> One specific use case for this alternate constructor is to easily convert
> the result of indexing operations on ``bytes`` and other binary sequences
> from an integer to a ``bytes`` object. The documentation for this API
> should note that its counterpart for the reverse conversion is ``ord()``.
>
> For ``bytearray``, a ``from_len`` constructor is proposed that preallocates
> the buffer filled with a particular value (default to ``0``) as a direct
> replacement for the current constructor behaviour, rather than having to
> use
> sequence repetition to achieve the same effect in a less intuitive way::
>
>     >>> bytearray.from_len(3)
>     bytearray(b'\x00\x00\x00')
>     >>> bytearray.from_len(3, 6)
>     bytearray(b'\x06\x06\x06')
>
> This part of the proposal was covered by an existing issue
> [empty-buffer-issue]_ and a variety of names have been proposed
> (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The
> specific name currently proposed was chosen by analogy with
> ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely
> explicit that it is an alternate constructor rather than an in-place
> mutation, as well as how it differs from the standard constructor.
>
>
> Open questions
> ^^^^^^^^^^^^^^
>
> * Should ``bytearray.byte()`` also be added? Or is
>   ``bytearray(bytes.byte(x))`` sufficient for that case?
> * Should ``bytes.from_len()`` also be added? Or is sequence repetition
>   sufficient for that case?
>

I prefer keeping them consistent across the types myself.

* Should ``bytearray.from_len()`` use a different name?
>

This name works for me.


> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary
>   sequences with more than one element? The ``TypeError`` currently
> proposed
>   is copied (with slightly improved wording) from the behaviour of
> ``ord()``
>   with sequences containing more than one code point, while ``ValueError``
>   would be more consistent with the existing handling of out-of-range
>   integer values.
> * ``bytes.byte()`` is defined above as accepting length 1 binary sequences
>   as individual bytes, but this is currently inconsistent with the main
>   ``bytes`` constructor::
>

I don't like that bytes.byte() would accept anything other than an int. It
should not accept length 1 binary sequences at all.  I'd prefer to see
bytes.byte(b"X") raise a TypeError.


>       >>> bytes([b"a", b"b", b"c"])
>       Traceback (most recent call last):
>         File "<stdin>", line 1, in <module>
>       TypeError: 'bytes' object cannot be interpreted as an integer
>
>   Should the ``bytes`` constructor be changed to accept iterables of
> length 1
>   bytes objects in addition to iterables of integers? If so, should it
>   allow a mixture of the two in a single iterable?
>
>
> Iteration
> ---------
>
> Iteration over ``bytes`` objects and other binary sequences produces
> integers. Rather than proposing a new method that would need to be added
> not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially
> to third party types as well, this PEP proposes that iteration to produce
> length 1 ``bytes`` objects instead be handled by combining ``map`` with
> the new ``bytes.byte()`` alternate constructor proposed above::
>
>     for x in map(bytes.byte, data):
>         # x is a length 1 ``bytes`` object, rather than an integer
>         # This works with *any* container of integers in the range
>         # 0 to 255 inclusive
>
>
> Consistent support for different input types
> --------------------------------------------
>
> In Python 3.3, the binary search operations (``in``, ``count()``,
> ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to
> accept integers in the range 0 to 255 (inclusive) as their first argument
> (in addition to the existing support for binary sequences).
>
> This PEP proposes extending that behaviour of accepting integers as being
> equivalent to the corresponding length 1 binary sequence to several other
> ``bytes`` and ``bytearray`` methods that currently expect a ``bytes``
> object for certain parameters. In essence, if a value is an acceptable
> input to the new ``bytes.byte`` constructor defined above, then it would
> be acceptable in the roles defined here (in addition to any other already
> supported inputs):
>
> * ``startswith()`` prefix(es)
> * ``endswith()`` suffix(es)
>
> * ``center()`` fill character
> * ``ljust()`` fill character
> * ``rjust()`` fill character
>
> * ``strip()`` character to strip
> * ``lstrip()`` character to strip
> * ``rstrip()`` character to strip
>
> * ``partition()`` separator argument
> * ``rpartition()`` separator argument
>
> * ``split()`` separator argument
> * ``rsplit()`` separator argument
>
> * ``replace()`` old value and new value
>
> In addition to the consistency motive, this approach also makes it easier
> to work with the indexing behaviour , as the result of an indexing
> operation
> can more easily be fed back in to other methods.
>
> For ``bytearray``, some additional changes are proposed to the current
> integer based operations to ensure they remain consistent with the proposed
> constructor changes::
>
> * ``append()``: updated to be consistent with ``bytes.byte()``
> * ``remove()``: updated to be consistent with ``bytes.byte()``
> * ``+=``: updated to be consistent with ``bytes()`` changes (if any)
>

Where was a change to += behavior mentioned? I don't see that above (or did
I miss something?).


> * ``extend()``: updated to be consistent with ``bytes()`` changes (if any)
>
>
> Acknowledgement of surprising behaviour of some ``bytearray`` methods
> ---------------------------------------------------------------------
>
> Several of the ``bytes`` and ``bytearray`` methods have their origins in
> the
> Python 2 ``str`` API. As ``str`` is an immutable type, all of these
> operations are defined as returning a *new* instance, rather than operating
> in place. This contrasts with methods on other mutable types like ``list``,
> where ``list.sort()`` and ``list.reverse()`` operate in-place and return
> ``None``, rather than creating a new object.
>
> Backwards compatibility constraints make it impractical to change this
> behaviour at this point, but it may be appropriate to explicitly call out
> this quirk in the documentation for the ``bytearray`` type. It affects the
> following methods that could reasonably be expected to operate in-place on
> a mutable type:
>
> * ``center()``
> * ``ljust()``
> * ``rjust()``
> * ``strip()``
> * ``lstrip()``
> * ``rstrip()``
> * ``replace()``
> * ``lower()``
> * ``upper()``
> * ``swapcase()``
> * ``title()``
> * ``capitalize()``
> * ``translate()``
> * ``expandtabs()``
> * ``zfill()``
>
> Note that the following ``bytearray`` operations *do* operate in place, as
> they're part of the mutable sequence API in ``bytearray``, rather than
> being
> inspired by the immutable Python 2 ``str`` API:
>
> * ``+=``
> * ``append()``
> * ``extend()``
> * ``reverse()``
> * ``remove()``
> * ``pop()``
>
>
> References
> ==========
>
> .. [ideas-thread1]
> https://mail.python.org/pipermail/python-ideas/2014-March/027295.html
> .. [empty-buffer-issue] http://bugs.python.org/issue20895
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/4575b9d0/attachment-0001.html>

From ncoghlan at gmail.com  Sun Mar 30 08:31:37 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 16:31:37 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
 <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com>
Message-ID: <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com>

On 30 March 2014 16:10, Gregory P. Smith <greg at krypto.org> wrote:
>> Open questions
>> ^^^^^^^^^^^^^^
>>
>> * Should ``bytearray.byte()`` also be added? Or is
>>   ``bytearray(bytes.byte(x))`` sufficient for that case?
>> * Should ``bytes.from_len()`` also be added? Or is sequence repetition
>>   sufficient for that case?
>
> I prefer keeping them consistent across the types myself.
>
>> * Should ``bytearray.from_len()`` use a different name?
>
> This name works for me.
>
>>
>> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary
>>   sequences with more than one element? The ``TypeError`` currently
>> proposed
>>   is copied (with slightly improved wording) from the behaviour of
>> ``ord()``
>>   with sequences containing more than one code point, while ``ValueError``
>>   would be more consistent with the existing handling of out-of-range
>>   integer values.
>> * ``bytes.byte()`` is defined above as accepting length 1 binary sequences
>>   as individual bytes, but this is currently inconsistent with the main
>>   ``bytes`` constructor::
>
>
> I don't like that bytes.byte() would accept anything other than an int. It
> should not accept length 1 binary sequences at all.  I'd prefer to see
> bytes.byte(b"X") raise a TypeError.

Unfortunately, it's not that simple, because accepting both is the
only way I see of rendering the current APIs coherent. The problem is
that the str-derived APIs expect bytes objects, the bytearray mutating
methods expect integers, and in Python 3.3, the substring search APIs
were updated to accept both. This means we currently have:

>>> data = bytes([1, 2, 3, 4])
>>> 3 in data
True
>>> b"\x03" in data
True
>>> data.count(3)
1
>>> data.count(b"\x03")
1
>>> data.replace(3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected bytes, bytearray or buffer compatible object
>>> data.replace(b"\x03", b"\x04")
b'\x01\x02\x04\x04'
>>> mutable = bytearray(data)
>>> mutable
bytearray(b'\x01\x02\x03\x04')
>>> mutable.append(b"\x05")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: an integer is required
>>> mutable.append(5)
>>> mutable
bytearray(b'\x01\x02\x03\x04\x05')

Since some APIs work one way, some work the other, the only backwards
compatible path I see to consistency is to always treat a length 1
byte string as an acceptable input for the APIs that currently accept
an integer and vice-versa.

That said, I think this hybrid nature accurately reflects the fact
that indexing and slicing bytes objects in Python 3 return different
types - the individual elements are integers, but the subsequences are
bytes objects, and several of these APIs are either
"element-or-subsequence" APIs (in which case they should accept both),
or else they *should* have been element APIs, but currently expect a
subsequence due to their Python 2 str heritage.

If we had the opportunity to redesign these APIs from scratch, we'd
likely make a much clearer distinction between element based APIs
(that would use integers) and subsequence APIs (that would accept
buffer implementing objects). As it is, I think the situation is
inherently ambiguous, and providing hybrid APIs to help deal with that
ambiguity is our best available option.

>> For ``bytearray``, some additional changes are proposed to the current
>> integer based operations to ensure they remain consistent with the
>> proposed
>> constructor changes::
>>
>> * ``append()``: updated to be consistent with ``bytes.byte()``
>> * ``remove()``: updated to be consistent with ``bytes.byte()``
>> * ``+=``: updated to be consistent with ``bytes()`` changes (if any)
>
>
> Where was a change to += behavior mentioned? I don't see that above (or did
> I miss something?).

It was an open question against the constructors - if bytes.byte() is
defined as the PEP suggests, then the case can be made that the
iterables accepted by the bytes() constructor should also be made more
permissive in terms of the contents of the iterables it accepts. If
*that* happens, then extending an existing bytearray should also
become more permissive.

Note that I'm not sold on actually changing that - that's why it's an
open question, rather than something the PEP is currently proposing.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg at krypto.org  Sun Mar 30 09:03:32 2014
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Mar 2014 00:03:32 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
 <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com>
 <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com>
Message-ID: <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com>

On Sat, Mar 29, 2014 at 11:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 30 March 2014 16:10, Gregory P. Smith <greg at krypto.org> wrote:
> >> Open questions
> >> ^^^^^^^^^^^^^^
> >>
> >> * Should ``bytearray.byte()`` also be added? Or is
> >>   ``bytearray(bytes.byte(x))`` sufficient for that case?
> >> * Should ``bytes.from_len()`` also be added? Or is sequence repetition
> >>   sufficient for that case?
> >
> > I prefer keeping them consistent across the types myself.
> >
> >> * Should ``bytearray.from_len()`` use a different name?
> >
> > This name works for me.
> >
> >>
> >> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for
> binary
> >>   sequences with more than one element? The ``TypeError`` currently
> >> proposed
> >>   is copied (with slightly improved wording) from the behaviour of
> >> ``ord()``
> >>   with sequences containing more than one code point, while
> ``ValueError``
> >>   would be more consistent with the existing handling of out-of-range
> >>   integer values.
> >> * ``bytes.byte()`` is defined above as accepting length 1 binary
> sequences
> >>   as individual bytes, but this is currently inconsistent with the main
> >>   ``bytes`` constructor::
> >
> >
> > I don't like that bytes.byte() would accept anything other than an int.
> It
> > should not accept length 1 binary sequences at all.  I'd prefer to see
> > bytes.byte(b"X") raise a TypeError.
>
> Unfortunately, it's not that simple, because accepting both is the
> only way I see of rendering the current APIs coherent. The problem is
> that the str-derived APIs expect bytes objects, the bytearray mutating
> methods expect integers, and in Python 3.3, the substring search APIs
> were updated to accept both. This means we currently have:
>
> >>> data = bytes([1, 2, 3, 4])
> >>> 3 in data
> True
> >>> b"\x03" in data
> True
> >>> data.count(3)
> 1
> >>> data.count(b"\x03")
> 1
> >>> data.replace(3, 4)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: expected bytes, bytearray or buffer compatible object
> >>> data.replace(b"\x03", b"\x04")
> b'\x01\x02\x04\x04'
> >>> mutable = bytearray(data)
> >>> mutable
> bytearray(b'\x01\x02\x03\x04')
> >>> mutable.append(b"\x05")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: an integer is required
> >>> mutable.append(5)
> >>> mutable
> bytearray(b'\x01\x02\x03\x04\x05')
>
> Since some APIs work one way, some work the other, the only backwards
> compatible path I see to consistency is to always treat a length 1
> byte string as an acceptable input for the APIs that currently accept
> an integer and vice-versa.
>
> That said, I think this hybrid nature accurately reflects the fact
> that indexing and slicing bytes objects in Python 3 return different
> types - the individual elements are integers, but the subsequences are
> bytes objects, and several of these APIs are either
> "element-or-subsequence" APIs (in which case they should accept both),
> or else they *should* have been element APIs, but currently expect a
> subsequence due to their Python 2 str heritage.
>
> If we had the opportunity to redesign these APIs from scratch, we'd
> likely make a much clearer distinction between element based APIs
> (that would use integers) and subsequence APIs (that would accept
> buffer implementing objects). As it is, I think the situation is
> inherently ambiguous, and providing hybrid APIs to help deal with that
> ambiguity is our best available option.
>

Okay I see where you're going with this.  So long as we limit this to APIs
specifically surrounding bytes and bytearray I guess I'm "fine" with it
(given the status quo and existing mess of never knowing which behaviors
will be allowed where).

Other APIs that accept numbers outside of bytes() and bytearray() related
methods should *never* accept a bytes() of any length as valid numeric
input.

Thanks for exploring all of the APIs, we do have quite a mess that can be
made better.


> >> For ``bytearray``, some additional changes are proposed to the current
> >> integer based operations to ensure they remain consistent with the
> >> proposed
> >> constructor changes::
> >>
> >> * ``append()``: updated to be consistent with ``bytes.byte()``
> >> * ``remove()``: updated to be consistent with ``bytes.byte()``
> >> * ``+=``: updated to be consistent with ``bytes()`` changes (if any)
> >
> >
> > Where was a change to += behavior mentioned? I don't see that above (or
> did
> > I miss something?).
>
> It was an open question against the constructors - if bytes.byte() is
> defined as the PEP suggests, then the case can be made that the
> iterables accepted by the bytes() constructor should also be made more
> permissive in terms of the contents of the iterables it accepts. If
> *that* happens, then extending an existing bytearray should also
> become more permissive.
>
> Note that I'm not sold on actually changing that - that's why it's an
> open question, rather than something the PEP is currently proposing.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/a373f86a/attachment.html>

From ncoghlan at gmail.com  Sun Mar 30 09:10:46 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Mar 2014 17:10:46 +1000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
 <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com>
 <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com>
 <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com>
Message-ID: <CADiSq7d8vsGcTW5M+N5e6RFUh+dEWEipRcGqk9R8TdOrxW2GYA@mail.gmail.com>

On 30 March 2014 17:03, Gregory P. Smith <greg at krypto.org> wrote:
> Okay I see where you're going with this.  So long as we limit this to APIs
> specifically surrounding bytes and bytearray I guess I'm "fine" with it
> (given the status quo and existing mess of never knowing which behaviors
> will be allowed where).
>
> Other APIs that accept numbers outside of bytes() and bytearray() related
> methods should *never* accept a bytes() of any length as valid numeric
> input.

Yeah, agreed.

I've also reworked the relevant section of the PEP to give the
examples I posted in my reply to you - I think they do a good job of
showing why the current behaviour is problematic, and why "accept
both" is the most plausible backwards compatible remedy available to
us.

> Thanks for exploring all of the APIs, we do have quite a mess that can be
> made better.

Thank Brandon Rhodes, too - he recently pointed me to bytes.replace()
not accepting integers as a specific example of the current behaviour
being confusing for users, and after I started down that rabbithole...
well, this thread and the PEP were the end result :)

Cheers,
Nick.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From skip at pobox.com  Sun Mar 30 13:35:26 2014
From: skip at pobox.com (Skip Montanaro)
Date: Sun, 30 Mar 2014 06:35:26 -0500
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com>
Message-ID: <CANc-5Uxogk=FME-Ryz+2naE2eRg=hdba7PPhfuaNKbGDzt31Rg@mail.gmail.com>

> Google lets you search a specific URL (and its subpages):
>
> site:https://mail.python.org/mailman/listinfo/python-ideas/ ...
>
> That's been the most effective tool I've used.

+1. So effective, that I am almost completely hamstrung searching
mailing list archives, web sites or Google Groups that are hidden from
Google's view behind firewalls or login barriers.

Skip

From cjwelborn at live.com  Sun Mar 30 17:20:45 2014
From: cjwelborn at live.com (Christopher Welborn)
Date: Sun, 30 Mar 2014 10:20:45 -0500
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com>
References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando>
 <lg228f$d92$1@ger.gmane.org>
 <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com>
Message-ID: <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl>

On 03/30/2014 05:44 AM, Liam Marsh wrote:
> hello,
> is it possible to accept this syntax?
> it is about the same subject, but an other proposition...
> so, here it is:
>
>  >>>for x in xs , y in ys:
> ... ? ?***instructions***
> thank you!
>
>

I think you meant to send this to the list, instead of my personal 
email. I use thunderbird and news.gmane.org to send things to 
python-ideas, but I think you can just send a mail to: 
Python-ideas at python.org
with 'Re: <the original subject line>' such as:
Re: [Python-ideas] for-loop-if like list comps have?

-- Christopher Welborn (cj)
    cjwelborn at live.com
    http://welbornprod.com

From bcannon at gmail.com  Sun Mar 30 18:54:38 2014
From: bcannon at gmail.com (Dr. Brett Cannon)
Date: Sun, 30 Mar 2014 16:54:38 +0000
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
Message-ID: <CAP1=2W7ka+HgqfdKMJ7-75mApM8s260MwZeZH21YvddrxFBo5A@mail.gmail.com>

Can we make bytes.from_len's second argument be keyword-only? Passing in
two integer literals looks ambiguous if you don't know what the second
argument is for.

On Saturday, March 29, 2014 10:18:04 PM, Nick Coghlan <ncoghlan at gmail.com>
wrote:

On 30 March 2014 07:07, Nick Coghlan <ncoghlan
<ncoghlan at gmail.com>@<ncoghlan at gmail.com>
gmail.com <ncoghlan at gmail.com>> wrote:
> I already have a draft PEP written that covers the constructor issue,
> iteration and adding acceptance of integer inputs to the remaining
> methods that don't currently handle them. There was some background
> explanation of the text/binary domain split in the Python 2->3
> transition that I wanted Guido's feedback on before posting, but I
> just realised I can cut that out for now, and then add it back after
> Guido has had a chance to review it.
>
> So I'll tidy that up and get the draft posted later today.

Guido pointed out most of the stuff I had asked him to look at wasn't
actually relevant to the PEP, so I just cut most of it entirely.
Suffice to say, after stepping back and reviewing them systematically
for the first time in years, I believe the APIs for the core binary
data types in Python 3 could do with a little sprucing up :)

Web version: http:// <http://www.python.org/dev/peps/pep-0467/>
www.python.org <http://www.python.org/dev/peps/pep-0467/>/dev/<http://www.python.org/dev/peps/pep-0467/>
peps <http://www.python.org/dev/peps/pep-0467/>/pep-0467/<http://www.python.org/dev/peps/pep-0467/>

======================================
PEP: 467
Title: Improved API consistency for bytes and bytearray
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan <ncoghlan at gmail.com>@ <ncoghlan at gmail.com>
gmail.com <ncoghlan at gmail.com>>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-03-30
Python-Version: 3.5
Post-History: 2014-03-30

Abstract
========

During the initial development of the Python 3 language specification, the
core ``bytes`` type for arbitrary binary data started as the mutable type
that is now referred to as ``bytearray``. Other aspects of operating in
the binary domain in Python have also evolved over the course of the Python
3 series.

This PEP proposes a number of small adjustments to the APIs of the ``bytes``
and ``bytearray`` types to make their behaviour more internally consistent
and to make it easier to operate entirely in the binary domain for use cases
that actually involve manipulating binary data directly, rather than
converting it to a more structured form with additional modelling
semantics (such as ``str``) and then converting back to binary format after
processing.

Background
==========

Over the course of Python 3's evolution, a number of adjustments have been
made to the core ``bytes`` and ``bytearray`` types as additional practical
experience was gained with using them in code beyond the Python 3 standard
library and test suite. However, to date, these changes have been made
on a relatively ad hoc tactical basis as specific issues were identified,
rather than as part of a systematic review of the APIs of these types. This
approach has allowed inconsistencies to creep into the API design as to
which
input types are accepted by different methods. Additional inconsistencies
linger from an earlier pre-release design where there was *no* separate
``bytearray`` type, and instead the core ``bytes`` type was mutable (with
no immutable counterpart), as well as from the origins of these types in
the text-like behaviour of the Python 2 ``str`` type.

This PEP aims to provide the missing systematic review, with the goal of
ensuring that wherever feasible (given backwards compatibility constraints)
these current inconsistencies are addressed for the Python 3.5 release.

Proposals
=========

As a "consistency improvement" proposal, this PEP is actually about a number
of smaller micro-proposals, each aimed at improving the self-consistency of
the binary data model in Python 3. Proposals are motivated by one of three
factors:

* removing remnants of the original design of ``bytes`` as a mutable type
* more consistently accepting length 1 ``bytes`` objects as input where an
  integer between ``0`` and ``255`` inclusive is expected, and vice-versa
* allowing users to easily convert integer output to a length 1 ``bytes``
  object

Alternate Constructors
----------------------

The ``bytes`` and ``bytearray`` constructors currently accept an integer
argument, but interpret it to mean a zero-filled object of the given length.
This is a legacy of the original design of ``bytes`` as a mutable type,
rather than a particularly intuitive behaviour for users. It has become
especially confusing now that other ``bytes`` interfaces treat integers
and the corresponding length 1 bytes instances as equivalent input.
Compare::

    >>> b"\x03" in bytes([1, 2, 3])
    True
    >>> 3 in bytes([1, 2, 3])
    True

    >>> bytes(b"\x03")
    b'\x03'
    >>> bytes(3)
    b'\x00\x00\x00'

This PEP proposes that the current handling of integers in the bytes and
bytearray constructors by deprecated in Python 3.5 and removed in Python
3.6, being replaced by two more type appropriate alternate constructors
provided as class methods. The initial python-ideas thread [ideas-thread1]_
that spawned this PEP was specifically aimed at deprecating this constructor
behaviour.

For ``bytes``, a ``byte`` constructor is proposed that converts integers
(as indicated by ``operator.index``) in the appropriate range to a ``bytes``
object, converts objects that support the buffer API to bytes, and also
passes through length 1 byte strings unchanged::

    >>> bytes.byte(3)
    b'\x03'
    >>> bytes.byte(bytearray(bytes([3])))
    b'\x03'
    >>> bytes.byte(memoryview(bytes([3])))
    b'\x03'
    >>> bytes.byte(bytes([3]))
    b'\x03'
    >>> bytes.byte(512)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: bytes must be in range(0, 256)
    >>> bytes.byte(b"ab")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: bytes.byte() expected a byte, but buffer of length 2 found

One specific use case for this alternate constructor is to easily convert
the result of indexing operations on ``bytes`` and other binary sequences
from an integer to a ``bytes`` object. The documentation for this API
should note that its counterpart for the reverse conversion is ``ord()``.

For ``bytearray``, a ``from_len`` constructor is proposed that preallocates
the buffer filled with a particular value (default to ``0``) as a direct
replacement for the current constructor behaviour, rather than having to use
sequence repetition to achieve the same effect in a less intuitive way::

    >>> bytearray.from_len(3)
    bytearray(b'\x00\x00\x00')
    >>> bytearray.from_len(3, 6)
    bytearray(b'\x06\x06\x06'



This part of the proposal was covered by an existing issue
[empty-buffer-issue]_ and a variety of names have been proposed
(``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The
specific name currently proposed was chosen by analogy with
``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely
explicit that it is an alternate constructor rather than an in-place
mutation, as well as how it differs from the standard constructor.

Open questions
^^^^^^^^^^^^^^

* Should ``bytearray.byte()`` also be added? Or is
  ``bytearray(bytes.byte(x))`` sufficient for that case?
* Should ``bytes.from_len()`` also be added? Or is sequence repetition
  sufficient for that case?
* Should ``bytearray.from_len()`` use a different name?
* Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary
  sequences with more than one element? The ``TypeError`` currently proposed
  is copied (with slightly improved wording) from the behaviour of ``ord()``
  with sequences containing more than one code point, while ``ValueError``
  would be more consistent with the existing handling of out-of-range
  integer values.
* ``bytes.byte()`` is defined above as accepting length 1 binary sequences
  as individual bytes, but this is currently inconsistent with the main
  ``bytes`` constructor::

      >>> bytes([b"a", b"b", b"c"])
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      TypeError: 'bytes' object cannot be interpreted as an integer

  Should the ``bytes`` constructor be changed to accept iterables of length
1
  bytes objects in addition to iterables of integers? If so, should it
  allow a mixture of the two in a single iterable?

Iteration
---------

Iteration over ``bytes`` objects and other binary sequences produces
integers. Rather than proposing a new method that would need to be added
not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially
to third party types as well, this PEP proposes that iteration to produce
length 1 ``bytes`` objects instead be handled by combining ``map`` with
the new ``bytes.byte()`` alternate constructor proposed above::

    for x in map(bytes.byte, data):
        # x is a length 1 ``bytes`` object, rather than an integer
        # This works with *any* container of integers in the range
        # 0 to 255 inclusive

Consistent support for different input types
--------------------------------------------

In Python 3.3, the binary search operations (``in``, ``count()``,
``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to
accept integers in the range 0 to 255 (inclusive) as their first argument
(in addition to the existing support for binary sequences).

This PEP proposes extending that behaviour of accepting integers as being
equivalent to the corresponding length 1 binary sequence to several other
``bytes`` and ``bytearray`` methods that currently expect a ``bytes``
object for certain parameters. In essence, if a value is an acceptable
input to the new ``bytes.byte`` constructor defined above, then it would
be acceptable in the roles defined here (in addition to any other already
supported inputs):

* ``startswith()`` prefix(es)
* ``endswith()`` suffix(es)

* ``center()`` fill character
* ``ljust()`` fill character
* ``rjust()`` fill character

* ``strip()`` character to strip
* ``lstrip()`` character to strip
* ``rstrip()`` character to strip

* ``partition()`` separator argument
* ``rpartition()`` separator argument

* ``split()`` separator argument
* ``rsplit()`` separator argument

* ``replace()`` old value and new value

In addition to the consistency motive, this approach also makes it easier
to work with the indexing behaviour , as the result of an indexing operation
can more easily be fed back in to other methods.

For ``bytearray``, some additional changes are proposed to the current
integer based operations to ensure they remain consistent with the proposed
constructor changes::

* ``append()``: updated to be consistent with ``bytes.byte()``
* ``remove()``: updated to be consistent with ``bytes.byte()``
* ``+=``: updated to be consistent with ``bytes()`` changes (if any)
* ``extend()``: updated to be consistent with ``bytes()`` changes (if any)

Acknowledgement of surprising behaviour of some ``bytearray`` methods
---------------------------------------------------------------------

Several of the ``bytes`` and ``bytearray`` methods have their origins in the
Python 2 ``str`` API. As ``str`` is an immutable type, all of these
operations are defined as returning a *new* instance, rather than operating
in place. This contrasts with methods on other mutable types like ``list``,
where ``list.sort()`` and ``list.reverse()`` operate in-place and return
``None``, rather than creating a new object.

Backwards compatibility constraints make it impractical to change this
behaviour at this point, but it may be appropriate to explicitly call out
this quirk in the documentation for the ``bytearray`` type. It affects the
following methods that could reasonably be expected to operate in-place on
a mutable type:

* ``center()``
* ``ljust()``
* ``rjust()``
* ``strip()``
* ``lstrip()``
* ``rstrip()``
* ``replace()``
* ``lower()``
* ``upper()``
* ``swapcase()``
* ``title()``
* ``capitalize()``
* ``translate()``
* ``expandtabs()``
* ``zfill()``

Note that the following ``bytearray`` operations *do* operate in place, as
they're part of the mutable sequence API in ``bytearray``, rather than being
inspired by the immutable Python 2 ``str`` API:

* ``+=``
* ``append()``
* ``extend()``
* ``reverse()``
* ``remove()``
* ``pop()``

References
==========

.. [ideas-thread1]
https<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
:// <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
mail.python.org<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
/ <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
pipermail<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
/python-ideas/2014-March/027295.<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
html <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html>
.. [empty-buffer-issue] http:// <http://bugs.python.org/issue20895>
bugs.python.org
<http://bugs.python.org/issue20895>/issue20895<http://bugs.python.org/issue20895>

Copyright
=========

This document has been placed in the public domain.

--
Nick Coghlan   |   ncoghlan <ncoghlan at gmail.com>@ <ncoghlan at gmail.com>
gmail.com <ncoghlan at gmail.com>   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python-ideas@ <Python-ideas at python.org>python.org <Python-ideas at python.org>
https <https://mail.python.org/mailman/listinfo/python-ideas>://<https://mail.python.org/mailman/listinfo/python-ideas>
mail.python.org <https://mail.python.org/mailman/listinfo/python-ideas>
/mailman/ <https://mail.python.org/mailman/listinfo/python-ideas>listinfo<https://mail.python.org/mailman/listinfo/python-ideas>
/python-ideas <https://mail.python.org/mailman/listinfo/python-ideas>
Code of Conduct: http://
<http://python.org/psf/codeofconduct/>python.org<http://python.org/psf/codeofconduct/>
/ <http://python.org/psf/codeofconduct/>psf<http://python.org/psf/codeofconduct/>
/ <http://python.org/psf/codeofconduct/>codeofconduct<http://python.org/psf/codeofconduct/>
/ <http://python.org/psf/codeofconduct/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/5ed046dc/attachment-0001.html>

From guido at python.org  Sun Mar 30 18:05:44 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 30 Mar 2014 09:05:44 -0700
Subject: [Python-ideas] Fixing the Python 3 bytes constructor
In-Reply-To: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com>
 <5336F9A6.2030006@stoneleaf.us>
 <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com>
 <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com>
 <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com>
Message-ID: <CAP7+vJL4dQ3z9XMR6=iUgcsPK0kZDA=s16ty-ZoScA6AkDjKag@mail.gmail.com>

On Sat, Mar 29, 2014 at 7:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > I already have a draft PEP written that covers the constructor issue,
> > iteration and adding acceptance of integer inputs to the remaining
> > methods that don't currently handle them. There was some background
> > explanation of the text/binary domain split in the Python 2->3
> > transition that I wanted Guido's feedback on before posting, but I
> > just realised I can cut that out for now, and then add it back after
> > Guido has had a chance to review it.
> >
> > So I'll tidy that up and get the draft posted later today.
>
> Guido pointed out most of the stuff I had asked him to look at wasn't
> actually relevant to the PEP, so I just cut most of it entirely.
> Suffice to say, after stepping back and reviewing them systematically
> for the first time in years, I believe the APIs for the core binary
> data types in Python 3 could do with a little sprucing up :)
>

Thanks for cutting it down, it's easier to concentrate on the essentials
now.


> Web version: http://www.python.org/dev/peps/pep-0467/
>
> ======================================
> PEP: 467
> Title: Improved API consistency for bytes and bytearray
> Version: $Revision$
> Last-Modified: $Date$
> Author: Nick Coghlan <ncoghlan at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 2014-03-30
> Python-Version: 3.5
> Post-History: 2014-03-30
>
>
> Abstract
> ========
>
> During the initial development of the Python 3 language specification, the
> core ``bytes`` type for arbitrary binary data started as the mutable type
> that is now referred to as ``bytearray``. Other aspects of operating in
> the binary domain in Python have also evolved over the course of the Python
> 3 series.
>
> This PEP proposes a number of small adjustments to the APIs of the
> ``bytes``
> and ``bytearray`` types to make their behaviour more internally consistent
> and to make it easier to operate entirely in the binary domain for use
> cases
> that actually involve manipulating binary data directly, rather than
> converting it to a more structured form with additional modelling
> semantics (such as ``str``) and then converting back to binary format after
> processing.
>

I hope you don't mind I cut the last 60% of this sentence (everything after
"binary domain").

>
>
> Background
> ==========
>
> Over the course of Python 3's evolution, a number of adjustments have been
> made to the core ``bytes`` and ``bytearray`` types as additional practical
> experience was gained with using them in code beyond the Python 3 standard
> library and test suite. However, to date, these changes have been made
> on a relatively ad hoc tactical basis as specific issues were identified,
> rather than as part of a systematic review of the APIs of these types.


I'm not sure you can claim that. We probably have more information based on
experience now than when we did the redesign. (At that time most experience
was based on using str() for binary data.)


> This
> approach has allowed inconsistencies to creep into the API design as to
> which
> input types are accepted by different methods. Additional inconsistencies
> linger from an earlier pre-release design where there was *no* separate
> ``bytearray`` type, and instead the core ``bytes`` type was mutable (with
> no immutable counterpart), as well as from the origins of these types in
> the text-like behaviour of the Python 2 ``str`` type.
>

You make it sound as if modeling bytes() after Python 2's str() was an
accident. It wasn't.


> This PEP aims to provide the missing systematic review, with the goal of
> ensuring that wherever feasible (given backwards compatibility constraints)
> these current inconsistencies are addressed for the Python 3.5 release.
>

I would like to convince you to aim lower, drop the "systematic review",
and just focus on some changes that are likely to improve users' experience
(which includes porting Python 2 code).

>
>
> Proposals
> =========
>
> As a "consistency improvement" proposal, this PEP is actually about a
> number
> of smaller micro-proposals, each aimed at improving the self-consistency of
> the binary data model in Python 3. Proposals are motivated by one of three
> factors:
>
> * removing remnants of the original design of ``bytes`` as a mutable type
>

Yes.

* more consistently accepting length 1 ``bytes`` objects as input where an
>   integer between ``0`` and ``255`` inclusive is expected, and vice-versa
>

Not sure I like this as a goal. OK, stronger: I don't like this goal.


> * allowing users to easily convert integer output to a length 1 ``bytes``
>   object
>

I think you meant integer values instead of output? In Python 2 we did this
with the global function chr(), but in Python 3 that creates a str(). (The
history of chr() and ord() sa built-in functions is that they long predates
the notion of methods (class- or otherwise), and their naming comes
straight from Pascal.)

Anyway, I don't know that the use case is so common that it needs more than
bytes([i]) or bytearray([i]) -- if there is an argument to be made for
bytes.byte(i) and bytearray.byte(i) it would be that the [i] in the
constructor is somewhat hard to grasp.

>
>
> Alternate Constructors
> ----------------------
>
> The ``bytes`` and ``bytearray`` constructors currently accept an integer
> argument, but interpret it to mean a zero-filled object of the given
> length.
>

This is one of the two legacies of the original "mutable bytes" design, and
I agree we should strive to replace it -- although I think one round of
deprecation may be too quick. (The other legacy is of course that b[i] is
an int, not a bytes -- it's the worse problem, but I don't think we can fix
it without breaking more than the fix would be worth.)


> This is a legacy of the original design of ``bytes`` as a mutable type,
> rather than a particularly intuitive behaviour for users. It has become
> especially confusing now that other ``bytes`` interfaces treat integers
> and the corresponding length 1 bytes instances as equivalent input.
> Compare::
>
>     >>> b"\x03" in bytes([1, 2, 3])
>     True
>     >>> 3 in bytes([1, 2, 3])
>     True
>
>     >>> bytes(b"\x03")
>     b'\x03'
>     >>> bytes(3)
>     b'\x00\x00\x00'
>
> This PEP proposes that the current handling of integers in the bytes and
> bytearray constructors by deprecated in Python 3.5 and removed in Python
> 3.6, being replaced by two more type appropriate alternate constructors
> provided as class methods. The initial python-ideas thread [ideas-thread1]_
> that spawned this PEP was specifically aimed at deprecating this
> constructor
> behaviour.
>
> For ``bytes``, a ``byte`` constructor is proposed that converts integers
> (as indicated by ``operator.index``)


I know why you reference this, but it feels confusing to me. At this point
in the narrative it's better to just say "integer" and explain how it
decides "integer-ness" later.


> in the appropriate range to a ``bytes``
> object, converts objects that support the buffer API to bytes, and also
> passes through length 1 byte strings unchanged::
>

I think the second half (accepting bytes instances of length 1) is wrong
here and doesn't actually have a practical use case. I'll say more below.


>
>     >>> bytes.byte(3)
>     b'\x03'
>     >>> bytes.byte(bytearray(bytes([3])))
>     b'\x03'
>     >>> bytes.byte(memoryview(bytes([3])))
>     b'\x03'
>     >>> bytes.byte(bytes([3]))
>     b'\x03'
>     >>> bytes.byte(512)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     ValueError: bytes must be in range(0, 256)
>     >>> bytes.byte(b"ab")
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: bytes.byte() expected a byte, but buffer of length 2 found
>
> One specific use case for this alternate constructor is to easily convert
> the result of indexing operations on ``bytes`` and other binary sequences
> from an integer to a ``bytes`` object. The documentation for this API
> should note that its counterpart for the reverse conversion is ``ord()``.
>

However, in a pinch, b[0] will do as well, assuming you don't need the
length check implied by ord().

>
> For ``bytearray``, a ``from_len`` constructor is proposed that preallocates
> the buffer filled with a particular value (default to ``0``) as a direct
> replacement for the current constructor behaviour, rather than having to
> use
> sequence repetition to achieve the same effect in a less intuitive way::
>
>     >>> bytearray.from_len(3)
>     bytearray(b'\x00\x00\x00')
>     >>> bytearray.from_len(3, 6)
>     bytearray(b'\x06\x06\x06')
>
> This part of the proposal was covered by an existing issue
> [empty-buffer-issue]_ and a variety of names have been proposed
> (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The
> specific name currently proposed was chosen by analogy with
> ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely
> explicit that it is an alternate constructor rather than an in-place
> mutation, as well as how it differs from the standard constructor.
>

I think you need to brainstorm more on the name; from_len() looks pretty
awkward. And I think it's better to add it to bytes() as well, since the
two classes intentionally try to be as similar as possible.

>
>
> Open questions
> ^^^^^^^^^^^^^^
>
> * Should ``bytearray.byte()`` also be added? Or is
>   ``bytearray(bytes.byte(x))`` sufficient for that case?
>

It should be added.


> * Should ``bytes.from_len()`` also be added? Or is sequence repetition
>   sufficient for that case?
>

It should be added.


> * Should ``bytearray.from_len()`` use a different name?
>

Yes.


> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary
>   sequences with more than one element? The ``TypeError`` currently
> proposed
>   is copied (with slightly improved wording) from the behaviour of
> ``ord()``
>   with sequences containing more than one code point, while ``ValueError``
>   would be more consistent with the existing handling of out-of-range
>   integer values.
>

It should not accept any bytes arguments. But if somehow you convince me
otherwise, it should be ValueError (and honestly, ord() is wrong there).


> * ``bytes.byte()`` is defined above as accepting length 1 binary sequences
>   as individual bytes, but this is currently inconsistent with the main
>   ``bytes`` constructor::
>
>       >>> bytes([b"a", b"b", b"c"])
>       Traceback (most recent call last):
>         File "<stdin>", line 1, in <module>
>       TypeError: 'bytes' object cannot be interpreted as an integer
>
>   Should the ``bytes`` constructor be changed to accept iterables of
> length 1
>   bytes objects in addition to iterables of integers? If so, should it
>   allow a mixture of the two in a single iterable?
>

Noooooooooooooooooooooooooo!!!!!


>
>
> Iteration
> ---------
>
> Iteration over ``bytes`` objects and other binary sequences produces
> integers. Rather than proposing a new method that would need to be added
> not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially
> to third party types as well, this PEP proposes that iteration to produce
> length 1 ``bytes`` objects instead be handled by combining ``map`` with
> the new ``bytes.byte()`` alternate constructor proposed above::
>
>     for x in map(bytes.byte, data):
>         # x is a length 1 ``bytes`` object, rather than an integer
>         # This works with *any* container of integers in the range
>         # 0 to 255 inclusive
>

I can see why you don't like a new method, but this idiom is way too
verbose and unintuitive to ever gain traction. Let's just add a new method
to all three types, 3rd party types will get the message.


>
>
> Consistent support for different input types
> --------------------------------------------
>
> In Python 3.3, the binary search operations (``in``, ``count()``,
> ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to
> accept integers in the range 0 to 255 (inclusive) as their first argument
> (in addition to the existing support for binary sequences).
>

I wonder if that wasn't a bit over-zealous. While 'in', count() and index()
are sequence methods (looking for elements) that have an extended meaning
(looking for substrings) for string types, the find() and r*() variants are
only defined for strings.


> This PEP proposes extending that behaviour of accepting integers as being
> equivalent to the corresponding length 1 binary sequence to several other
> ``bytes`` and ``bytearray`` methods that currently expect a ``bytes``
> object for certain parameters. In essence, if a value is an acceptable
> input to the new ``bytes.byte`` constructor defined above, then it would
> be acceptable in the roles defined here (in addition to any other already
> supported inputs):
>
> * ``startswith()`` prefix(es)
> * ``endswith()`` suffix(es)
>
> * ``center()`` fill character
> * ``ljust()`` fill character
> * ``rjust()`` fill character
>
> * ``strip()`` character to strip
> * ``lstrip()`` character to strip
> * ``rstrip()`` character to strip
>
> * ``partition()`` separator argument
> * ``rpartition()`` separator argument
>
> * ``split()`` separator argument
> * ``rsplit()`` separator argument
>
> * ``replace()`` old value and new value
>
> In addition to the consistency motive, this approach also makes it easier
> to work with the indexing behaviour , as the result of an indexing
> operation
> can more easily be fed back in to other methods.
>

I think herein lies madness. The intention seems to be to paper over as
much as possible the unfortunate behavior of b[i]. But how often does any
of these methods get called with such a construct? And how often will that
be in a context where this is the *only* thing that is affected by b[i]
returning an int in Python 3 but a string in Python 2? (In my experience
these are mostly called with literal arguments, except inside wrapper
functions that are themselves intended to be called with a literal
argument.) Weakening the type checking here seems a bad idea -- it would
accept integers in *any* context, and that would just cause more nasty
debugging issues.


> For ``bytearray``, some additional changes are proposed to the current
> integer based operations to ensure they remain consistent with the proposed
> constructor changes::
>
> * ``append()``: updated to be consistent with ``bytes.byte()``
> * ``remove()``: updated to be consistent with ``bytes.byte()``
> * ``+=``: updated to be consistent with ``bytes()`` changes (if any)
> * ``extend()``: updated to be consistent with ``bytes()`` changes (if any)
>

Eew again. These are operations from the MutableSequence ABC and there is
no reason to make their signatures fuzzier.


>
>
> Acknowledgement of surprising behaviour of some ``bytearray`` methods
> ---------------------------------------------------------------------
>
> Several of the ``bytes`` and ``bytearray`` methods have their origins in
> the
> Python 2 ``str`` API.


You make it sound as if this is a bad thing or an accident.


> As ``str`` is an immutable type, all of these
> operations are defined as returning a *new* instance, rather than operating
> in place. This contrasts with methods on other mutable types like ``list``,
> where ``list.sort()`` and ``list.reverse()`` operate in-place and return
> ``None``, rather than creating a new object.
>

So does bytestring.reverse(). And if you really insist we can add
bytestring.sort(). :-)


>
> Backwards compatibility constraints make it impractical to change this
> behaviour at this point, but it may be appropriate to explicitly call out
> this quirk in the documentation for the ``bytearray`` type. It affects the
> following methods that could reasonably be expected to operate in-place on
> a mutable type:
>
> * ``center()``
> * ``ljust()``
> * ``rjust()``
> * ``strip()``
> * ``lstrip()``
> * ``rstrip()``
> * ``replace()``
> * ``lower()``
> * ``upper()``
> * ``swapcase()``
> * ``title()``
> * ``capitalize()``
> * ``translate()``
> * ``expandtabs()``
> * ``zfill()``
>

That all feels like hypercorrection. These are string methods and it would
be completely wrong if bytearray changed them to modify the object
in-place. I also don't see why anyone would think these would modify the
object, given that everybody encounters these first for the str() type,
then for bytes(), then finally (by extension) for bytearray().

The *only* place where there should be any confusion about whether the
value is mutated or the variable is updated with a new object would be the
+= operator (and *=) but that's due to that operator's ambiguity.


> Note that the following ``bytearray`` operations *do* operate in place, as
> they're part of the mutable sequence API in ``bytearray``, rather than
> being
> inspired by the immutable Python 2 ``str`` API:
>
> * ``+=``
> * ``append()``
> * ``extend()``
> * ``reverse()``
> * ``remove()``
> * ``pop()``
>

Right. And there's nothing wrong with this.

>
>
> References
> ==========
>
> .. [ideas-thread1]
> https://mail.python.org/pipermail/python-ideas/2014-March/027295.html
> .. [empty-buffer-issue] http://bugs.python.org/issue20895
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/06a5ffd0/attachment-0001.html>

From ram.rachum at gmail.com  Sun Mar 30 19:59:27 2014
From: ram.rachum at gmail.com (Ram Rachum)
Date: Sun, 30 Mar 2014 10:59:27 -0700 (PDT)
Subject: [Python-ideas] Allow specifying list of functions to `map`
Message-ID: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>

If `map` accepted a tuple for its first argument, things like this: 

    stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate()))

Could become this: 

    stdout, stderr = map((bytes.decode, str.strip), popen.communicate())

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/f844b68c/attachment.html>

From python at mrabarnett.plus.com  Sun Mar 30 21:06:25 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 30 Mar 2014 20:06:25 +0100
Subject: [Python-ideas] Allow specifying list of functions to `map`
In-Reply-To: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
Message-ID: <53386B31.8040505@mrabarnett.plus.com>

On 2014-03-30 18:59, Ram Rachum wrote:
> If `map` accepted a tuple for its first argument, things like this:
>
>      stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate()))
>
> Could become this:
>
>      stdout, stderr = map((bytes.decode, str.strip), popen.communicate())
>
I think that the problem there is that it's not obvious in what order
they are applied. You could, for example, argue that map((f, g) x)
should do map(f, map(g, x) rather than map(g, map(f, x)).

From abarnert at yahoo.com  Sun Mar 30 23:12:42 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 30 Mar 2014 14:12:42 -0700 (PDT)
Subject: [Python-ideas] for-loop-if like list comps have?
In-Reply-To: <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl>
References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando>
 <lg228f$d92$1@ger.gmane.org>
 <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com>
 <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl>
Message-ID: <1396213962.86004.YahooMailNeo@web181001.mail.ne1.yahoo.com>

From: Christopher Welborn <cjwelborn at live.com>

> On 03/30/2014 05:44 AM, Liam Marsh wrote:
>>  hello,
>>  is it possible to accept this syntax?
>>  it is about the same subject, but an other proposition...
>>  so, here it is:
>> 
>> ? >>>for x in xs , y in ys:
>>  ... ? ?***instructions***
>>  thank you!

(This is obviously a reply to Liam, not Christopher.)


It's not clear which of these you want that to mean:

? ? for x, y in zip(xs, ys):
? ? ? ? do_stuff(x, y)

? ? for x, y in product(xs, ys):
? ? ? ? do_stuff(x, y)

And the existing versions are already pretty clear and readable.

In a listcomp, it's often clearer to use two for clauses than product? but usually in that case you write them on separate lines, and in a statement, it would be even clearer to use nested statements. Compare this somewhat realistic use case:

? ? transformed_values = (long expression with value and sublist
? ? ? ? ? ? ? ? ? ? ? ? ? for sublist in?long_function_that_returns_a_list(various, arguments)
? ? ? ? ? ? ? ? ? ? ? ? ? for value in sublist)

? ? for sublist in long_function_that_returns_a_list(various, arguments):
? ? ? ? for value in sublist:
? ? ? ? ? ? yield long expression with value and sublist

Would you really want to write the second one on one line, even if you could?

? ? for sublist in?long_function_that_returns_a_list(various, arguments), value in sublist:
? ? ? ? yield long expression with value and sublist

From rosuav at gmail.com  Mon Mar 31 00:00:47 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 31 Mar 2014 09:00:47 +1100
Subject: [Python-ideas] Allow specifying list of functions to `map`
In-Reply-To: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
Message-ID: <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com>

On Mon, Mar 31, 2014 at 4:59 AM, Ram Rachum <ram.rachum at gmail.com> wrote:
> If `map` accepted a tuple for its first argument, things like this:
>
>     stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate()))
>
> Could become this:
>
>     stdout, stderr = map((bytes.decode, str.strip), popen.communicate())

stdout, stderr = map(lambda x: x.decode().strip(), popen.communicate())

Or:

stdout, stderr = popen.communicate()
stdout = stdout.decode().strip()
stderr = stderr.decode().strip()

Yes, the latter is a bit of duplication, but it's clear what's going
on. (Obviously appropriate only where you know exactly how many
elements there'll be, as in your example.)

ChrisA

From cs at zip.com.au  Mon Mar 31 00:22:34 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 31 Mar 2014 09:22:34 +1100
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
Message-ID: <20140330222234.GA14300@cskk.homeip.net>

On 30Mar2014 08:05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Richard Prosser <richard.prosser at mail.com> writes:
> > We have generally-appicable services that index the web and make much
> > better search tools. I don't often find a site's own custom search tool
> > to be sufficiently better to use it, when DuckDuckGo is available.
> 
> That said, migrating to Mailman3 + HyperKitty is definitely in the
> longer term plans for the python.org mailing list infrastructure, and
> that includes integrated search on the HyperKitty side of things.
>
> While Mailman3 has been an ongoing project for quite some time (I
> believe the remaining blockers mostly relate to handling migrations of
> existing Mailman 2 installations), the HyperKitty work is mostly being
> driven by some Fedora folks in order to upgrade Fedora's own
> infrastructure. You can see the current state of the prototype here:
> https://lists.stg.fedoraproject.org/archives/

Please tell me Mailman3 still includes the pipermail stuff still.

Looking at the Fedora HyperKitty page you cite, is it just me or
is this really slow and clunky? [Ten minutes later...] It is slightly
quicker now, but that first page load took minutes.

The main page search (multilist) doesn't show the list name in the
search results, making it harder to gauge relevance.

I think this is an argument for the OP's "convenient search of
relevant python lists" request, even if that were a quick hack that
readirected to a general purpose search engine with a funky search
criterion. But a search engine covers multiple sites; this can only
search its own lists.

Also, there are no archive download links.
Nor can I see the original text of any message (no headers or anything).
This cripples personal searching.

When I join a list my first act, if possible, is to download the
pipermail archives and unpack them into the mail folder in which
future lists posts will go.

That gets me:

  - local search of local data
    access when offline, privacy, speed, whatever tools or indices I prefer

  - search tools of my choosing (eg mairix or mu), including the
    search facilities of my mail reader

  - using my mail reader search lets me read messages in my _preferred_ form,
    as it would for anyone else using their preferrer mail reader

Without an archive download, this is all far far less useful.

HyperKitty looks... pretty... I suppose (for some definition of the
term; I'm no far of the great big blocks of whitespace on either
side, etc), for someone who enjoys web forums.

Disgruntledly yours,
-- 
Cameron Simpson <cs at zip.com.au>

From python at mrabarnett.plus.com  Mon Mar 31 00:24:30 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 30 Mar 2014 23:24:30 +0100
Subject: [Python-ideas] Allow specifying list of functions to `map`
In-Reply-To: <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com>
References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
 <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com>
Message-ID: <5338999E.3050206@mrabarnett.plus.com>

On 2014-03-30 23:00, Chris Angelico wrote:
> On Mon, Mar 31, 2014 at 4:59 AM, Ram Rachum <ram.rachum at gmail.com> wrote:
>> If `map` accepted a tuple for its first argument, things like this:
>>
>>     stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate()))
>>
>> Could become this:
>>
>>     stdout, stderr = map((bytes.decode, str.strip), popen.communicate())
>
> stdout, stderr = map(lambda x: x.decode().strip(), popen.communicate())
>
> Or:
>
> stdout, stderr = popen.communicate()
> stdout = stdout.decode().strip()
> stderr = stderr.decode().strip()
>
Or:

stdout, stderr = (x.decode().strip() for x in popen.communicate())

for that matter.

> Yes, the latter is a bit of duplication, but it's clear what's going
> on. (Obviously appropriate only where you know exactly how many
> elements there'll be, as in your example.)
>


From ncoghlan at gmail.com  Mon Mar 31 00:37:02 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 31 Mar 2014 08:37:02 +1000
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <20140330222234.GA14300@cskk.homeip.net>
References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
 <20140330222234.GA14300@cskk.homeip.net>
Message-ID: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>

On 31 Mar 2014 08:22, "Cameron Simpson" <cs at zip.com.au> wrote:
>
> On 30Mar2014 08:05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote:
> > > Richard Prosser <richard.prosser at mail.com> writes:
> > > We have generally-appicable services that index the web and make much
> > > better search tools. I don't often find a site's own custom search
tool
> > > to be sufficiently better to use it, when DuckDuckGo is available.
> >
> > That said, migrating to Mailman3 + HyperKitty is definitely in the
> > longer term plans for the python.org mailing list infrastructure, and
> > that includes integrated search on the HyperKitty side of things.
> >
> > While Mailman3 has been an ongoing project for quite some time (I
> > believe the remaining blockers mostly relate to handling migrations of
> > existing Mailman 2 installations), the HyperKitty work is mostly being
> > driven by some Fedora folks in order to upgrade Fedora's own
> > infrastructure. You can see the current state of the prototype here:
> > https://lists.stg.fedoraproject.org/archives/
>
> Please tell me Mailman3 still includes the pipermail stuff still.

Mailman3 includes a pluggable archiver model - that's how HyperKitty can be
a separate project, specifically aimed at tackling the "seamless web
gateway" problem.

> That gets me:
>
>   - local search of local data
>     access when offline, privacy, speed, whatever tools or indices I
prefer
>
>   - search tools of my choosing (eg mairix or mu), including the
>     search facilities of my mail reader
>
>   - using my mail reader search lets me read messages in my _preferred_
form,
>     as it would for anyone else using their preferrer mail reader
>
> Without an archive download, this is all far far less useful.

The pluggable archiver model in MM3 means HyperKitty itself doesn't need to
handle that usage model. However, I'm not aware of any current efforts to
create an MM3 archiver that is closer to a pure pipermail replacement (the
workflow you describe strikes me as being incredibly unusual these days, as
it requires either running your own mail server, or only accessing your
email from one device)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/fbeaa6fa/attachment.html>

From abarnert at yahoo.com  Mon Mar 31 00:53:31 2014
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 30 Mar 2014 15:53:31 -0700 (PDT)
Subject: [Python-ideas] Fw:  Allow specifying list of functions to `map`
In-Reply-To: <1396218299.38518.YahooMailNeo@web181005.mail.ne1.yahoo.com>
References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com>
 <1396218299.38518.YahooMailNeo@web181005.mail.ne1.yahoo.com>
Message-ID: <1396220011.5869.YahooMailNeo@web181002.mail.ne1.yahoo.com>

Sorry, google-grouped, so I'm not sure if this went out to the list or not; apologies if it's a dup?


From: Ram Rachum <ram.rachum at gmail.com>
Sent: Sunday, March 30, 2014 10:59 AM


> If `map` accepted a tuple for its first argument, things like this:?
> 
> ? ? stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate()))
> 
> Could become this:?
> 
> ? ? stdout, stderr = map((bytes.decode, str.strip), popen.communicate())

I'm not sure whether to read that as returning:


? ? map(bytes.decode, x), map(str.strip, x)
? ? map(compose(bytes.decode, str.strip), x)
? ? map(rcompose(bytes.decode, str.strip), x)


None of the functional languages that Python borrowed map from have any of?
these, but that's because they all have compose as a builtin, or even an 
operator:

? ? map(bytes.decode @ str.strip, popen.communicate())

But I think in Python this is much more readable as an expression on each value 
rather than a higher-order function:

? ? (text.decode().strip() for text in popen.communicate())


Even in Haskell, a listcomp is often more readable than mapping a compose or 
other higher-order function, but that's much more true when you're using 
Python-style method calls.


From cs at zip.com.au  Mon Mar 31 02:47:33 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 31 Mar 2014 11:47:33 +1100
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
References: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
Message-ID: <20140331004733.GA73427@cskk.homeip.net>

On 31Mar2014 08:37, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 31 Mar 2014 08:22, "Cameron Simpson" <cs at zip.com.au> wrote:
> > Please tell me Mailman3 still includes the pipermail stuff still.
> 
> Mailman3 includes a pluggable archiver model - that's how HyperKitty can be
> a separate project, specifically aimed at tackling the "seamless web
> gateway" problem.

If it can be run in parallel, fine. If it replaces pipermail, it
looks like a step backwards.

> > That gets me:
> >   - local search of local data
> >     access when offline, privacy, speed, whatever tools or indices I
> prefer
> >
> >   - search tools of my choosing (eg mairix or mu), including the
> >     search facilities of my mail reader
> >
> >   - using my mail reader search lets me read messages in my _preferred_
> form,
> >     as it would for anyone else using their preferrer mail reader
> >
> > Without an archive download, this is all far far less useful.
> 
> The pluggable archiver model in MM3 means HyperKitty itself doesn't need to
> handle that usage model. However, I'm not aware of any current efforts to
> create an MM3 archiver that is closer to a pure pipermail replacement (the
> workflow you describe strikes me as being incredibly unusual these days, as
> it requires either running your own mail server, or only accessing your
> email from one device)

My workflow is unusual only because most people don't think about
it and because the existing MUAs do not make it easy. There _should_
be a "join a mailing list" button with optional archive suck.

Most modern MUAs have a "search my mail" facility. Making the raw
list archive hard to obtain gets in the way of offering the user's
own search field for search.

_Nothing_ in my workflow requires running my own mail server, it
merely requires having local folders; which is entirely reasonable
with an IMAP service, and entirely reasonable with multiple devices.

Using one's own search tools is only unusual because unless you're
a nerd it is hard.

Likewise, sucking down the archives is difficult for end users for
various reasons:
  - even pipemail makes you do that in pieces
  - the mbox archive URLs (often gzipped) don't come down with a
    cool MIME type meaning "UNIX mailbox format"
  - the pipermail archives need some unobfuscation before use as
    pure mbox files
  - most MUAs do not install a platform hook to handle an mbox MIME
    type (if there even is one), thus making an mbox download that
    much harder on the user.

None of these is hard to address. I do it with a tiny shell script,
but that is not for most people.

I still content that any change that removes the simple ability to
fetch the list archive for use by the end user is a huge step
backwards, regardless of the prettiness of whatever replaces it.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Early to bed and early to rise makes a man healthy and wealthy with bags
under his eyes. - Dave Cochran, <cochran at dg-rtp.dg.com>

From turnbull at sk.tsukuba.ac.jp  Mon Mar 31 07:07:24 2014
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 31 Mar 2014 14:07:24 +0900
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <20140331004733.GA73427@cskk.homeip.net>
References: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
 <20140331004733.GA73427@cskk.homeip.net>
Message-ID: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp>

N.B. This is off-topic.  If you really care that much, people who want
to hear about it are over on mailman-developers at python.org.

Cameron Simpson writes:

 >   - the mbox archive URLs (often gzipped) don't come down with a
 >     cool MIME type meaning "UNIX mailbox format"

There isn't one, can't be one at present, and likely won't be one in
the future: http://www.jwz.org/doc/content-length.html.  This still
applies to some extent today despite Jamie's rant being almost 20
years old -- Mailman can't choose the mbox spec being used, that
depends on the MTA.

If you want something that could actually have a MIME-Type, there's
Babyl, MMDF, or (file-per-message) maildir, all of which have
reasonably precise specs -- but aren't in anywhere near as wide use.

Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's
far more robust and easier to handle than mbox.

 > I still content that any change that removes the simple ability to
 > fetch the list archive for use by the end user is a huge step
 > backwards, regardless of the prettiness of whatever replaces it.

It's not a question of "removing".  This has always been a user
choice, and many lists use third-party services (mail-archive.com) or
non-pipermail archivers (MHonArc).  It's just that Mailman 2 with its
bundled mbox-format archiver made it easy to provide by default.

If lots of users really prefer pipermail, for whatever reason, it's
not that hard to add by hand, and could be packaged on PyPI.


From cs at zip.com.au  Mon Mar 31 07:52:14 2014
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 31 Mar 2014 16:52:14 +1100
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140331055214.GA3830@cskk.homeip.net>

On 31Mar2014 14:07, Stephen J. Turnbull <turnbull at sk.tsukuba.ac.jp> wrote:
> N.B. This is off-topic.  If you really care that much, people who want
> to hear about it are over on mailman-developers at python.org.

I may join that list, though I'd be an informed commenter (== whinger)
rather than a dev.

> Cameron Simpson writes:
>  >   - the mbox archive URLs (often gzipped) don't come down with a
>  >     cool MIME type meaning "UNIX mailbox format"
> 
> There isn't one, can't be one at present, and likely won't be one in
> the future: http://www.jwz.org/doc/content-length.html.  This still
> applies to some extent today despite Jamie's rant being almost 20
> years old -- Mailman can't choose the mbox spec being used, that
> depends on the MTA.

It can cheerfully choose the mbox spec for delivery of an archive.
It is very simple and is a single flat file. As a bonus, many MUAs
can read it directly.

I've seen JMZ's content-length rant in the past, or something
equivalent; if one documents what is being done an mbox can still
be delivered usefully to the end user. If one doesn't want
Content-Length headers in the archive format, the ">From" hack works
quite well. Conversely, if is perfectly possible to create valid
Content-Length heders when constructing the archive.

We're talking about an archive/transfer format here, not whatever
raw format one works with locally.

> If you want something that could actually have a MIME-Type, there's
> Babyl, MMDF, or (file-per-message) maildir, all of which have
> reasonably precise specs -- but aren't in anywhere near as wide use.

I meant the BSD mailbox format, often called mbox. I know there are
several minor variants, but if mailman (or whatever subcomponent)
documents its chosen flavour that's perfectly tractable.

> Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's
> far more robust and easier to handle than mbox.

Maildir is great as a storage platform; I use it myself for many
folders. But as a delivery format for end users one then has to
pick some wrapper. Zip or tar or... mbox!

>  > I still content that any change that removes the simple ability to
>  > fetch the list archive for use by the end user is a huge step
>  > backwards, regardless of the prettiness of whatever replaces it.
> 
> It's not a question of "removing".  This has always been a user
> choice, and many lists use third-party services (mail-archive.com) or
> non-pipermail archivers (MHonArc).  It's just that Mailman 2 with its
> bundled mbox-format archiver made it easy to provide by default.

Effectively it is removal. If the python lists moved to mailman 3
and suddenly didn't have a "download the list archive", from an end
user point of view that is removal.

> If lots of users really prefer pipermail, for whatever reason, it's
> not that hard to add by hand, and could be packaged on PyPI.

Pipermail has two sides to it: the archive bundles and the message
browser. Plenty of things do a richer message browse (MHonArc seemed
quite nice when last I looked, many years ago).

I'm not wedded to pipermail; from a browsing point of view this new
thing may do well. And I've seen plenty of other simple things on
the net that expose the discussion threads nicely.

I'm wedded to having the archives available in a simple and easy
to fetch form, in a format that is easily imported by many MUAs.
Therefore: mbox.

When Google Groups became a popular site to host mailing lists I
became aghast at how crude and forumlike the interface is/was, and
how the historical list archive was effectively unavailable.  When
Google Groups sucked in the usenet archive tapes I was full of hope,
but I don't see those archives available for download either.

We have all chosen our preferred tool to read lists (be it in "mail"
or "usenet" forms); not being able to access old messages the same
way as current messages feels... shortsighted.


Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Archiving the net is like washing toilet paper! - Leader Kibo

From richard.prosser at mail.com  Mon Mar 31 14:09:48 2014
From: richard.prosser at mail.com (Richard Prosser)
Date: Mon, 31 Mar 2014 13:09:48 +0100
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
Message-ID: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>

I agree - links to search tools or at least explanations of how to use them
would be very welcome.

Better still I believe, abandon mailing lists altogether (because they can
clog up mail clients) and use forum/BB instead.


Richard


On 30 March 2014 05:02, Andrew Barnert <abarnert at yahoo.com> wrote:

> From: Terry Reedy <tjreedy at udel.edu>
>
> Sent: Saturday, March 29, 2014 3:08 PM
>
>
> > On 3/29/2014 4:54 PM, Ben Finney wrote:
> >>  Richard Prosser <richard.prosser at mail.com> writes:
> >>
> >>>  I wanted to search this list (and other Python ones) but it is
> > practically
> >>>  impossible to do so.
> >>
> >>  The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas>
> >>  has all its messages archived online for web browsing at
> >>  <URL:http://mail.python.org/pipermail/python-ideas/>.
> >>
> >>  So you can use any web search tool to search it. A good search engine
> >>  like DuckDuckGo will let you constrain the search to a particular
> subset
> >>  of the web:
> >>
> >>       site:http://mail.python.org/pipermail/python-ideas foo bar baz
> >>
> >>>  So why not add a tool like http://markmail.org/search/?q=python to
> >>>  https://mail.python.org/mailman/listinfo/python-ideas or the like?
> >>
> >>  We have generally-appicable services that index the web and make much
> >>  better search tools. I don't often find a site's own custom search
> > tool
> >>  to be sufficiently better to use it, when DuckDuckGo is available.
> >
> > This list and many others are also available through, archived at, and
> > searchable at news.gmane.org.
>
>
> How hard would it be to add appropriate links (or, better, custom search
> boxes, or, worse but still helpful, a paragraph explaining how to search
> each of those manually) to the mailing list archive pages?
>
> For example, if you link to this:
>
>
> http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas
>
>
> ... then people just need to fill in their search terms on the Advanced
> Search page that comes up.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/3c5e4303/attachment.html>

From phd at phdru.name  Mon Mar 31 14:23:56 2014
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 31 Mar 2014 14:23:56 +0200
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
 <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
Message-ID: <20140331122356.GA8976@phdru.name>

On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser <richard.prosser at mail.com> wrote:
> Better still I believe, abandon mailing lists altogether (because they can
> clog up mail clients) and use forum/BB instead.

   OMG, please, no! Mailing lists are certainly much better than any
forum.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From breamoreboy at yahoo.co.uk  Mon Mar 31 14:51:05 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 31 Mar 2014 13:51:05 +0100
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
 <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
Message-ID: <lhbobr$2mm$1@ger.gmane.org>

On 31/03/2014 13:09, Richard Prosser wrote:
>
> Better still I believe, abandon mailing lists altogether (because they
> can clog up mail clients) and use forum/BB instead.
>

To avoid clogging up your mail client I suggest gmane.comp.python.xyz, 
where in this case xyz is ideas.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com



From barry at python.org  Mon Mar 31 19:35:57 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Mar 2014 13:35:57 -0400
Subject: [Python-ideas] Browser for mailing lists
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
Message-ID: <20140331133557.7508e3d3@anarchist.wooz.org>

On Mar 29, 2014, at 03:29 PM, Richard Prosser wrote:

>I wanted to search this list (and other Python ones) but it is practically
>impossible to do so.

Gmane is my personal favorite.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/0404385b/attachment.sig>

From barry at python.org  Mon Mar 31 19:44:52 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Mar 2014 13:44:52 -0400
Subject: [Python-ideas] Browser for mailing lists
References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
 <20140330222234.GA14300@cskk.homeip.net>
 <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
Message-ID: <20140331134452.55a67550@anarchist.wooz.org>

Discussing specifics of Mailman 3 and archives is off-topic for this list, so
I won't post a huge amount of detail.  Everyone is welcome to come to the
mailman-developers list to further interact with us.

On Mar 31, 2014, at 08:37 AM, Nick Coghlan wrote:

>The pluggable archiver model in MM3 means HyperKitty itself doesn't need to
>handle that usage model. However, I'm not aware of any current efforts to
>create an MM3 archiver that is closer to a pure pipermail replacement (the
>workflow you describe strikes me as being incredibly unusual these days, as
>it requires either running your own mail server, or only accessing your
>email from one device)

I would certainly not start from the existing Pipermail code, unless you pine
for the late '90s.

While the possibility exists to allow for downloading the raw archives, we
disable that in Mailman 2 by default because of the feature's abuse for email
harvesters.

My personal favorite combination for drive-by engagements, historical archive
access, convenience, and inbox sanity is the NNTP API for Gmane.  My primary
MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have
to explicitly opt-in to Gmane, once they do, it can import the archive.
Countless times I've subscribed to a mailing list's newsgroup gateway,
interacted with that community for a short time, and then unsubscribed.

The architecture of MM3 allows, and I would someday love, to be able to offer
an NNTP interface to a mailing list's archives. (Gmane does a lot more and we
don't need to duplicate all of it.)  If that's something you'd be interested
in working on, come talk to us!

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/617c4d5c/attachment-0001.sig>

From barry at python.org  Mon Mar 31 19:48:58 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Mar 2014 13:48:58 -0400
Subject: [Python-ideas] Browser for mailing lists
References: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140331055214.GA3830@cskk.homeip.net>
Message-ID: <20140331134858.4781ff08@anarchist.wooz.org>

On Mar 31, 2014, at 04:52 PM, Cameron Simpson wrote:

>I'm wedded to having the archives available in a simple and easy
>to fetch form, in a format that is easily imported by many MUAs.
>Therefore: mbox.

I'm not sure that's really an effective use case.  For example, python-list
raw mbox archives are 156GB.  I'm positive it's highly compressable, but
still, why would you want to download some huge mbox like that?

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/b95463fa/attachment.sig>

From ericsnowcurrently at gmail.com  Mon Mar 31 20:22:57 2014
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Mon, 31 Mar 2014 12:22:57 -0600
Subject: [Python-ideas] Making sys.std* thread-local
In-Reply-To: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com>
References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com>
Message-ID: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com>

On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee at berkeley.edu> wrote:
> The recent discussion about lexical vs. dynamic scoping reminded me of the
> following issue, that plagues e.g. the new contextlib.redirect_stdout
> context manager: the standard file-like objects sys.std* are global instead
> of thread-local, making their manipulation unsafe in multithreaded programs
> (this issue is related to lexical vs dynamic scoping because in a language
> with (optional) dynamic scoping, these objects can be made dynamically
> scoped, thus solving the issue).

How much of a problem is this in practice?  What's your specific use
case?  I can't say that's I've ever had such a need, but then I
haven't used contextlib.redirect_stdout() in a multi-threaded context.
 At that point I'd probably opt for using Python's logging system
rather than sys.std*.

> Of course, changing sys.std* to being thread-local would be backwards
> incompatible, but perhaps some new objects, e.g. sys.thread_local_std*,
> could be added to sys, with print() and related functions using
> sys.thread_local_std* if it is set and sys.std* otherwise.

Regardless of the validity of the proposal, coming up with an API to
support thread-local sys.std* should be doable in a
backward-compatible way, as you imply.  It's just a question of how
much complexity we can justify in the underlying implementation and in
changes to sys.std* (if any).

FYI, for 3.5 I plan on working on two things related to your proposal:

* restructuring the sys module (or something like it);
* adding an API for managing thread-local contexts (a la decimal).

That's not to say that either will find enough support to actually
land, but I will be working on them. :)

-eric

From solipsis at pitrou.net  Mon Mar 31 20:27:03 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 31 Mar 2014 20:27:03 +0200
Subject: [Python-ideas] Browser for mailing lists
References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
 <20140330222234.GA14300@cskk.homeip.net>
 <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
 <20140331134452.55a67550@anarchist.wooz.org>
Message-ID: <20140331202703.36e33eb3@fsol>

On Mon, 31 Mar 2014 13:44:52 -0400
Barry Warsaw <barry at python.org> wrote:
> Discussing specifics of Mailman 3 and archives is off-topic for this list, so
> I won't post a huge amount of detail.  Everyone is welcome to come to the
> mailman-developers list to further interact with us.

I understand that, but is it actually required for simple feedback as
a mere Web user?

> My personal favorite combination for drive-by engagements, historical archive
> access, convenience, and inbox sanity is the NNTP API for Gmane.  My primary
> MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have
> to explicitly opt-in to Gmane, once they do, it can import the archive.

I also use claws-mail with Gmane NNTP, and I like it.
However, it's not really adequate for casual browsing of mailing-lists
you don't usually read, or for browsing of private mailing-lists with
protected archives, or for giving out hyperlinks to individual
mailing-list messages.

About Hyperkitty, I have to say that the following kind of UI, while
seemingly "nifty", is IMHO a massive step back from pipermail's ease of
use:
https://lists.stg.fedoraproject.org/archives/list/389-devel at lists.fedoraproject.org/

pipermail's dead simple threaded view is, ironically, the best UI I've
ever seen for *browsing* mailing-lists (Google Groups are a disaster in
that regard, and Hyperkitty seems to be heading in the same direction
as Google Groups). I would really hate to have to deal with
Hyperkitty's current UI on a daily basis.

Regards

Antoine.



From njs at pobox.com  Mon Mar 31 20:46:11 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 31 Mar 2014 19:46:11 +0100
Subject: [Python-ideas] Making sys.std* thread-local
In-Reply-To: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com>
References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com>
 <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com>
Message-ID: <CAPJVwBm5FT998o3JBOdsOBJtsP89sZSF0am0iTd861xR4M37pA@mail.gmail.com>

On Mon, Mar 31, 2014 at 7:22 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> * adding an API for managing thread-local contexts (a la decimal).

Numpy might be another consumer of such an API, for similar reasons to decimal:
   http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html#numpy.errstate
An interesting constraint from numpy's point of view is that we want
to be able to read the TLS context very quickly (e.g., no allocs) from
C code.

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From barry at python.org  Mon Mar 31 20:52:11 2014
From: barry at python.org (Barry Warsaw)
Date: Mon, 31 Mar 2014 14:52:11 -0400
Subject: [Python-ideas] Browser for mailing lists
References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com>
 <20140330222234.GA14300@cskk.homeip.net>
 <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com>
 <20140331134452.55a67550@anarchist.wooz.org>
 <20140331202703.36e33eb3@fsol>
Message-ID: <20140331145211.32787335@anarchist.wooz.org>

On Mar 31, 2014, at 08:27 PM, Antoine Pitrou wrote:

>I understand that, but is it actually required for simple feedback as
>a mere Web user?

No, but your feedback won't necessarily reach the developers who can do
something about it.

>I also use claws-mail with Gmane NNTP, and I like it.  However, it's not
>really adequate for casual browsing of mailing-lists you don't usually read,
>or for browsing of private mailing-lists with protected archives, or for
>giving out hyperlinks to individual mailing-list messages.

I like it for casual browsing, but general use the web ui for searching.  It
is a little tricky to then refer back to the NNTP interface for (IMO) more
easily catching up on a thread, or responding.

>pipermail's dead simple threaded view is, ironically, the best UI I've
>ever seen for *browsing* mailing-lists (Google Groups are a disaster in
>that regard, and Hyperkitty seems to be heading in the same direction
>as Google Groups). I would really hate to have to deal with
>Hyperkitty's current UI on a daily basis.

I think Pipermail's ui has problems, not the least of which is the arbitrary
splitting of threads across months, f.e.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/ccfe722b/attachment.sig>

From ethan at stoneleaf.us  Mon Mar 31 20:51:21 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 31 Mar 2014 11:51:21 -0700
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
 <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
Message-ID: <5339B929.1040000@stoneleaf.us>

On 03/31/2014 05:09 AM, Richard Prosser wrote:
>
> I agree - links to search tools or at least explanations of how to use them would be very welcome.

+1


> Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.

Even better than that:  use a decent mail client, and exercise one's mouse and keyboard to delete unnecessary context 
when replying to posts.

--
~Ethan~

From phd at phdru.name  Mon Mar 31 23:05:27 2014
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 31 Mar 2014 23:05:27 +0200
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <5339B929.1040000@stoneleaf.us>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
 <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
 <5339B929.1040000@stoneleaf.us>
Message-ID: <20140331210527.GA23523@phdru.name>

On Mon, Mar 31, 2014 at 11:51:21AM -0700, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 03/31/2014 05:09 AM, Richard Prosser wrote:
> >Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead.
> 
> Even better than that:  use a decent mail client, and exercise one's
> mouse and keyboard to delete unnecessary context when replying to
> posts.

   +many

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From antony.lee at berkeley.edu  Mon Mar 31 23:15:16 2014
From: antony.lee at berkeley.edu (Antony Lee)
Date: Mon, 31 Mar 2014 14:15:16 -0700
Subject: [Python-ideas] Making sys.std* thread-local
In-Reply-To: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com>
References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com>
 <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com>
Message-ID: <CAGRr6BE9VjXB68QvoOrOjfC0XfBtbuA-2A-qCZH9m2R65mojJA@mail.gmail.com>

My specific use case came from embedding the IPython QtConsole (
http://ipython.org/ipython-doc/stable/interactive/qtconsole.html) into a Qt
app.  Specifically, when a user types "print('foo')" into the QtConsole, I
expect 'foo' to be printed in the QtConsole; however when "print('foo')" is
called from the application code, I expect 'foo' to be printed in
__stdout__ (the original stdout, i.e. the terminal).  An in fact, IPython
achieves this through temporary stdout redirection while exec'ing user code
(
https://github.com/ipython/ipython/blob/master/IPython/kernel/inprocess/ipkernel.py#L113
).
Obviously, this breaks down if application code tries to call print while
redirection is taking place; more subtly, if user code (that is, typed by
the user at the QtConsole) starts a new thread, then it would be nice if
that new thread could inherit the current settings (that is, stdout being
redirected to print to the QtConsole), which is not the case right now
(delayed prints go to the terminal as the global stdout gets restored to
__stdout__, not to the QtConsole).
Perhaps a bit specific, but that was the original motivation for this
request.

Antony

2014-03-31 11:22 GMT-07:00 Eric Snow <ericsnowcurrently at gmail.com>:

> On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee at berkeley.edu>
> wrote:
> > The recent discussion about lexical vs. dynamic scoping reminded me of
> the
> > following issue, that plagues e.g. the new contextlib.redirect_stdout
> > context manager: the standard file-like objects sys.std* are global
> instead
> > of thread-local, making their manipulation unsafe in multithreaded
> programs
> > (this issue is related to lexical vs dynamic scoping because in a
> language
> > with (optional) dynamic scoping, these objects can be made dynamically
> > scoped, thus solving the issue).
>
> How much of a problem is this in practice?  What's your specific use
> case?  I can't say that's I've ever had such a need, but then I
> haven't used contextlib.redirect_stdout() in a multi-threaded context.
>  At that point I'd probably opt for using Python's logging system
> rather than sys.std*.
>
> > Of course, changing sys.std* to being thread-local would be backwards
> > incompatible, but perhaps some new objects, e.g. sys.thread_local_std*,
> > could be added to sys, with print() and related functions using
> > sys.thread_local_std* if it is set and sys.std* otherwise.
>
> Regardless of the validity of the proposal, coming up with an API to
> support thread-local sys.std* should be doable in a
> backward-compatible way, as you imply.  It's just a question of how
> much complexity we can justify in the underlying implementation and in
> changes to sys.std* (if any).
>
> FYI, for 3.5 I plan on working on two things related to your proposal:
>
> * restructuring the sys module (or something like it);
> * adding an API for managing thread-local contexts (a la decimal).
>
> That's not to say that either will find enough support to actually
> land, but I will be working on them. :)
>
> -eric
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/6674e395/attachment.html>

From ncoghlan at gmail.com  Mon Mar 31 23:51:39 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 1 Apr 2014 07:51:39 +1000
Subject: [Python-ideas] Browser for mailing lists
In-Reply-To: <5339B929.1040000@stoneleaf.us>
References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com>
 <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org>
 <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com>
 <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com>
 <5339B929.1040000@stoneleaf.us>
Message-ID: <CADiSq7fLMKVrAVQgd-Py320=QRZ=mE8eEW74+6SfYAWxHEMhHA@mail.gmail.com>

On 1 Apr 2014 05:40, "Ethan Furman" <ethan at stoneleaf.us> wrote:
>
> On 03/31/2014 05:09 AM, Richard Prosser wrote:
>> Better still I believe, abandon mailing lists altogether (because they
can clog up mail clients) and use forum/BB instead.
>
>
> Even better than that:  use a decent mail client, and exercise one's
mouse and keyboard to delete unnecessary context when replying to posts.

No need to get personal folks. I use a lot of both mailing lists *and* web
based communication interfaces, and see a place for both of them.

Mailing lists, I prefer when I'm highly invested. Just as an RSS reader
brings blogs I care about into a single interface, mailing lists do the
same for collaborative communication. Appropriate use of filters and "mark
all as read" then deals with the volume of traffic.

However, mailing lists are *terrible* if you just want to drop in, ask one
question & leave again. One possible answer to that is AskBot (a
self-hosted site-specific, Stack Overflow style Q&A forum), but the
MM3/HyperKitty web gateway idea is aimed at making the existing *already
invested* list participants more readily available to new users without
having to ask those existing participants to change their reading habits
and look in a new place for questions.

In the meantime, making GMane more discoverable by suggesting it as an
alternative to subscription in the various list descriptions sounds
reasonable to me.

Cheers,
Nick.

>
> --
> ~Ethan~
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140401/8dcc5a60/attachment.html>