From greg.ewing at canterbury.ac.nz Sat Mar 1 00:34:05 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 01 Mar 2014 12:34:05 +1300 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> Message-ID: <53111CED.2050801@canterbury.ac.nz> Chris Angelico wrote: > With this proposal, your > star_lambda function's declaration changes the call site - instead of > evaluating a*b+c, it has to construct an anonymous function and pass > it along. And when you consider that this could happen with any argument to any function, depending on what the function turns out to be like at run time, it means that *all* function arguments would need to be passes as anonymous functions. That would be very awkward an inefficient. -- Greg From ron3200 at gmail.com Sat Mar 1 03:13:50 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 28 Feb 2014 20:13:50 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> <leqork$it2$1@ger.gmane.org> <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com> Message-ID: <lerfog$9ej$1@ger.gmane.org> On 02/28/2014 04:33 PM, Nick Coghlan wrote: > On 1 Mar 2014 05:43, "Ron Adam" > <ron3200 at gmail.com > <mailto:ron3200 at gmail.com>> wrote: > > > > > > > > On 02/28/2014 11:54 AM, Chris Angelico wrote: > >> > >> On Sat, Mar 1, 2014 at 4:17 AM, Ron > Adam<ron3200 at gmail.com > <mailto:ron3200 at gmail.com>> wrote: > >>> > >>> >This returns a lambda-like function. > >>> > > >>> > def star_lambda(***expr): return expr > >>> > > >>> > > >>> >And is used this way... > >>> > > >>> > result = star_lambda(a * b + c) # captures expression. > >>> > > >>> > actual_result = ***result # *** resolves "result" here! > >>> > > >> > >> Interesting, but I don't like the way the interpretation of a function > >> call depends on the target function. With both * and ** notations, > >> there's absolutely no difference: the function is called with these > >> positional and those keyword arguments, whether they came from actual > >> args or from * or ** unpack/repacks;and there's no difference between > > > > > a function that collects args with *args,**kwargs and one that > > > collects them with individual names (or a C-level function that might > > > do something altogether different). > > > > > > It's not clear what differences you mean here... can you show some examples? > > Remember that at compile time, Python has *no idea* what the actual > signature of the target function is. Thus, all Python function calls use > the following sequence (ignoring optimisations of special cases): > > 1. At the call site, the arguments are collected into a tuple of positional > arguments and a dict of keyword arguments. > 2. The interpreter hands that tuple and dict over to the target callable > 3. The *target callable* then maps the supplied arguments to the defined > parameters including filling in any default values. > > Any function related proposals need to account for the fact that from the > compiler's point of view *every* function signature looks like "(*args, > **kwds)" (although it may have optimised paths for the no-args case and the > positional-args-only case), and that the target callable may not even be > written in Python. So functions can't be extended to take a triplet instead of a pair... (*args, **kwds, ***expr) Looking up... I think this is what you're referring to in ceval.c. #------------------- /* External interface to call any callable object. The arg must be a tuple or NULL. The kw must be a dict or NULL. */ PyObject * PyEval_CallObjectWithKeywords(PyObject *func, PyObject *arg, PyObject *kw) { #-------------------- And it wouldn't work in the normal case any way, as expressions are evaluated as they are put on the stack before calling the function. Somehow I was thinking this morning the code inside the function call parentheses f(...), could be parsed later than it actually is. And in the context of the function definition, possibly similar to how a comprehension is evaluated. But it would take some pretty big changes to do that I supose. Cheers, Ron From rosuav at gmail.com Sat Mar 1 03:14:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Mar 2014 13:14:52 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <leqork$it2$1@ger.gmane.org> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> <leqork$it2$1@ger.gmane.org> Message-ID: <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com> On Sat, Mar 1, 2014 at 6:42 AM, Ron Adam <ron3200 at gmail.com> wrote: > It's not clear what differences you mean here... can you show some examples? These are exactly the same: x = (1,2) f(*x) f(1,2) I played with dis.dis() and it seems there are special-case opcodes for calling a function with a variable number of arguments and/or with variable keyword args; but once it arrives at the other side, the two are identical: >>> def f(*args): print(args) >>> f(1,2) (1, 2) >>> x=(1,2) >>> f(*x) (1, 2) The runtime fetches some callable, gives it some args, and says "Go do your stuff!". It doesn't care what that callable is - it could be a classic function defined with 'def' or 'lambda', it could be a type, it could be an object with __call__, it could be a built-in that's backed by a C function, anything. All that ends up arriving on the other side is: You have these positional args and these keyword args. Adding the tri-star to the mix suddenly changes that. A function is now capable of taking an expression, rather than an object. That's completely different, and it depends on the called function to distinguish one from the other. ChrisA From rosuav at gmail.com Sat Mar 1 03:17:10 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Mar 2014 13:17:10 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <lerfog$9ej$1@ger.gmane.org> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> <leqork$it2$1@ger.gmane.org> <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com> <lerfog$9ej$1@ger.gmane.org> Message-ID: <CAPTjJmrvQUihCG6kuPA3Y_9mzH5O3PqH5VKHk1dQC1Ahdzzt0Q@mail.gmail.com> On Sat, Mar 1, 2014 at 1:13 PM, Ron Adam <ron3200 at gmail.com> wrote: > Somehow I was thinking this morning the code inside the function call > parentheses f(...), could be parsed later than it actually is. And in the > context of the function definition, possibly similar to how a comprehension > is evaluated. But it would take The best way to do it would be to adorn the call site. We can currently do that with lambda: f(a * b + c) f(lambda: a * b + c) Those are completely different from each other, but completely consistent with themselves - it doesn't matter what f is, one of them passes the sum of the product and the other passes a callable. ChrisA From ian.team.python at gmail.com Sat Mar 1 04:09:02 2014 From: ian.team.python at gmail.com (ian o) Date: Fri, 28 Feb 2014 19:09:02 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> Message-ID: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> > > And we don't actually mind all that much if applications don't migrate > in the near term - Python 2 will have commercial support available > well past 2020. (The developers of *those applications* might mind, > though, just as anyone maintaining Python 2.4 compatibility for the > benefit of RHEL/CentOS 5 typically isn't happy about it) > > > Your response is most dissapointing, and I would hope does not represent the community overall. We have 3 teams working server systems currently with python as the implementation model. All work is currently in python 2.x and with a solution to this the work could immediately move to 3.x. However, you state that our situation, and that of the other 60% of python programmers as at end January 2014 who are constrained from using python 3 by python 2 dependencies: "we don't mind about those developers" And of course the dependent library module developers will not update the library, because 'no one out there uses python 3.x'. And from your perspective this is simply not of any interest. No wonder Tiobe index shows python dropping and in the words of those trying to persuade me to move the projects to another language 'python 3.x' came out 5 years ago and we still cannot move to it, this is going no where. I would like to still have the developers work in python, and to do that there needs to be a path to python 3. I also would prefer to be in python 3 anyway. But even if a workable solution is there, your answer is who cares about 60% of developers anyway! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/d60d0773/attachment.html> From steve at pearwood.info Sat Mar 1 04:46:11 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 1 Mar 2014 14:46:11 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <leqgah$b0o$1@ger.gmane.org> References: <leqgah$b0o$1@ger.gmane.org> Message-ID: <20140301034611.GD28804@ando> On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote: > > Starting new thread because this bike has a different shape and color. > > Yesterday I was thinking that just making the keyword lambda assignable > like True, False, and None, would be enough. You can't assign to True, False or None. (You can assign to True and False in Python 2, but shouldn't.) [...] > This morning I thought we could have in a functions definition something, > like "*", and "**", to take an expression. Similar to Nicks idea with =:, > but more general. > > The idea is to have "***" used in def mean to take "any" call expression > and not evaluate it until *** is used on it. [...] > A function call that captures an expression may be tricky to do. Here's one > approach that requires sugar when a function defined with "***" is called. I think it would be useful to have a way to delay execution of an expression, that is to say, have a way to capture an expression for later evaluation, something more lightweight than a function, but I don't think that limiting it to inside function calls is the right approach. Something perhaps like a thunk might be appropriate? We can *almost* do that now, since Python has a compile function: thunk = compile("x + 3", "", "eval") # much later eval(thunk) Some problems with that approach: - You have to write the expression as a string, which means you lose any possibility of syntax highlighting. - The temptation is to pass some arbitrary untrusted string, which leads to serious security implications. The argument here should be limited to an actual expression, not a string containing an expression which might have come from who knows where. - It should be as lightweight as possible. The actual compilation of the expression should occur at compile-time, not run-time. That implies some sort of syntax for making thunks, rather than a function call. - Likewise actually evaluating the thunk should be really lightweight, which may rule out a function call to eval. - How should scoping work? I can see use-cases for flat scoping, static scoping, dynamic scoping, and the ability to optionally provide custom globals and locals, but I have no idea how practical any of them would be or what syntax they should use. All of this is pie-in-the-sky at the moment, and not a serious proposal for Python 3.5. -- Steven From ncoghlan at gmail.com Sat Mar 1 04:57:42 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Mar 2014 13:57:42 +1000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> Message-ID: <CADiSq7fo-gDCtZ0NrNkk4BftBgaCTSvR1X8FBs-97AT914WxFA@mail.gmail.com> On 1 March 2014 13:09, ian o <ian.team.python at gmail.com> wrote: >> And we don't actually mind all that much if applications don't migrate >> in the near term - Python 2 will have commercial support available >> well past 2020. (The developers of *those applications* might mind, >> though, just as anyone maintaining Python 2.4 compatibility for the >> benefit of RHEL/CentOS 5 typically isn't happy about it) >> > Your response is most dissapointing, and I would hope does not represent the > community overall. My apologies, I wrote my post assuming you had already read http://docs.python.org/dev/howto/pyporting.html and decided that the existing approaches described there weren't sufficient for your purposes. Those cases where the recommended migration process isn't considered acceptable for some reason (usually non-technical ones) are the cases that I consider to fall into the "it's OK to just keep using Python 2" category. However, I just realised two things: * that version of the guide is currently only in the Python 3.4 docs, and so the page at http://docs.python.org/3/howto/pyporting.html is still displaying the older 3.3 version of the guide (the page in the Python 2.7 docs is similarly outdated). * that guide is currently aimed primarily at library and framework authors, and doesn't explicitly discuss the migration of integrated applications. I have now filed http://bugs.python.org/issue20812 and http://bugs.python.org/issue20813 to address those limitations, but in the meantime, I will provide the missing information here: For developers of integrated applications that currently still have some dependencies on Python 2, the preferred migration path is to use tools like python-modernize or python-future to shift first into the large common subset of Python 2 and Python 3, and then only later switch fully to Python 3. This approach permits application developers to take the following path: 1. Python 2 only (status quo) 2. Python 2/3 compatible on Python 2 (waiting for dependencies) 3. Python 2/3 compatible on Python 3 (dependencies ported or replaced) 4. Python 3 only (drop Python 2 support) Brett Cannon's "caniusepython3" tool (https://pypi.python.org/pypi/caniusepython3/) is designed to automate the dependency analysis to see if all declared dependencies are Python 3 compatible (or have suitable alternatives available). However, if you're using system packages for dependency management, some data transformations will be needed to convert them to a form that the tool understands. My original reply assumed that you were already aware of the details of this preferred approach before posting your proposal, and I now realise that assumption was likely incorrect. Once again, my apologies. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From abarnert at yahoo.com Sat Mar 1 04:58:08 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 28 Feb 2014 19:58:08 -0800 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <lerfog$9ej$1@ger.gmane.org> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> <leqork$it2$1@ger.gmane.org> <CADiSq7d7gFGCugHPBLNKjmYmwAsjsh9JFQ+wU=PsvPQXqtcRJQ@mail.gmail.com> <lerfog$9ej$1@ger.gmane.org> Message-ID: <D88EB01E-48A7-418E-AF5F-3C1895CAA568@yahoo.com> On Feb 28, 2014, at 18:13, Ron Adam <ron3200 at gmail.com> wrote: > So functions can't be extended to take a triplet instead of a pair... > > (*args, **kwds, ***expr) The more you elaborate this, the more this looks like Ruby procs: You get any number of normal arguments, then at most one special kind of callback thing which must come at the end. I'm not sure which use cases this solves. It doesn't work for expressions that need an argument, like a sorting key function. It also doesn't work for pre-existing functions that weren't designed to take ***expr, like Button or takewhile. And wrapping the expression in a call to a forwarding function doesn't seem any less verbose or more readable than just using lambda. Anyway, there are two fundamental problems with anything that doesn't require any syntax at the call site. First, it's not obvious, or even easily discoverable, to the reader when you're passing a quoted expression and when you're passing a value. Second, it's not discoverable to the compiler. Going back to the analogies from other languages again, Lisp gets around this by making functions and macros different things. Among other differences, a macro always gets expressions instead of values, without any need to quote them at the call site. And, as Haoyi Li has pointed our multiple times on the other threads, you can already do the same thing in Python (with MacroPy). From ron3200 at gmail.com Sat Mar 1 05:07:27 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 28 Feb 2014 22:07:27 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <CAPTjJmq3Ywqc9bUeDAemdGbTV8b=Jd1Ko01S2pe91WgNG_rgzg@mail.gmail.com> <leqork$it2$1@ger.gmane.org> <CAPTjJmqP1_m0Z=bVnhsk7ZHM0hY48+bTFYF=14-n=S=+mJ75vQ@mail.gmail.com> Message-ID: <lermdh$9lh$1@ger.gmane.org> On 02/28/2014 08:14 PM, Chris Angelico wrote: > On Sat, Mar 1, 2014 at 6:42 AM, Ron Adam <ron3200 at gmail.com> wrote: >> It's not clear what differences you mean here... can you show some examples? > > These are exactly the same: > > x = (1,2) > f(*x) > > f(1,2) > > I played with dis.dis() and it seems there are special-case opcodes > for calling a function with a variable number of arguments and/or with > variable keyword args; but once it arrives at the other side, the two > are identical: > >>>> def f(*args): > print(args) >>>> f(1,2) > (1, 2) >>>> x=(1,2) >>>> f(*x) > (1, 2) > > The runtime fetches some callable, gives it some args, and says "Go do > your stuff!". It doesn't care what that callable is - it could be a > classic function defined with 'def' or 'lambda', it could be a type, > it could be an object with __call__, it could be a built-in that's > backed by a C function, anything. All that ends up arriving on the > other side is: You have these positional args and these keyword args. > > Adding the tri-star to the mix suddenly changes that. A function is > now capable of taking an expression, rather than an object. That's > completely different, and it depends on the called function to > distinguish one from the other. Right.. To make it work in the way I was thinking would require each function consisting of two parts. One part to build a name-space, and the other for executing the code. Then at call time, the first part could evaluate the expressions inside the function parentheses. This part would be new, and possibly work like a dict comprehension, except more specific to call signatures. Then the constructed name space would be passed directly to the code part for the actual call. Which would be quite different than what happens presently. I've refactored dis once for fun, and also played around with ceval enough to know how this stuff works, but when I haven't looked at the code recently, (like now) I tend to think more in abstract terms which helps with creativity and seeing new ways to do things. (And it is why I like reading this list), But I sometimes misses on the objectivity side... [Note to self... look at the code more.] It comes down to this... "Creativity and Objectivity often don't want to occupy the same space at the same time." also helped test some patches to dis as well. And Cheers, Ron From steve at pearwood.info Sat Mar 1 05:07:40 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 1 Mar 2014 15:07:40 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> Message-ID: <20140301040740.GF28804@ando> On Thu, Feb 27, 2014 at 10:03:42PM -0800, ian o wrote: > Chris, great reply. > > Yes it is not easy. Especially if you wish to make use of python2 modules > painless. > > And I agree wholeheartedly that serious thought is needed on the 'could it > work so well that it actually slow the migration of library modules?'. I think that such a bridge will slow the migration of library modules, since it reduces the need to migrate. As a library author, why should I spend tens or hundreds of hours migrating my library to Python 3 when I can spend 10 minutes adding a note to the library's website telling people to use the bridge? The problem with this suggestion is that it does nothing to encourage library authors to migrate to 3, it simply enables them to delay migrating even longer, possibly for so long that they lose interest altogether. That's library authors. How about the developers who use those libraries? Right now, there is some motivation for people to move to 3: cool new features, and getting cooler by the year. With this suggested bridge, they can migrate to 3 while still remaining dependent on 2. This might seem like a win if you only focus on the short term uptake of 3, but it misses the fact that they're still dependent on 2. They've now *trebled* the number of language dependencies, from Python only to two versions of Python plus the bridge software. Having spent the time and effort to migrate to 3, in a few years they'll have to go through the same pain again to get rid of the dependency on 2 and the bridge. Nobody will want to do that. The pressure on the core developers to keep supporting 2 forever will increase, not decrease. > This is an important debate to have. I suggest if the answer to 'can a > specification be created that will significantly accelerate migration?', > then it is worth the effort to deliver this. I think you are making a classic mistake of technically-minded people: looking for a technical solution to a social problem. The problem with Python 3 uptake is not a technical problem, or at least not *only* a technical problem. It is mostly a social problem: the costs of migrating are greater than the benefits. Python 3 is great, but for most people Python 2 does the job. Another social problem is that the minority of library authors who haven't migrated haven't done so because they personally don't care about Python 3, or they don't have the time or money to spend on it. Adding a bridge doesn't suddenly give them more time or money. For application developers, the social problem is less because of a few libraries and more to do with the supported standard Python on their system. That will change once Linux distros start providing 3 as their standard system Python; out of the big distros, Fedora is about to do so, so things will start changing soon. But perhaps the biggest reason for slow uptake of Python 3 with existing projects is that nobody wants to spend money just to migrate to 3 for the sake of migrating to 3. "Written in Python 3" is not a selling point. Even the availability of security updates is not much of a selling point, although it will become more important when Python 2.7 is end-of-lifed. The reality is that many apps work perfectly well now using Python 2.7, or 2.6, or even 2.4, and they'll continue to work well in five or ten years so there's no real need to update. At the last US PyCon, there was even one fellow still using Python 1.5. It works, he didn't need security updates, so why migrate? *This is not a problem to be solved.* It's okay for people to stick with an old version of Python. There's no rule that says Python has failed if some developers stick to old versions. > As things stand, the popularity of Python has dropped by most measures > since the release of Python 3. Improve the language and lose market share? > I suggest that is all about the pain of the transition. That sounds remarkably like FUD to me. Where is your evidence that the popularity of Python has fallen? Personally, I think all the "language popularity" measures are dubious, or at least need to be treated with considerable skepticism. Take this one: http://langpop.com/ which lists BrainF--- in the 40 most popular languages. Sure it is. The PyPL popularity index here: https://sites.google.com/site/pydatalog/pypl/PyPL-PopularitY-of-Programming-Language shows Python as increasing in popularity. In contrast, TIOBE shows Python as losing popularity -- but so did Java, PHP, C++ and Ruby, and none of them have a major version change. (TIOBE also shows that despite a large number, possibly majority, of VB 6 developers boycotting VB.Net, VB.Net has just entered the top 10 most popular languages.) And then there's CodeEval, which for the third year in a row has found that Python is more popular than Java, C++, C, PHP and Javascript. In fact more popular than Java, C and PHP *together*. http://blog.codeeval.com/codeevalblog/2014 Bless their little white socks, but methinks their methodology is perhaps a tad flawed. I think it is interesting to compare and contrast these different ways of measuring popularity, but then I'm a statistics wonk and I find even flawed statistics interesting. Remember that there is no one true definition of popularity, even if there were there is no accurate way of measuring it, and all of the sites that claim to do so are actually measuring subtley different things. Most importantly, chasing popularity for its own sake is a waste of time. What makes Python Python is not just the syntax and std lib, but also the culture -- we're neither the stultifying corporate culture of "Java shops", nor the wild-west anything goes of cowboy PHP developers, but something unique. (As are other languages, of course. Perl culture is not Haskell culture is not Erlang culture.) In part, people take up languages because they like the culture. Python will never attract the corporate suits who like the Java philosophy, nor will it attract the cowbody web developers who like the PHP way of doing things. Should we try to ape PHP or Java to attract the cowboys or suits? Or should we continue to make Python the best language that *we* like, regardless of what other people prefer? Untimately, apart from winning pissing contests on Slashdot and Reddit, what does it matter if Python is third most popular or ninth most popular? Its not a popularity context where the winner takes all. According to TIOBE, the most popular language, C, has less than 20% share. If you think Python is a failure because it is in position 8, what does that make Javascript in position 9? > You do not have to search hard to find comments to the effect 'nobody is > using python3'. Or 'even if you use python 3, do not use the new features > since you need to ensure code is compatible with both versions'. You don't have to search very hard to find comments that Elvis was kidnapped by aliens. https://duckduckgo.com/html/?q=elvis+kidnapped+by+aliens A belief which has rather more going for it than either of the two comments you list above, since there's a miniscule, microscopic chance that perhaps Elvis *was* kidnapped by aliens, whereas I know for a fact beyond any doubt that some people are using Python 3 and using the new features. -- Steven From ian.team.python at gmail.com Sat Mar 1 05:10:04 2014 From: ian.team.python at gmail.com (ian o) Date: Fri, 28 Feb 2014 20:10:04 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> Message-ID: <f00701bc-f041-4f65-aa24-c75bc9b835ec@googlegroups.com> > And we don't actually mind all that much if applications don't migrate > in the near term - Python 2 will have commercial support available > well past 2020. (The developers of *those applications* might mind, > though, > ..... > The group we *do* really care about supporting is authors of existing > Python *libraries*, and "2in3" and "3in2" approaches don't really help > them at all, since library and framework authors with users on both > Python 2 and Python 3 will likely face demand to support both versions > natively anyway. Cheers, > Nick. > > Nic, > Lets assume for the moment that the 'we don't really care about the 60% of python developers blocked from moving to python 3 by depencies' was not really what you meant. I am going to assume that what you really meant was 'I do not think helping these people directly is the best solution'. And this attitude is based around the fear that if the developers can move to python 3 before the libraries all support python 3, then that will remove pressure from the libraries to support python 3. This fear i can understand. But I suggest some real world experience shows this approach is actually not assisting either group. It is certainly not assisting the developers wishing to move to python 3, but it is also making life hard for the "people we do really care about" So not supporting a "2in3" solution forces programmers to with python2 dependencies to hold off moving to python3 until their dependencies migrate, but hopefully with sound motives. Now take a typical dependency. Web2py. Speak to them on migration and they will telly you 'none of our users have moved to python 3'. So the users are held back from moving by the dependencies, and the dependencies are held back by the users. I would suggest that making it easier to support both python2 and ptyhon3 from a library is simply reducing what it a painful thing. I would suggest that allowing all the end users to migrate to python3 as soon as possible is a far more attractive solution. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/5a908817/attachment.html> From abarnert at yahoo.com Sat Mar 1 06:09:03 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 28 Feb 2014 21:09:03 -0800 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> Message-ID: <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com> On Feb 28, 2014, at 19:09, ian o <ian.team.python at gmail.com> wrote: > However, you state that our situation, and that of the other 60% of python programmers as at end January 2014 who are constrained from using python 3 by python 2 dependencies: "we don't mind about those developers" You're twisting the numbers. I'm part of the 60% who have to work on Python 2.x for at least one project. But first, I have others--including shipping commercial code both on the desktop and the server--using 3.x. And second, in all but one case, the 2.x apps are 2.x not because of lacking library support, but because we have an already-working and -deployed app that's effectively in maintenance mode, and it's not worth the effort or risk to port it to 3.x. (In other words, most of my 2.x work points to the _success_ of the migration plan, not a failure.) And you're also twisting Nick's words. He said that he cares more about getting library developers to 3.x than application developers. And he explained why that makes life better for application developers. To take that as "the Python devs don't care about application developers" is willfully misinterpreting the point. > And of course the dependent library module developers will not update the library, because 'no one out there uses python 3.x'. None of the library devs I've ever dealt with has ever said that (well, not since the early 3.1 days). Most projects that don't support 3.x today want to do so, it just isn't always the very highest thing on their priority list. Three times I've had to port a library myself rather than waiting around--but in every case, the library's dev was happy to take my code, even if it meant losing 2.5 support, reviewing large changelists, etc. I'm sure there are some exceptions, but it's certainly not the ubiquitous case you're presenting. There's a reason the Python 3 Wall of Shame changed it's name to Wall of Superpowers and the competing py3ksupport page stopped updating: because people have been porting important libraries to 3.x, and continue to do so. > No wonder Tiobe index shows python dropping And is the Python 3 migration also to blame for Python's direct competitors (Ruby, Perl, and PHP) also dropping)? Or for the general trend in Tiobe numbers to show ObjC and Windows-specific languages continually gaining on cross-platform languages? > and in the words of those trying to persuade me to move the projects to another language 'python 3.x' came out 5 years ago and we still cannot move to it, this is going no where. This is pure FUD. And I don't see how repeating that FUD to the Python devs is going to help you. If you want to convince these people to stay on Python, you need to start talking specifics instead of talking points. What libraries are stopping you from moving to 3.x? Does Ruby or Node.js or whatever actually have a better alternative for all of those libraries than Python 3 does? Would it really be easier to write those missing libraries yourself in a new language than to port existing Python 2.x libraries to 3.x? For that matter, have you actually checked whether those libraries really are blocking you? I've lost count of the number of people who've claimed they can't switch to Python 3 because of some library that's been ported since 2010 or that has a 3.x-compatible drop-in replacement. From ian.team.python at gmail.com Sat Mar 1 06:53:13 2014 From: ian.team.python at gmail.com (ian o) Date: Fri, 28 Feb 2014 21:53:13 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <20140301040740.GF28804@ando> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> Message-ID: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> Steven, Great post. Arguments can be enjoyed when people with differing views argue them well. Let me try and put the counter case as well. I have to start with the Elvis abducted by aliens just because it is such a great place to start. I enjoyed your point. But let me counter by saying that the information that some people say 'Elvis was abducted by aliens' does not mean this is what actually happened, but it does tell us that some people believe this! And there is rather surprising information in the fact that 'some people believe Elvis was abducted by Aliens'. Similarly I do not believe 'no-one is using python 3'. I program in python3 (and wish i could for all programming). But it is still interesting that people are saying it. Other comments inline below: I think that such a bridge will slow the migration of library modules, > since it reduces the need to migrate. As a library author, why should I > spend tens or hundreds of hours migrating my library to Python 3 when I > can spend 10 minutes adding a note to the library's website telling > people to use the bridge? > > The problem with this suggestion is that it does nothing to encourage > library authors to migrate to 3, it simply enables them to delay > migrating even longer, possibly for so long that they lose interest > altogether. > > I suggest the counter point here is to consider life a library developer. Which motivates you: that fact that people using your library all program in python3, or hearing that all your users program in python2? > That's library authors. How about the developers who use those > libraries? > > Right now, there is some motivation for people to move to 3: cool new > features, and getting cooler by the year. With this suggested bridge, > they can migrate to 3 while still remaining dependent on 2. This might > seem like a win if you only focus on the short term uptake of 3, but it > misses the fact that they're still dependent on 2. They've now *trebled* > the number of language dependencies, from Python only to two versions of > Python plus the bridge software. > > Are you assuming that it will always be either a) all dependencies require python2 or b) all dependencies are available in python 3. Rather than c) some dependencies are available in python3 but the python3 version cannot be used because I am being held back to python2 by some other critical dependencies. For the projects i work on, it is c). It is not like there has been no success getting packages to migrate. The problem is that an entire project is held back by one single dependency. And for us we would wear some pain right now and make that move. Most of the project doesn't have tow steps and can be moved immediately. If a bridge is offered, it would not be a great idea to force everyone to use it. If EVERY imported module is still in python2 then I suggest it is too early to move. Demonstrably, some people like the idea, and I suggest it would be interesting to discover how many. There may be cases where it is not the right choice because too many dependencies are only available in python3 , but alone that does not mean the choice should not be offered. Personally I beleive that now many dependencies can be found in python3, hence the frustration of having to wait until every single one moves seems unfortunate. Having spent the time and effort to migrate to 3, in a few years they'll > have to go through the same pain again to get rid of the dependency on 2 > and the bridge. Nobody will want to do that. The pressure on the core > developers to keep supporting 2 forever will increase, not decrease. > > I cannot see that getting rid of the bridge would be that difficult, particularly if it is not for many modules. The real pain is living in world where there is so much holding everything back to the older version. Again, let me declare the self interest here. We have development teams and projects we wish to move to python3. A bridge like this would allow us to move now, and that would allow pressure on the module providers. Stopping a step like this on the basis it stops people like us moving to python3 just seems counter productive if the goal is to move to python3. > > > This is an important debate to have. I suggest if the answer to 'can a > > specification be created that will significantly accelerate migration?', > > then it is worth the effort to deliver this. > > I think you are making a classic mistake of technically-minded people: > looking for a technical solution to a social problem. > > The problem with Python 3 uptake is not a technical problem, or at least > not *only* a technical problem. It is mostly a social problem: the costs > of migrating are greater than the benefits. Python 3 is great, but for > most people Python 2 does the job. > > I am a person experiencing the problem. And our experience is that our core developers use code supplied by a third party who does not produce a python3 version. So the core has to be done in python2. The code produced by these core developers is then supplied to other companies - forcing them to stay in python2. It is dependency hell. One module anywhere down the chain that cannot be moved and the whole project is forced to stay python2. > Another social problem is that the minority of library authors who > haven't migrated haven't done so because they personally don't care > about Python 3, or they don't have the time or money to spend on it. > Adding a bridge doesn't suddenly give them more time or money. > Agreed. But in the real world, an example of a major dependency requiring python2, web2py, states they will be motivated to python3 when the users of their framework move. But this policy of forcing all dependencies to move before the end users move means this will never happen. What it does mean is many of their customers can move. And it is even possible for them to ask their holdout customers to either stay with the old version or move, since these customers are no longer held back by other dependencies. For application developers, the social problem is less because of a few > libraries and more to do with the supported standard Python on their > system. That will change once Linux distros start providing 3 as their > standard system Python; out of the big distros, Fedora is about to do > so, so things will start changing soon. > > But perhaps the biggest reason for slow uptake of Python 3 with existing > projects is that nobody wants to spend money just to migrate to 3 for > the sake of migrating to 3. "Written in Python 3" is not a selling > point. Even the availability of security updates is not much of a > selling point, although it will become more important when Python 2.7 is > end-of-lifed. The reality is that many apps work perfectly well now > using Python 2.7, or 2.6, or even 2.4, and they'll continue to work well > in five or ten years so there's no real need to update. At the last US > PyCon, there was even one fellow still using Python 1.5. It works, he > didn't need security updates, so why migrate? > > Perhaps the biggest reason ..... Why not ask? Well, python.org conducted a survey as of January 2014 an answer given was that 60% of developers were held back from moving projects because of dependencies. This is a big number and it is not just guessing but real figures. *This is not a problem to be solved.* It's okay for people to stick with > an old version of Python. There's no rule that says Python has failed if > some developers stick to old versions. > > OK. So you are telling us forget moving our developers to python3 until every last dependency has moved. And we should also force the people who in turn use our code to stay with python2 and it is not a problem. > > > As things stand, the popularity of Python has dropped by most measures > > since the release of Python 3. Improve the language and lose market > share? > > I suggest that is all about the pain of the transition. > > That sounds remarkably like FUD to me. Where is your evidence that the > popularity of Python has fallen? > > Personally, I think all the "language popularity" measures are dubious, > or at least need to be treated with considerable skepticism. Take this > one: > > http://langpop.com/ > > which lists BrainF--- in the 40 most popular languages. Sure it is. > > The PyPL popularity index here: > > > https://sites.google.com/site/pydatalog/pypl/PyPL-PopularitY-of-Programming-Language<https://www.google.com/url?q=https%3A%2F%2Fsites.google.com%2Fsite%2Fpydatalog%2Fpypl%2FPyPL-PopularitY-of-Programming-Language&sa=D&sntz=1&usg=AFQjCNEJ__tv-fiB_BTXtt3qu6Y_UD49kg> > > Thanks for that- useful link! Please remember I am the converted, so it helps to have arguments to push back with when others push tiobe in front of me. > shows Python as increasing in popularity. In contrast, TIOBE shows > Python as losing popularity -- but so did Java, PHP, C++ and Ruby, and > none of them have a major version change. (TIOBE also shows that despite > a large number, possibly majority, of VB 6 developers boycotting VB.Net, > VB.Net has just entered the top 10 most popular languages.) > > And then there's CodeEval, which for the third year in a row has found > that Python is more popular than Java, C++, C, PHP and Javascript. In > fact more popular than Java, C and PHP *together*. > > http://blog.codeeval.com/codeevalblog/2014<http://www.google.com/url?q=http%3A%2F%2Fblog.codeeval.com%2Fcodeevalblog%2F2014&sa=D&sntz=1&usg=AFQjCNEcIR2Zkg32DfKkhns8eq7KVgFJ1Q> > > Perhaps overstated, but it helps to have sources like this. It does say coding tests and challenges. I live in a country where school children have one national coding contest/challenge, and it is in python. (And in python 3, no dependencies :) ) Bless their little white socks, but methinks their methodology is > perhaps a tad flawed. > > I think it is interesting to compare and contrast these different ways > of measuring popularity, but then I'm a statistics wonk and I find even > flawed statistics interesting. Remember that there is no one true > definition of popularity, even if there were there is no accurate way of > measuring it, and all of the sites that claim to do so are actually > measuring subtley different things. > > Most importantly, chasing popularity for its own sake is a waste of > time. What makes Python Python is not just the syntax and std lib, but > also the culture -- we're neither the stultifying corporate culture of > "Java shops", nor the wild-west anything goes of cowboy PHP developers, > but something unique. (As are other languages, of course. Perl culture > is not Haskell culture is not Erlang culture.) In part, people take up > languages because they like the culture. Python will never attract the > corporate suits who like the Java philosophy, nor will it attract the > cowbody web developers who like the PHP way of doing things. Should we > try to ape PHP or Java to attract the cowboys or suits? Or should we > continue to make Python the best language that *we* like, regardless of > what other people prefer? > All agreed on the culture. But i suggest this is the biggest point. Stopping measures like a bridge that would allow many more end users to code in python3 just seems a negative culture. Effectively you are telling us that our team should not move to python3. > > Untimately, apart from winning pissing contests on Slashdot and Reddit, > what does it matter if Python is third most popular or ninth most > popular? Its not a popularity context where the winner takes all. > According to TIOBE, the most popular language, C, has less than 20% > share. If you think Python is a failure because it is in position 8, > what does that make Javascript in position 9? > > > > > You do not have to search hard to find comments to the effect 'nobody is > > using python3'. Or 'even if you use python 3, do not use the new > features > > since you need to ensure code is compatible with both versions'. > > You don't have to search very hard to find comments that Elvis was > kidnapped by aliens. > > https://duckduckgo.com/html/?q=elvis+kidnapped+by+aliens > > A belief which has rather more going for it than either of the two > comments you list above, since there's a miniscule, microscopic chance > that perhaps Elvis *was* kidnapped by aliens, whereas I know for a fact > beyond any doubt that some people are using Python 3 and using the new > features. > > I think I already covered this one. Again the point is that some people say it. We both know it is not literally true. If i did not program in python3 I would not bother posting suggestions to try and allow more programming in python3. Ian > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python... at python.org <javascript:> > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > Steven, -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140228/c5c0e15b/attachment-0001.html> From ron3200 at gmail.com Sat Mar 1 07:07:14 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 01 Mar 2014 00:07:14 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140301034611.GD28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> Message-ID: <lerte5$4bn$1@ger.gmane.org> On 02/28/2014 09:46 PM, Steven D'Aprano wrote: > On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote: >> > >> >Starting new thread because this bike has a different shape and color. >> > >> >Yesterday I was thinking that just making the keyword lambda assignable >> >like True, False, and None, would be enough. > You can't assign to True, False or None. (You can assign to True and > False in Python 2, but shouldn't.) I meant it the other way around. For example... def is not assignable to anything.. D = def # won't work T = True # works F = False N = None L = lambda # won't work lambda isn't an object like True, False, and None. > [...] >> >This morning I thought we could have in a functions definition something, >> >like "*", and "**", to take an expression. Similar to Nicks idea with =:, >> >but more general. >> > >> >The idea is to have "***" used in def mean to take "any" call expression >> >and not evaluate it until *** is used on it. > [...] >> >A function call that captures an expression may be tricky to do. Here's one >> >approach that requires sugar when a function defined with "***" is called. > I think it would be useful to have a way to delay execution of an > expression, that is to say, have a way to capture an expression for > later evaluation, something more lightweight than a function, but I > don't think that limiting it to inside function calls is the right > approach. The expressing part isn't limited to inside function calls.. Or wouldn't be if it was a doable idea. > Something perhaps like a thunk might be appropriate? We can*almost* do > that now, since Python has a compile function: > > thunk = compile("x + 3", "", "eval") > # much later > eval(thunk) def thunk(***expr): return expr def do thunks(*args): for expr in args: ***expr start = t = time() update_timer = thunk(t = time()) show_timer = thunk(print(t-start)) show_status = thunk(do_thunks(upate_timer, show_timer)) Then as you do things, possibly in different functions as well. ... ***show_status # update t, and prints elapsed time. ... ***show_status ... ***show_status ... Yes, it could be done with lambda too. Here's where it differs... So if we have this, where obj is nested expressions. While 1: try: obj = ***obj except TypeError: break Then the result of the obj expression, could be a callable without any conflict. Only delayed expressions would be expressed. This was one of the properties I was looking for. > Some problems with that approach: > > - You have to write the expression as a string, which means you lose any > possibility of syntax highlighting. > > - The temptation is to pass some arbitrary untrusted string, which leads > to serious security implications. The argument here should be limited > to an actual expression, not a string containing an expression which > might have come from who knows where. > > - It should be as lightweight as possible. The actual compilation of the > expression should occur at compile-time, not run-time. That implies some > sort of syntax for making thunks, rather than a function call. > > - Likewise actually evaluating the thunk should be really lightweight, > which may rule out a function call to eval. > > - How should scoping work? I can see use-cases for flat scoping, static > scoping, dynamic scoping, and the ability to optionally provide custom > globals and locals, but I have no idea how practical any of them would > be or what syntax they should use. That's where the idea I mentioned in another response to this thread comes in. Separating the namespace constructing from the code part of a function in a well defined way. If that can be done... also pie in the sky. Then we could also evaluate an expression in a previously defined namespace. It's hypothetical, so supply your own syntax for it. The compiler will still compile the code, so most of the work is still done at compile time. But we have these useful pieces we can take apart and put back together again. (without calling eval, or exec.) BTW, you can do that now, but it's very hard to get right. help(type(lambda:1)) > All of this is pie-in-the-sky at the moment, and not a serious proposal > for Python 3.5. Agree. Cheers, Ron From rosuav at gmail.com Sat Mar 1 07:43:50 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Mar 2014 17:43:50 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> Message-ID: <CAPTjJmo2i3uZ=x_LyiB+f2vijAPRS3feA7eh1GosgX26s=wE7Q@mail.gmail.com> On Sat, Mar 1, 2014 at 4:53 PM, ian o <ian.team.python at gmail.com> wrote: > A bridge like this would allow us to move now, and that would allow pressure > on the module providers. Think about what the bridge really means. It means that you move now, yes, but it also means that the existing module is working fine. That means there is *no* pressure on the module provider. The pain of maintaining the bridge is all yours. You want to put pressure on the module provider? *Don't* have the bridge. Then you have an incentive to get that module ported, because then you could migrate. Maybe that means you lean on the developers; maybe that means you run the code through 2to3 and then start dealing with test suite failures, and then submit the code back to them and say "Here's Python 3.3 support, these are the downsides, will you accept it?". That's the sort of pressure that's likely to work. Saying "Hey, we can use your module in Python 3 now, thanks to a Python-in-C-in-Python bridge" will evoke the response "Okay fine, no problem then!". ChrisA From ncoghlan at gmail.com Sat Mar 1 07:55:37 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Mar 2014 16:55:37 +1000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> Message-ID: <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> On 1 March 2014 15:53, ian o <ian.team.python at gmail.com> wrote: > Steven, >> *This is not a problem to be solved.* It's okay for people to stick with >> an old version of Python. There's no rule that says Python has failed if >> some developers stick to old versions. >> > OK. So you are telling us forget moving our developers to python3 until > every last dependency has moved. And we should also force the people who in > turn use our code to stay with python2 and it is not a problem. Ian, please keep in mind that you are asking people to do extra work for you *for free*, after you have already been benefiting for years from work that we already made available to you free of charge. The latter part is OK, and entirely the way non-commercial open source works, but it *does* mean that you *don't* get to tell the core development team our priorities are wrong when you haven't paid us a cent. If that's the kind of relationship you want, pay a commercial vendor to listen to your concerns - mediating between customers and the community one of the services commercial open source vendors provide (and the best ones will then contribute upstream development effort on your behalf). By contrast,community oriented open source operates on a time-based economy - the only ways for people to get things are done to dedicate their own time, inspire others to dedicate their time, or else to pay developers (either directly or indirectly through a vendor) to spend *their* time on projects of interest to those providing the funds. The CPython core development team has made it clear we don't want to support running Python 2 and 3 code in the same process because doing so is *hard* (and perhaps even impossible without completely redesigning the interpreter) and certainly *not fun*. The inspiration based approach is unlikely to work in this case (because we've already considered the possibility and dismissed it as impractical), but that still leaves open the possibility of finding someone else that is interested in attempting to prove us wrong, as well as people finding a way to pay someone to make it happen. However, in both cases, keep in mind that the people *most* familiar with the CPython code base think it's a bad idea that probably won't work. Instead, with the aid of many other members of the community, we've created a source based migration approach that involves taking advantage of the large common subset of Python 2 & 3. If commercial operations higher in the stack would like to migrate to Python 3, but are relying on dependencies that the community has provided for free, then it's time to think seriously about *investing* in the community and helping those projects to migrate (either by contributing developer time directly or by contributing funds to the existing developers for those projects). Note that the Python Software Foundation is able to help mediate requests for assistance with getting dependencies ported. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sat Mar 1 08:26:23 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Mar 2014 17:26:23 +1000 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140301034611.GD28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> Message-ID: <CADiSq7cqGe9uS5YxjTTAXqVoQdc+MVKWpKJ+YPfu+10CnCyBfg@mail.gmail.com> On 1 March 2014 13:46, Steven D'Aprano <steve at pearwood.info> wrote: > > Something perhaps like a thunk might be appropriate? We can *almost* do > that now, since Python has a compile function: > > thunk = compile("x + 3", "", "eval") > # much later > eval(thunk) FWIW, I think this is why PEP 312 (simple implicit lambda) garnered any interest at all: sure, you can't pass such a lambda any arguments, but it works fine as a *closure*. It's also potentially worth trawling the python-ideas archives for the various discussions about deferred and delayed expressions - they're generally all variants of the same basic idea, a less esoteric, easier to learn way to handle one-shot callbacks and other forms of lazy expression evaluation. The "lambda" keyword in the current syntax ends up being a distraction rather than an aid to understanding. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From abarnert at yahoo.com Sat Mar 1 08:44:49 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 28 Feb 2014 23:44:49 -0800 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> Message-ID: <E21B635C-F8C8-4B7D-BD84-6FAF2C6575DD@yahoo.com> On Feb 28, 2014, at 21:53, ian o <ian.team.python at gmail.com> wrote: > Agreed. But in the real world, an example of a major dependency requiring python2, web2py, states they will be motivated to python3 when the users of their framework move. This is an interesting choice for an example. web2py isn't just a library, it's a framework. Its main purpose is to run your code in the right situations. Which is exactly the kind of thing your bridge cannot handle, for the reasons you already gave. And I don't think this is a coincidence. The same thing is true of many of the most prominent packages that haven't been ported yet--Twisted, Paste, gevent, supervisor, etc. On top of that, web2py is all about passing Internet strings around. The same thing that makes it difficult to port--the bytes/Unicode distinction--would make it just as hard to use through the bridge. Anyway, if you're interesting in pursuing this, obviously nobody is going to stop you from doing it and putting it on PyPI. And if there's as much interest in the idea as you think, you should start getting outside contributions as soon as you have a workable beta (or you might even be able to get funded development before that). And if it became popular enough, you could come back to the core dev team and say, "Like it or not, there are tons of major projects relying on my bridge, and the consequences aren't nearly as gloomy as you suspected, so what about putting it in the stdlib?" From ian.team.python at gmail.com Sat Mar 1 09:03:05 2014 From: ian.team.python at gmail.com (ian o) Date: Sat, 1 Mar 2014 00:03:05 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> Message-ID: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> > Ian, please keep in mind that you are asking people to do extra work > for you *for free*, after you have already been benefiting for years > from work that we already made available to you free of charge. The > latter part is OK, and entirely the way non-commercial open source > works, but it *does* mean that you *don't* get to tell the core > development team our priorities are wrong when you haven't paid us a > cent. Nic, Why are you so aggressive? No one is trying to 'tell' people to do something. I was proposing an idea here. I thought it was a relevant useful idea, so i have been arguing the case that idea is useful. Others argue the case that it is not useful and to me this is a constructive way to share ideas, hear other points of view so they can be considered. For commercial purposes you have managed to convince me I should cease to lobby for python to be used in the projects where I assist. Obviously commercial use is clearly not desired by the community, that is the first lesson i have learned. As I am not paid to champion python in the work environment, having been enlightened that python community does not desire commercial usage, I will cease. I came this forum hoping to contribute. I do not get paid to post here, nor will python being used in projects where i am involved generate a financial outcome for me. I simply feel a passion for the language and have been supporting the use of python in education and it helps push case for students learning python when there is also commercial usage. Is educational usage also to be discouraged? So now I am being told that not only are commercial users not to be encouraged, but also opinions and ideas of those who have been involved in commercial projects are not welcome. Disappointing. If a welcome as a result of a debate it was agreed that the approach I feel will help both library module developers and end users and was worth supporting but the community then the decision question would come as to who would donate time to implementation. While i would get no commercial benefit and do not have a lot time i would have tried to help. I think the idea has merit. But what is the point? It seems even debate on desirability of the idea in your opinion is not to be encouraged. From others I get the sense of community. But from you I get the sense of a closed club. Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/a3d261fd/attachment.html> From rosuav at gmail.com Sat Mar 1 09:05:52 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Mar 2014 19:05:52 +1100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> Message-ID: <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com> On Sat, Mar 1, 2014 at 7:03 PM, ian o <ian.team.python at gmail.com> wrote: > Why are you so aggressive? > No one is trying to 'tell' people to do something. > > I was proposing an idea here. If your idea is accepted, someone has to implement it :) That's work, and a lot of it. So unless you're volunteering to actually write this thunker/bridge/shim/etc, you're asking someone else to. ChrisA From abarnert at yahoo.com Sat Mar 1 09:29:02 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 1 Mar 2014 00:29:02 -0800 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140301034611.GD28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> Message-ID: <807BB590-DE68-4E3C-9DC3-F37335DFBFAD@yahoo.com> On Feb 28, 2014, at 19:46, Steven D'Aprano <steve at pearwood.info> wrote: > Something perhaps like a thunk might be appropriate? We can *almost* do > that now, since Python has a compile function: To save Haoyi Li the trouble of saying it: we _can_ do this now, since Python has import hooks, so you can use MacroPy to quote an expression, giving you an AST to evaluate later, or automatically wrapping it up in a function (like quote and function, respectively, in Lisp). There's also an in-between option of compiling it to a code object but not wrapping that in a function object. There's one other crazy option for representing quoted expressions/thunks: as lazy futures. This is how Alice does it, and it's also implicitly what dataflow languages like Oz are doing (every variable is a lazy future). In Python, this would really just be a wrapper around a function call, so I don't think it would but anything. Which one of those do people want? I'm not sure. Since you did a great job listing the problems of the string-quoting solution, we can compare the options for each one. > - You have to write the expression as a string, which means you lose any > possibility of syntax highlighting. Obviously not a problem here. > - The temptation is to pass some arbitrary untrusted string, which leads > to serious security implications. The argument here should be limited > to an actual expression, not a string containing an expression which > might have come from who knows where. Again, not a problem. > - It should be as lightweight as possible. The actual compilation of the > expression should occur at compile-time, not run-time. That implies some > sort of syntax for making thunks, rather than a function call. Not a problem for the function or code versions. For the AST version, you're doing part of the compilation at compile-time, and the rest at runtime. > - Likewise actually evaluating the thunk should be really lightweight, > which may rule out a function call to eval. A function is obviously no better or worse than using lambda today. A code object has to be evaluated by passing it to eval, or wrapping it in a function and calling it. I suspect the former may be lighter weight than calling a function, but I really don't know. The latter, or the other hand, is obviously heavier than calling a function, but not by that much. An AST has to be compiled to a code object, after which you do the same as the above. Obviously this is heavier than not having to compile. > - How should scoping work? I can see use-cases for flat scoping, static > scoping, dynamic scoping, and the ability to optionally provide custom > globals and locals, but I have no idea how practical any of them would > be or what syntax they should use. This, I think, is the big question. A function is clearly a normal, static-scoped closure. An AST or code object, you could do almost any form of scoping you want _except_ static, either by building a function around it or by calling eval on it. (You can do additional tricks with an AST, but I don't think most programs would want to. For statements, this could be useful for optional hygienic variables, but for expressions that isn't an issue.) If we want static scoping, we really need functions. At least we need some kind of object that has some form of code plus a closure mechanism--and that's pretty much all functions are. On the other hand, if we want dynamic, flat, or customizable scoping, we need something we can either build a function object out of dynamically, or evaluate dynamically--and that's pretty much what code objects are. From ian.team.python at gmail.com Sat Mar 1 09:40:58 2014 From: ian.team.python at gmail.com (ian o) Date: Sat, 1 Mar 2014 00:40:58 -0800 (PST) Subject: [Python-ideas] A python bridge between versions In-Reply-To: <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com> Message-ID: <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com> Chris, First the idea has to be accepted as desirable, then feasible with an agreed specification. All to no avail unless someone can then implement it. I would volunteer to assist but i assume there have to be enough others to volunteer as well if it was any help. But there is no intent to tell people to do something. There has to be sufficient enthusiasm for anyone involved to actually want to do it. But I have been told that those who would benefit are people that there is no desire to help. My suggestion that it would also be helpful to people there is a desire to help seems to have been met with the response 'we do not want to hear this'. I guess I expected a more open community. I have learned a lot, but not what I expected. I am still not sure what the criteria is to actually be welcome here. I have very much appreciated your comments though. Ian On Saturday, March 1, 2014 7:05:52 PM UTC+11, Chris Angelico wrote: > > On Sat, Mar 1, 2014 at 7:03 PM, ian o <ian.tea... at gmail.com <javascript:>> > wrote: > > Why are you so aggressive? > > No one is trying to 'tell' people to do something. > > > > I was proposing an idea here. > > If your idea is accepted, someone has to implement it :) > > That's work, and a lot of it. So unless you're volunteering to > actually write this thunker/bridge/shim/etc, you're asking someone > else to. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python... at python.org <javascript:> > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/99b65545/attachment-0001.html> From g.brandl at gmx.net Sat Mar 1 10:11:59 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 01 Mar 2014 10:11:59 +0100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com> <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com> Message-ID: <les86i$37r$1@ger.gmane.org> Am 01.03.2014 09:40, schrieb ian o: > Chris, > > First the idea has to be accepted as desirable, then feasible with an agreed > specification. All to no avail unless someone can then implement it. > > I would volunteer to assist but i assume there have to be enough others to > volunteer as well if it was any help. > > But there is no intent to tell people to do something. There has to be > sufficient enthusiasm for anyone involved to actually want to do it. > > But I have been told that those who would benefit are people that there is no > desire to help. My suggestion that it would also be helpful to people there is > a desire to help seems to have been met with the response 'we do not want to > hear this'. > > I guess I expected a more open community. I have learned a lot, but not what I > expected. I am still not sure what the criteria is to actually be welcome here. Hi Ian, I'm sorry you feel like this. It's definitely not the intention of this list to exclude people -- just look at the wild ideas being discussed all the time... The Python 2/3 topic is simply a "sore point" for many of us, having discussed it time and again, often times with quite aggressive people (with the "why have you ruined my Python" mindset). Therefore we tend to become a little categorical faster than with other topics, even to people like you who aren't being aggressive. I hope you can forgive Nick, who has put tons of work into the excellent 2 vs 3 FAQ, being a bit thin-skinned :) In other words: Welcome to the Python community - don't let the 16-ton weight scare you off! cheers, Georg From ncoghlan at gmail.com Sat Mar 1 10:46:57 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 1 Mar 2014 19:46:57 +1000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <les86i$37r$1@ger.gmane.org> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> <CAPTjJmrGzJsDtNXKj7aEQRqKBEYaJs_+80AMuDihELqXxhWmFQ@mail.gmail.com> <d2ef3081-184a-4fbe-978b-86460f8ea6d1@googlegroups.com> <les86i$37r$1@ger.gmane.org> Message-ID: <CADiSq7dTeUgv-b48YpEkwEWBC8e+50UyWFD8ucPLW=_dyOtUAA@mail.gmail.com> On 1 March 2014 19:11, Georg Brandl <g.brandl at gmx.net> wrote: > Am 01.03.2014 09:40, schrieb ian o: >> I guess I expected a more open community. I have learned a lot, but not what I >> expected. I am still not sure what the criteria is to actually be welcome here. > > Hi Ian, > > I'm sorry you feel like this. It's definitely not the intention of this list > to exclude people -- just look at the wild ideas being discussed all the time... > > The Python 2/3 topic is simply a "sore point" for many of us, having discussed > it time and again, often times with quite aggressive people (with the "why have > you ruined my Python" mindset). Therefore we tend to become a little > categorical faster than with other topics, even to people like you who aren't > being aggressive. I hope you can forgive Nick, who has put tons of work into > the excellent 2 vs 3 FAQ, being a bit thin-skinned :) Aye, I hit the "This is holding back Python!" in Ian's original post, and subsequently interpreted the rest of the post (and thread) in an extremely negative light. We get a *lot* of people making such accusations (understandably so, even though those accusations aren't adequately backed up by the available data), so I have a very low tolerance at the moment for anything that even hints at commercial operations complaining about the quality of open source work they aren't funding (or otherwise contributing to), but are relying on anyway. However, Ian, It sounds like that wasn't your intent, so my apologies (once again) for taking it that way. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From stefan_ml at behnel.de Sat Mar 1 11:51:23 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 01 Mar 2014 11:51:23 +0100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> Message-ID: <lese2v$s63$1@ger.gmane.org> ian o, 01.03.2014 04:09: > We have 3 teams working server systems currently with python as the > implementation model. All work is currently in python 2.x and with a > solution to this the work could immediately move to 3.x. You are mixing things up here. If you are developing application code, you don't really have a problem. Either you stay with 2.x or you switch to 3.x. You may even decide to stay in Py2 with some of your applications and migrate the others. Or migrate one after the other, over months or years. You really don't have to break your running systems, and definitely not all of them at once. It's entirely your choice, and doing the switch (if you want to) really isn't all that hard. Nick pointed you to a couple of resources that can help you in the progress. Many, many others have done it before, and have found that breaking all bridges after you have passed over them is a perfectly reasonable and efficient way to migrate, *specifically* for application code. The real problem, and that's what Nick was referring to, is writing and maintaining libraries that others depend on, because these others may use Python 2.x (and may not even want to switch to Py3), or they may use Python 3.x (and do not care about Python 2.x). But the library authors do not control the application code. Therefore, to cater for both user groups, library maintainers currently have to write code that works on both platforms, which is annoying and time consuming. Since you seem to be interested in migrating your application code in order to take advantage of Python 3, my advice is to (ta-da!) start migrating your code. And when that's done, stop looking back. Any "solution" that involves running a mix of Py2 and Py3 inside of one application is just screaming for endless trouble. Don't forget that "temporary solutions" tend to be the most durable in practice. Stefan From g.rodola at gmail.com Sat Mar 1 12:33:02 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Sat, 1 Mar 2014 12:33:02 +0100 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> <A58A205B-A3EA-4A36-8A5F-E7F402D2C689@yahoo.com> Message-ID: <CAFYqXL-FyZ=SYk=oUYJ7_Fj32nF6qD-07Gn46ncuVoWLUAKsQg@mail.gmail.com> On Sat, Mar 1, 2014 at 6:09 AM, Andrew Barnert <abarnert at yahoo.com> wrote: > For that matter, have you actually checked whether those libraries really > are blocking you? I've lost count of the number of people who've claimed > they can't switch to Python 3 because of some library that's been ported > since 2010 or that has a 3.x-compatible drop-in replacement. > The problem you mention here is absolutely *real*. It's not only a matter of whether all your deps have currently been ported already. Software evolves over time and you can also remain stuck later, when you suddenly decide you need lib X and lib X hasn't been ported yet (uh-oh! what do I do now?). And please, note that *as of now* lib X includes names such as Twisted and Paramiko, which means hundreds of thousands of users which are stuck as we speak. How can a company decide to make the switch starting from such an assumption? The assumption that lib X or Y might not be ported soon, or even *ever*? IMHO as long as we won't get close to a 100% of libs being ported the current situation is not likely to change. FWIW here's some stats (inspired by recent Brett Cannon's post about openstack): http://stackoverflow.com/a/22113627/376587 -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/44d44660/attachment.html> From breamoreboy at yahoo.co.uk Sat Mar 1 16:48:48 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 01 Mar 2014 15:48:48 +0000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> Message-ID: <lesvfq$557$1@ger.gmane.org> On 27/02/2014 23:26, ian o wrote: I have read every entry so far on this thread and have been forced to the conclusion that this is A Bridge Too Far. Sorry :( -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From Steve.Dower at microsoft.com Sat Mar 1 16:54:26 2014 From: Steve.Dower at microsoft.com (Steve Dower) Date: Sat, 1 Mar 2014 15:54:26 +0000 Subject: [Python-ideas] Update the required C compiler for Windows to a supported version. In-Reply-To: <CAH-ZgAeEB4+fEwbxHno-cbJeQypMRQX_b9pJMQvNyzYDsoqt2Q@mail.gmail.com> References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>, <CAH-ZgAeEB4+fEwbxHno-cbJeQypMRQX_b9pJMQvNyzYDsoqt2Q@mail.gmail.com> Message-ID: <682219fab07d40a5acee0eaf50b7992c@BLUPR03MB389.namprd03.prod.outlook.com> Not at my desk, so I can only get you started (didn't want to keep you waiting until Monday). It'll look something like this: IF %1 eq x86 CALL bin\vcvars32.bat %* IF %1 eq amd64 CALL bin\vcvars64.bat %* (similar for x86_amd64) The file should go at C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat If you want to redistribute something that will work for others, it's possible to monkey patch distutils from your setup.py. I can't remember the specifics right now, so I can't help with that until I'm back at work. I think that's the best long term solution for 2.7 packages that want to build on Windows. Cheers, Steve Top-posted from my Windows Phone ________________________________ From: Vernon D. Cole<mailto:vernondcole at gmail.com> Sent: ?3/?1/?2014 2:24 To: Steve Dower<mailto:Steve.Dower at microsoft.com> Cc: Stephen J. Turnbull<mailto:stephen at xemacs.org>; python-ideas<mailto:python-ideas at python.org> Subject: Re: [Python-ideas] Update the required C compiler for Windows to a supported version. Sorry to resurrect this thread after so long a time, but I am starting to work on a project where I need this again... Any chance that I can get a copy (or a link to) " the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one" ? It's only simple to add one if you know what changes to make. -- VC On Wed, Nov 27, 2013 at 7:24 PM, Steve Dower <Steve.Dower at microsoft.com<mailto:Steve.Dower at microsoft.com>> wrote: Stephen J. Turnbull wrote: > Vernon D. Cole writes: > >> I cannot compile a Python extension module with any Microsoft compiler >> I can obtain. > > Your pain is understood, but it's not simple to address it. FWIW, I'm working on making the compiler easily obtainable. The VS 2008 link that was posted is unofficial, and could theoretically disappear at any time (I'm not in control of that), but the Windows SDK for Windows 7 and .NET 3.5 SP1 (http://www.microsoft.com/en-us/download/details.aspx?id=3138) should be around for as long as Windows 7 is supported. The correct compiler (VC9) is included in this SDK, but unfortunately does not install the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one that will switch on %1 and call the correct vcvars(86|64|...).bat.) The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made. > The basic problem is that the ABI changes. Therefore it's going to require > a complete new set of *all* C extensions for Windows, and the duplication > of download links for all those extensions from quite a few different vendors > is likely to confuse a lot of users. Specifically, the CRT changes. The CRT is an interesting mess of data structures that are exposed in header files, which means while you can have multiple CRTs loaded, they cannot touch each other's data structures at all or things will go bad/crash, and there's no nice way to set it up to avoid this (my colleague who currently owns MSVCRT suggested a not-very-nice way to do it, but I don't think it's going to be reliable enough). Python's stable ABI helps, but does not solve this problem. The file APIs are the worst culprits. The layout of FILE* objects can and do change between CRT versions, and file descriptors are simply indices into an array of these objects that is exposed through macros rather than function calls. As a result, you cannot mix either FILE pointers or file descriptors between CRTs. The only safe option is to build with the matching CRT, and for MSVCRT, this means with the matching compiler. It's unfortunate, and the responsible teams are well aware of the limitation, but it's history at this point, so we have no choice but to work with it. Cheers, Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140301/a8bc2053/attachment.html> From oscar.j.benjamin at gmail.com Sat Mar 1 18:34:34 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 1 Mar 2014 17:34:34 +0000 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> Message-ID: <CAHVvXxQE_N4Nj9Qz=ezhM7Cpws7v+OWJe4uFh=46JUwM1QLF=A@mail.gmail.com> On 1 March 2014 03:09, ian o <ian.team.python at gmail.com> wrote: > >> >> And we don't actually mind all that much if applications don't migrate >> in the near term - Python 2 will have commercial support available >> well past 2020. (The developers of *those applications* might mind, >> though, just as anyone maintaining Python 2.4 compatibility for the >> benefit of RHEL/CentOS 5 typically isn't happy about it) > > Your response is most dissapointing, and I would hope does not represent the > community overall. > > We have 3 teams working server systems currently with python as the > implementation model. All work is currently in python 2.x and with a > solution to this the work could immediately move to 3.x. > > However, you state that our situation, and that of the other 60% of python > programmers as at end January 2014 who are constrained from using python 3 > by python 2 dependencies: "we don't mind about those developers" Hi Ian, In the past I've sometimes seen Nick being slightly careless with his words with the result that someone misunderstands him and gets upset. That's not what's happening here though. I'm not sure if you're wilfully misunderstanding him or just unable to see this from his perspective. Nick clearly did not say that he (or anyone else) doesn't care about any particular group of *developers*. What he meant is that if you as an application author have decided that sticking with Python 2 is the right thing to do right now then he won't try to argue with you. If your application is currently working and has a dependency on web2py and it's not worth your time/money to port web2py yourselves then sticking with Python 2 may be the right decision for you. OTOH if you were the author of web2py and said "I don't want to port web2py to Python 3" Nick would mind. He would want to know why you didn't want to port to Python 3 and whether or not there was something (reasonable) that the core Python devs could do to help. This is because if a library is not ported then that has a knock-on effect on application authors (such as you) who would like to port an existing application or write a new application for Python 3. Oscar From ncoghlan at gmail.com Sun Mar 2 01:39:17 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 2 Mar 2014 10:39:17 +1000 Subject: [Python-ideas] Update the required C compiler for Windows to a supported version. In-Reply-To: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com> References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com> Message-ID: <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com> On 28 November 2013 04:24, Steve Dower <Steve.Dower at microsoft.com> wrote: > The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made. The main challenge we face on that front is the fact that Visual Studio 2013 is Windows 7+ only, so we'd potentially be causing problems for people still developing on XP or Vista if we update. Windows XP goes EOL next month, so that will be dropped for 3.5 next year in accordance with PEP 11. However, Vista is still in extended support until 2017 - under the current PEP 11 guidelines, that means it will be supported for at least 3.5, and likely 3.6 as well. If we update the required compiler we'd end up in a situation where CPython 3.5 nominally supports Vista, but Vista users would still need at least one Windows 7 machine to install the toolchain to create compatible C extensions. That may be OK given the number of people that jumped straight from XP to Windows 7, but it's still a consideration that would need to be taken into account. Longer term, we'd love to add a build farm to PyPI to take the drudgery out of creating pre-compiled wheel files (and MSI installers for Windows), and that would potentially address this problem as well (since the build farm would include the requisite Windows 7 images). However, I currently think it would be rather optimistic to assume that we'll have that in place by the time 3.5 rolls around (there's a lot of other things that need to be addressed before a build farm will make it to the top of the distutils-sig todo list). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From shibturn at gmail.com Sun Mar 2 12:42:52 2014 From: shibturn at gmail.com (Richard Oudkerk) Date: Sun, 02 Mar 2014 11:42:52 +0000 Subject: [Python-ideas] Update the required C compiler for Windows to a supported version. In-Reply-To: <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com> References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com> <CADiSq7e1VsxKzXefGFtRSnf71T-ZLM6vrJVuAcWBcRQqmsL2Xw@mail.gmail.com> Message-ID: <5313193C.309@gmail.com> On 02/03/2014 12:39 am, Nick Coghlan wrote: > Windows XP goes EOL next month, so that will be dropped for 3.5 next > year in accordance with PEP 11. > > However, Vista is still in extended support until 2017 - under the > current PEP 11 guidelines, that means it will be supported for at > least 3.5, and likely 3.6 as well. BTW, Windows Server 2003 has extended support until 14 July 2015. Will it be supported in Python 3.5? -- Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140302/4a6dae32/attachment.html> From stephen at xemacs.org Sun Mar 2 15:55:43 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 02 Mar 2014 23:55:43 +0900 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <lese2v$s63$1@ger.gmane.org> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <CAPTjJmpx+tkOPm6Thsf4SamEV6wAqk970OY4iEurA6bJnbhUXQ@mail.gmail.com> <CADiSq7e2sU4xXv9tTcE4BhswwP3w_bBvxdjpSNYtURSxXr7M0A@mail.gmail.com> <fa105374-a4b6-46cb-865f-ec5a1d1d4272@googlegroups.com> <lese2v$s63$1@ger.gmane.org> Message-ID: <87iorwsl8w.fsf@uwakimon.sk.tsukuba.ac.jp> Stefan Behnel writes: > Since you seem to be interested in migrating your application code > in order to take advantage of Python 3, my advice is to (ta-da!) > start migrating your code. And when that's done, stop looking back. I'm sure he's thinking that way for himself. To be fair, though, as he describes it, Ian is evidently not the boss in the commercial projects he's talking about projects. He needs to convince others that Python 3 is a practical solution to their problems. This makes the idea of lobbying for Python a real enthusiasm drainer. I'm still all for it, but we should recognize the social aspects as well as the technical issues. Unfortunately, I'm not sure what, if anything, we can do to help support developers like Ian who can't impose Python 3 -- but rather has to expend personal energy and reputation in lobbying. From stephen at xemacs.org Sun Mar 2 15:37:48 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 02 Mar 2014 23:37:48 +0900 Subject: [Python-ideas] A python bridge between versions In-Reply-To: <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> References: <2d03c3e3-d036-4d68-b028-0848df54786a@googlegroups.com> <5b0b1caa-d1e1-4064-99ac-185f4dfae521@googlegroups.com> <20140301040740.GF28804@ando> <bb33a4a6-16b8-49fb-a0de-4bfac0bb85fb@googlegroups.com> <CADiSq7eWAXtmotQE+qQrCtCJwZEP6QaHq3PeRr1fcdaTz_5MpA@mail.gmail.com> <3b02e645-b04f-4998-a3ec-ef1f21f3f4a4@googlegroups.com> Message-ID: <87k3ccsm2r.fsf@uwakimon.sk.tsukuba.ac.jp> >>>>> Ian O. writes: > Nick Coghlan writes: >> Ian, please keep in mind that you are asking people to do extra work >> for you *for free*, after you have already been benefiting for years > Why are you so aggressive? > No one is trying to 'tell' people to do something. In Nick's defense, (1) he spends a *lot* of his time explaining why Python development proceeds as it does, (2) "just proposing ideas" sometimes is indeed a passive-aggressive ploy, (3) your timing is bad due to the appearance of an occasional poster who does do exactly that, and (4) Nick's probably harried with a major release in process and PyCon coming up. > For commercial purposes you have managed to convince me I should > cease to lobby for python to be used in the projects where I > assist. I think that's *exactly* the wrong lesson to learn, unless your management doesn't understand that free software is an investment and requires its users to spend resources to take advantage of it. It's just that the form of the resources required is often quite different from those required to use proprietary software. And of course it differs from product to product even within open source. So all you've learned from Nick is that Python is unlikely to dramatically reduce the resources required to migrate from Python 2 to Python 3 using technology like a "bridge". "Everyone would like a pony, but we don't have everyone's favorite pony." OTOH, it remains the case that most new applications can be written in Python 3 because there are good libraries to support them, although you may have to give up on some frameworks that depend on libraries that haven't been ported. That may be, but is not necessarily, a show-stopper. Now you say "lobbying". "Lobbying" implies effort and marshalling resources. Even if you strongly favor a particular framework that is *apparently* blocked by an unported library, it's surely theoretically possible that the shortest path to serving your company's software needs is spend company resources to port that library[1] -- as well as the personal resources to get the necessary support and resources from management! In practice, while I can't offhand give a case where a commercial company has done that port internally, I know at least one noncommercial project that is considering paying for a port of one of its Python3 blockers. I'm willing to bet you can find companies that have done 2to3 ports of libraries considering that to be the "shortest path to a product release", if you look around a bit. > Obviously commercial use is clearly not desired by the community, > that is the first lesson i have learned. As I am not paid to > champion python in the work environment, having been enlightened > that python community does not desire commercial usage, I will > cease. Are you aware how passive-aggressive your phrasing sounds? I'm sure you're well aware that not only are many Python developers involved in commercial development, but all of Python from the license on up is oriented toward encouragement of commercial use of Python. And it's been quite successful in commercial use. It would be nice if the language improvements in Python 3 came at zero cost. They don't, and we all feel that pinch at one time or another. But remember, innovations succeed not only by inventing completely new ideas that nobody ever thought of before, but also by hurdling cost barriers that others considered so high as to make known "interesting in theory" ideas "commercially impractical". I am not going to tell you that your cost barrier is lower than you think -- you know your situation, I don't. But I assure you that others, with high but not insuperable cost barriers, have indeed made that hurdle in porting libraries to Python 3, and I'm sure some are taking on new challenges today. Footnotes: [1] Just as if it were a completely different language ("that Perl module would be perfect if only it were in Python..." :-). From robert.kern at gmail.com Sun Mar 2 16:05:59 2014 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 02 Mar 2014 15:05:59 +0000 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com> Message-ID: <levhc8$h3f$1@ger.gmane.org> On 2014-02-27 22:41, Guido van Rossum wrote: > I haven't followed this recommendation for years. :-) Good riddance! FWIW, PEP 8 still suggests the old form, though the example was fixed to conform to the new suggestion: http://hg.python.org/peps/file/272debef6b77/pep-0008.txt#l548 http://hg.python.org/peps/rev/5efe00002b3e > (However, the one about two spaces after a period stands. :-) Alas! -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From guido at python.org Sun Mar 2 18:29:45 2014 From: guido at python.org (Guido van Rossum) Date: Sun, 2 Mar 2014 09:29:45 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <levhc8$h3f$1@ger.gmane.org> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAP7+vJL8NWmUKkYEJOVcu26_-Xn4QPfH7EMeC8w8ZRoXW-FtLQ@mail.gmail.com> <levhc8$h3f$1@ger.gmane.org> Message-ID: <CAP7+vJJ52894BAQ02DgkymOwOOL8uHfb==szar_pZqkQMWEdRw@mail.gmail.com> Fixed. Thanks for pointing this out! On Sun, Mar 2, 2014 at 7:05 AM, Robert Kern <robert.kern at gmail.com> wrote: > On 2014-02-27 22:41, Guido van Rossum wrote: > >> I haven't followed this recommendation for years. :-) Good riddance! >> > > FWIW, PEP 8 still suggests the old form, though the example was fixed to > conform to the new suggestion: > > http://hg.python.org/peps/file/272debef6b77/pep-0008.txt#l548 > http://hg.python.org/peps/rev/5efe00002b3e > > > (However, the one about two spaces after a period stands. :-) >> > > Alas! > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140302/70356447/attachment-0001.html> From harrismh777 at gmail.com Mon Mar 3 16:41:40 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Mon, 3 Mar 2014 09:41:40 -0600 Subject: [Python-ideas] Python3.3 Decimal Library Released Message-ID: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> Greetings, Python3.3 Decimal Library v0.3 is Released here: https://code.google.com/p/pythondecimallibrary/ pdeclib.py is the decimal library, pilib.py is the PI library. pdeclib.py provides scientific and transcendental functions for the C Accelerated Decimal module written by Stefan Krah. The library is open source, GLPv3, comprised of two py files. My idea for python is to two things really, 1) make floating point decimal the default floating point type in python4.x, and 2) make these functions ( pdeclib.py ) or equiv available in python4.x by default. Thank you for your consideration. -- Kind regards, Mark H. Harris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/9ba9cb5a/attachment.html> From solipsis at pitrou.net Mon Mar 3 16:58:02 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 03 Mar 2014 16:58:02 +0100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> Message-ID: <lf28ps$d7n$1@ger.gmane.org> Hello Mark, Le 03/03/2014 16:41, Mark H. Harris a ?crit : > Greetings, > Python3.3 Decimal Library v0.3 is Released here: > > https://code.google.com/p/pythondecimallibrary/ > > pdeclib.py is the decimal library, pilib.py is the PI library. > > pdeclib.py provides scientific and transcendental functions > for the C Accelerated Decimal module written by Stefan Krah. The > library is open source, GLPv3, comprised of two py files. > > My idea for python is to two things really, 1) make floating point > decimal the default floating point type in python4.x, and 2) make > these functions ( pdeclib.py ) or equiv available in python4.x by > default. If you want to contribute those functions to Python, the first required step would be to license them under terms compatible with the contributor agreement: http://www.python.org/psf/contrib/contrib-form/ (i.e. under the Academic Free License v. 2.1 or the Apache License, Version 2.0) Regards Antoine. From harrismh777 at gmail.com Mon Mar 3 17:23:49 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Mon, 3 Mar 2014 08:23:49 -0800 (PST) Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <lf28ps$d7n$1@ger.gmane.org> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <lf28ps$d7n$1@ger.gmane.org> Message-ID: <aed07157-482b-4fa8-b5be-effc14357ced@googlegroups.com> On Monday, March 3, 2014 9:58:02 AM UTC-6, Antoine Pitrou wrote: > > > Hello Mark, > > Le 03/03/2014 16:41, Mark H. Harris a ?crit : > > Greetings, > > Python3.3 Decimal Library v0.3 is Released here: > > > > https://code.google.com/p/pythondecimallibrary/ > > > > pdeclib.py is the decimal library, pilib.py is the PI library. > > > > pdeclib.py provides scientific and transcendental functions > > for the C Accelerated Decimal module written by Stefan Krah. The > > library is open source, GLPv3, comprised of two py files. > > > > My idea for python is to two things really, 1) make floating point > > decimal the default floating point type in python4.x, and 2) make > > these functions ( pdeclib.py ) or equiv available in python4.x by > > default. > > If you want to contribute those functions to Python, the first required > step would be to license them under terms compatible with the > contributor agreement: > http://www.python.org/psf/contrib/contrib-form/ > > (i.e. under the Academic Free License v. 2.1 or the Apache License, > Version 2.0) > Antoine, thank you for the response. I have e-signed the Academic Free License, and it is filed. Mark H Harris marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/f79c57ec/attachment.html> From oscar.j.benjamin at gmail.com Mon Mar 3 17:52:01 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 3 Mar 2014 16:52:01 +0000 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> Message-ID: <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com> On 3 March 2014 15:41, Mark H. Harris <harrismh777 at gmail.com> wrote: > Greetings, > Python3.3 Decimal Library v0.3 is Released here: > > https://code.google.com/p/pythondecimallibrary/ > > pdeclib.py is the decimal library, pilib.py is the PI library. > > pdeclib.py provides scientific and transcendental functions > for the C Accelerated Decimal module written by Stefan Krah. The > library is open source, GLPv3, comprised of two py files. > > My idea for python is to two things really, 1) make floating point decimal > the default floating point type in python4.x, This is an interesting suggestion. It's hard to judge how much code would be broken by such an explicit change. A preliminary less controversial step would just be to introduce decimal literals e.g 1.23d. > and 2) make > these functions ( pdeclib.py ) or equiv available in python4.x by > default. If there was a desire to add these then there's no reason why they would have to wait until the (hypothetical) release of Python 4.x. However I doubt that they would be accepted without having first spent some time maturing on PyPI and certainly not without unit tests. Oscar From harrismh777 at gmail.com Mon Mar 3 18:10:07 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Mon, 3 Mar 2014 09:10:07 -0800 (PST) Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com> Message-ID: <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com> On Monday, March 3, 2014 10:52:01 AM UTC-6, Oscar Benjamin wrote: > > > This is an interesting suggestion. It's hard to judge how much code > would be broken by such an explicit change. A preliminary less > controversial step would just be to introduce decimal literals e.g > 1.23d. > > If there was a desire to add these then there's no reason why they > would have to wait until the (hypothetical) release of Python 4.x. > However I doubt that they would be accepted without having first spent > some time maturing on PyPI and certainly not without unit tests. > Oscar, sounds right. I would like to see it just the other way round. We have a very speedy decimal module, its 2014, and we just don't need any more of this: >>> from decimal import * >>> Decimal(.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') >>> I'm from back in that day. You know, about 1982. We've embedded floats and doubles in everything from the processor to python and beyond. Memory is cheap, technology has advanced (heck, python is one of the coolest most sophisticated languages on the planet), its time to use decimal floating point. I'm thinking we ought to have decimal by default, and the binary literal 1.23b as the option. As far as how much code breaks; who knows, but it can't be any worse than moving from 2.7.x to 3.3.4. I mean, everyone thought that was going to break the world, and people have adapted just fine. Some folks are staying put on 2.7.6 (and that's fine) and some folks like me are out there on the bloody edge downloading 3.3.5 ! Thanks for the discussion Oscar. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/cdc8356b/attachment-0001.html> From stefan_ml at behnel.de Mon Mar 3 18:24:48 2014 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 03 Mar 2014 18:24:48 +0100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <CAHVvXxQEWcQuDsR+PgBhw96qse1=UU3GQosRv0dqNOzQEMTb3w@mail.gmail.com> <e65bbdf3-ee1a-4bc8-a3c6-3cff00aed4b7@googlegroups.com> Message-ID: <lf2dsi$i60$1@ger.gmane.org> Mark H. Harris, 03.03.2014 18:10: > We > have a very speedy decimal module, its 2014, and we just don't need any > more of this: > > >>> from decimal import * > >>> Decimal(.1) > Decimal('0.1000000000000000055511151231257827021181583404541015625') I assume you are aware that the correct way to do this is >>> Decimal('0.1') Decimal('0.1') i.e., if you want exact precision, use exact literals. That being said, did you read through the previous discussions where people suggested to add decimal literals to Python? Stefan From steve at pearwood.info Mon Mar 3 18:54:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Mar 2014 04:54:29 +1100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> Message-ID: <20140303175429.GL28804@ando> On Mon, Mar 03, 2014 at 09:41:40AM -0600, Mark H. Harris wrote: > My idea for python is to two things really, 1) make floating point decimal > the default floating point type in python4.x, and I think it is premature to be talking about what goes into Python 4.x, which is why I refer to it as "Python 4000". There's no concrete plans for a Python 4 yet, or even whether there will be a Python 4, what the last Python 3.x version will be, or what sort of changes will be considered. But I would expect that any such Python 4 will probably be at least four years away, although given the extended life of 2.7 possibly more like eight. (Given the stress of the 2->3 migration, I think *nobody* is exactly in a hurry for yet another backwards incompatible version. Perhaps we should be delaying such things until Python 5000.) > 2) make > these functions ( pdeclib.py ) or equiv available in python4.x by > default. If it's worth having a decimal maths library, its probably worth having it in 3.5 or 3.6. -- Steven From zachary.ware+pyideas at gmail.com Mon Mar 3 19:23:58 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Mon, 3 Mar 2014 12:23:58 -0600 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140303175429.GL28804@ando> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> Message-ID: <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> On Mon, Mar 3, 2014 at 11:54 AM, Steven D'Aprano <steve at pearwood.info> wrote: > If it's worth having a decimal maths library, its probably worth having > it in 3.5 or 3.6. Agreed. I'm -1 on a decimal-specific math library, though. What I would rather see is a type-agnostic math library, that does what (IIRC) the new statistics module does: take in numbers of any type, coerce only as is strictly necessary, calculate, and return an answer of the same type as the input with the goal of correctness over performance. If all input is float (or complex), calculation could be delegated to the existing math (or cmath) libraries, and performance wouldn't be too much worse than using those libraries directly. -- Zach From ziad.sawalha at rackspace.com Mon Mar 3 19:55:17 2014 From: ziad.sawalha at rackspace.com (Ziad Sawalha) Date: Mon, 3 Mar 2014 18:55:17 +0000 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com> <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com> <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com> Message-ID: <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com> Thanks, Guido. I?ll follow up with updates to common tools as I come across them (ex. pep257: https://github.com/GreenSteam/pep257/pull/64). Ziad On Feb 28, 2014, at 11:53 AM, Guido van Rossum <guido at python.org<mailto:guido at python.org>> wrote: Updated! On Thu, Feb 27, 2014 at 8:58 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote: Approved :-) On Feb 27, 2014, at 9:33 PM, "Guido van Rossum" <guido at python.org<mailto:guido at python.org>> wrote: How about this patch? http://codereview.appspot.com/69870043 On Thu, Feb 27, 2014 at 5:48 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote: On Feb 27, 2014, at 4:42 PM, "Bruce Leban" <bruce at leapyear.org<mailto:bruce at leapyear.org>> wrote: To clarify are you asking to delete the recommendation of a blank line or the entire recommendation? That is are you suggesting the recommendation change to ?It is recommended to place the closing quotes on a line by themselves." That's right. I just want to get rid of the recommendation for the blank line. I think deleting it completely is a bad idea as otherwise it's hard to see where the docstring ends and the code begins, especially when using doctest. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban On Thu, Feb 27, 2014 at 2:06 PM, Ziad Sawalha <ziad.sawalha at rackspace.com<mailto:ziad.sawalha at rackspace.com>> wrote: PEP-257 includes this recommendation: ?The BDFL [3] recommends inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes, placing the closing quotes on a line by themselves. This way, Emacs' fill-paragraph command can be used on it.? I believe emacs no longer has this limitation. "If you do fill-paragraph in emacs in Python mode within a docstring, emacs already ignores the closing triple-quote. In fact, the most recent version of emacs supports several different docstring formatting styles and gives you the ability to switch between them.? - quoting Kevin L. Mitchell who is more familiar with emacs than I am. I?m considering removing that recommendation and updating some of the examples in PEP-257, but I?d like some thoughts from this group before I submit the patch. Any thoughts or references to conversations that may have already been had on this topic? Regards, Ziad _______________________________________________ Python-ideas mailing list Python-ideas at python.org<mailto:Python-ideas at python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ _______________________________________________ Python-ideas mailing list Python-ideas at python.org<mailto:Python-ideas at python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- --Guido van Rossum (python.org/~guido<http://python.org/~guido>) -- --Guido van Rossum (python.org/~guido<http://python.org/~guido>) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/377693d1/attachment.html> From tjreedy at udel.edu Mon Mar 3 20:25:50 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 03 Mar 2014 14:25:50 -0500 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140303175429.GL28804@ando> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> Message-ID: <lf2kvr$aje$1@ger.gmane.org> On 3/3/2014 12:54 PM, Steven D'Aprano wrote: > I think it is premature to be talking about what goes into Python 4.x, > which is why I refer to it as "Python 4000". There's no concrete plans > for a Python 4 yet, or even whether there will be a Python 4, what the > last Python 3.x version will be, Given that Guido does not want double-digit minor version numbers, 3.9, at the latest, will be followed by 4.0. > or what sort of changes will be considered. There will be several deprecated items removed, such as http://docs.python.org/2/library/unittest.html#deprecated-aliases I think it would be worth making a consolidated list. I think a few modules that have replacements will be considered for removal. Optparse (if argparse is an adequate replacement)? Asyncore (if the new async catches on)? Changing the meaning of a core feature like float literals is a different matter. > But I would expect that any such Python 4 will probably be > at least four years away, although given the extended life of 2.7 > possibly more like eight. 3.5, ..., 3.9, 4.0 is 6 releases, which should be about 10 years. > (Given the stress of the 2->3 migration, I think *nobody* is exactly in > a hurry for yet another backwards incompatible version. Perhaps we > should be delaying such things until Python 5000.) -- Terry Jan Reedy From tjreedy at udel.edu Mon Mar 3 20:27:26 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 03 Mar 2014 14:27:26 -0500 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com> <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com> <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com> <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com> Message-ID: <lf2l2q$aje$2@ger.gmane.org> On 3/3/2014 1:55 PM, Ziad Sawalha wrote: > Thanks, Guido. > > I?ll follow up with updates to common tools as I come across them (ex. > pep257: https://github.com/GreenSteam/pep257/pull/64). Does pep8.py try to enforce the extra blank? -- Terry Jan Reedy From p.f.moore at gmail.com Mon Mar 3 21:12:53 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 3 Mar 2014 20:12:53 +0000 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> Message-ID: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> On 3 March 2014 18:23, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote: > I'm -1 on a decimal-specific math library, though. What I would > rather see is a type-agnostic math library, that does what (IIRC) the > new statistics module does: take in numbers of any type, coerce only > as is strictly necessary, calculate, and return an answer of the same > type as the input with the goal of correctness over performance. Given that incorrect results aren't acceptable in any situation, fair enough, but what I'd like to see would be library functions that provided good performance for decimal first. If they can do so while remaining type-agnostic, then fine, but I'd favour decimal performance over the ability to calculate the sine of a rational... Ultimately, I'd like to feel that I pay a one-off cost for working with decimals (the cost of a software implementation rather than a hardware one) but I *don't* have to pay further because the algorithms used for decimals are suboptimal compared to the floating point ones. I'd say the same about rationals, but the set of functions involved is different (and basically wouldn't involve most of what's in the normal math library) Having said this, I can't actually think of a real-life domain where math-library functions (by which I assume we basically mean trancendental functions?) would be used and yet there would be a genuine need for decimal arithmetic rather than floating point. So although I'm +1 for a decimal math library in theory, I'm unsure of its practical value. Paul From mertz at gnosis.cx Mon Mar 3 21:44:12 2014 From: mertz at gnosis.cx (David Mertz) Date: Mon, 3 Mar 2014 12:44:12 -0800 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> Message-ID: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> Taking a look at the documentation for cdecimal itself (now part of Python 3.3) at http://www.bytereef.org/mpdecimal/benchmarks.html, it looks like for basic add/mul operations that don't exceed the precision of floating point, FP is more than twice as fast as optimized decimals. Of course, where the precision needed is more than FP handles at all, it's simply a choice between calculating a quantity and not doing so. This suggests to me a rather strong argument against making decimals the *default* numeric datatype. However, it *does* still remain overly cumbersome to create decimals, and a literal notation for them would be very welcomed. It is far from clear to me that decimals are generically the *right* answer to rounding issues (i.e. even if we don't care about the benchmark question). Yes, it's easier for humans who use base 10 to understand numeric results when using them. But the hardware on CPUs is oriented around binary floating point representations, and pretty much every other programming language uses BFP. It's a nice property to have 'a+b' in Python produce the same answer as 'a+b' in C, Ruby, Perl, Mathematica, Haskell, Java, etc.--I'm not saying that's the only nice property one might want, but it is one of them since data is exchanged between tools and libraries written in different languages (or even, e.g. Python often uses libraries written in C). I do recognize that in Python 4000, we *could* have default literals be decimal, and require some other literal notation for binary floating point, and that *would* be available for calculations meant to be compatible with outside tools. However, I haven't seen a case presented for why decimals are generically better as a default. Of possible note, I happen to work in a research lab where we worry a lot about bitwise identity of calculations, not merely about numeric stability. I know that is a somewhat unusual requirement, even within scientific computing. But it is one that exists, and it makes me think of (bitwise) compatibility with calculations in other programming languages. On Mon, Mar 3, 2014 at 12:12 PM, Paul Moore <p.f.moore at gmail.com> wrote: > On 3 March 2014 18:23, Zachary Ware <zachary.ware+pyideas at gmail.com> > wrote: > > I'm -1 on a decimal-specific math library, though. What I would > > rather see is a type-agnostic math library, that does what (IIRC) the > > new statistics module does: take in numbers of any type, coerce only > > as is strictly necessary, calculate, and return an answer of the same > > type as the input with the goal of correctness over performance. > > Given that incorrect results aren't acceptable in any situation, fair > enough, but what I'd like to see would be library functions that > provided good performance for decimal first. If they can do so while > remaining type-agnostic, then fine, but I'd favour decimal performance > over the ability to calculate the sine of a rational... Ultimately, > I'd like to feel that I pay a one-off cost for working with decimals > (the cost of a software implementation rather than a hardware one) but > I *don't* have to pay further because the algorithms used for decimals > are suboptimal compared to the floating point ones. I'd say the same > about rationals, but the set of functions involved is different (and > basically wouldn't involve most of what's in the normal math library) > > Having said this, I can't actually think of a real-life domain where > math-library functions (by which I assume we basically mean > trancendental functions?) would be used and yet there would be a > genuine need for decimal arithmetic rather than floating point. So > although I'm +1 for a decimal math library in theory, I'm unsure of > its practical value. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/8722d467/attachment.html> From guido at python.org Mon Mar 3 21:52:10 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Mar 2014 12:52:10 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <lf2l2q$aje$2@ger.gmane.org> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com> <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com> <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com> <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com> <lf2l2q$aje$2@ger.gmane.org> Message-ID: <CAP7+vJJu=-AdZY49caDgB7Ot5m0VMNjLz-uC=f4ZviyMU1FdFQ@mail.gmail.com> On Mon, Mar 3, 2014 at 11:27 AM, Terry Reedy <tjreedy at udel.edu> wrote: > On 3/3/2014 1:55 PM, Ziad Sawalha wrote: > >> Thanks, Guido. >> >> I'll follow up with updates to common tools as I come across them (ex. >> pep257: https://github.com/GreenSteam/pep257/pull/64). >> > I already did that. > > Does pep8.py try to enforce the extra blank? > I dunno; it shouldn't. (I use it regularly and it doesn't complain to me.) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/a92e609f/attachment-0001.html> From harrismh777 at gmail.com Mon Mar 3 23:32:37 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Mon, 3 Mar 2014 14:32:37 -0800 (PST) Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> Message-ID: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> On Monday, March 3, 2014 2:44:12 PM UTC-6, David Mertz wrote: However, I haven't seen a case presented for why decimals are generically > better as a default. > > hi David, here is one, right out of the python3.3 docs, "Decimal ?is based on a floating-point model which was designed with people in mind, and necessarily has a paramount guiding principle ? computers must provide an arithmetic that works in the same way as the arithmetic that people learn at school.? ? excerpt from the decimal arithmetic specification. Business apps require precision (banking, sales, marketing, finance, & on and on). One big issue that is going to confront everyone sooner than later is cryptography. Fast bignum support, fast factoring, and fast transcendentals are going to become more important as firms and individuals move into their own on crypto; not too far fetched, really. We have got to come up with this on our own, because we can not trust others to get it right for us. NSA and GCHQ have made that clear. PGP was one thing, but we have got to invent something better. But, really the real reason is the first paragraph. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/a70f9d1f/attachment.html> From mertz at gnosis.cx Mon Mar 3 23:47:06 2014 From: mertz at gnosis.cx (David Mertz) Date: Mon, 3 Mar 2014 14:47:06 -0800 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> Message-ID: <CAEbHw4aVZXT9wDDum2_DtPv1Mxnb48fJPu+CDjQhAwCh9U2bEA@mail.gmail.com> On Mon, Mar 3, 2014 at 2:32 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > Business apps require precision (banking, sales, marketing, finance, > & on and on). > I definitely agree the decimal numbers are much better for financial data than binary floating point. That's certainly a very legitimate domain. On the other hand, there are a great many other domains that are very different from this--i.e. scientific areas. For this, decimal is no better, and in many cases worse. At the very least, throwing in a 2x+ slowdown of calculations is notably worse for scientific computing. In other words, I'm glad we have decimals in Python now. I'd be more glad if they were expressed more easily (e.g. as literals), but for more domains than not, for backward and cross-language compatibility, and for speed, binary floating point remains better. > One big issue that is going to confront everyone sooner than later is > cryptography. Fast bignum support, fast factoring, and fast > transcendentals are going to become more important as firms and > individuals move into their own on crypto; not too far fetched, really. > Sounds far fetched to me to imagine that amateur cryptography will ever be of value over well-audited, well studied, crypto-analyzed techniques. But bracketing that, exceedingly few cryptographic techniques are likely to use either binary or decimal floating point operations. This is strictly the domain of integer math, so mentioning it seems like random hand waving unrelated to the topic of default or convenient representation of fractional numeric data. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/ea51db41/attachment.html> From greg.ewing at canterbury.ac.nz Mon Mar 3 23:54:08 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 04 Mar 2014 11:54:08 +1300 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> Message-ID: <53150810.20507@canterbury.ac.nz> Mark H. Harris wrote: > One big issue that is going to confront everyone sooner than later is > cryptography. Fast bignum support, fast factoring, and fast transcendentals > are going to become more important I don't see any reason that these have to be done in decimal, though. All the bignum arithmetic used in cryptography is integer, which is exact in any base. Also, the user never sees the resulting numbers as numbers. So the primary requirement is speed, which argues for binary rather than decimal, at least on current hardware. -- Greg From solipsis at pitrou.net Tue Mar 4 00:01:22 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 4 Mar 2014 00:01:22 +0100 Subject: [Python-ideas] Python3.3 Decimal Library Released References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> Message-ID: <20140304000122.4f61e287@fsol> On Tue, 04 Mar 2014 11:54:08 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Mark H. Harris wrote: > > One big issue that is going to confront everyone sooner than later is > > cryptography. Fast bignum support, fast factoring, and fast transcendentals > > are going to become more important > > I don't see any reason that these have to be done in > decimal, though. All the bignum arithmetic used in > cryptography is integer, which is exact in any base. > Also, the user never sees the resulting numbers as > numbers. So the primary requirement is speed, which > argues for binary rather than decimal, at least on > current hardware. For the record, int doesn't have a sqrt() method while Decimal has, so if you wanna take the exact square root of a large integer, you'd better convert it to a Decimal. Regards Antoine. From mertz at gnosis.cx Tue Mar 4 00:07:57 2014 From: mertz at gnosis.cx (David Mertz) Date: Mon, 3 Mar 2014 15:07:57 -0800 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140304000122.4f61e287@fsol> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> Message-ID: <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > For the record, int doesn't have a sqrt() method while Decimal has, so > if you wanna take the exact square root of a large integer, you'd better > convert it to a Decimal. > Well, actually, if you want to take the square root of a large integer, most times you'll need an irrational number as a value. Unfortunately, neither floats, decimals, nor fractions are able to do that (nor any finite representation that is numeric; you can only use symbolic ones). On the other hand, now that you mention it, a floor_sqrt() function that operated quickly on ints would be nice to have. But that's a different thread. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/3851bee6/attachment-0001.html> From solipsis at pitrou.net Tue Mar 4 00:10:48 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 4 Mar 2014 00:10:48 +0100 Subject: [Python-ideas] Python3.3 Decimal Library Released References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> Message-ID: <20140304001048.22d6442c@fsol> On Mon, 3 Mar 2014 15:07:57 -0800 David Mertz <mertz at gnosis.cx> wrote: > On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > > For the record, int doesn't have a sqrt() method while Decimal has, so > > if you wanna take the exact square root of a large integer, you'd better > > convert it to a Decimal. > > > > Well, actually, if you want to take the square root of a large integer, > most times you'll need an irrational number as a value. Well, unless you know by construction that your integer is a perfect square. Regards Antoine. From rosuav at gmail.com Tue Mar 4 00:42:57 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 4 Mar 2014 10:42:57 +1100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> Message-ID: <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com> On Tue, Mar 4, 2014 at 7:44 AM, David Mertz <mertz at gnosis.cx> wrote: > Taking a look at the documentation for cdecimal itself (now part of Python > 3.3) at http://www.bytereef.org/mpdecimal/benchmarks.html, it looks like for > basic add/mul operations that don't exceed the precision of floating point, > FP is more than twice as fast as optimized decimals. Of course, where the > precision needed is more than FP handles at all, it's simply a choice > between calculating a quantity and not doing so. > > This suggests to me a rather strong argument against making decimals the > *default* numeric datatype. However, it *does* still remain overly > cumbersome to create decimals, and a literal notation for them would be very > welcomed. You could probably make the same performance argument against making Unicode the default string datatype. But a stronger argument is that the default string should be the one that does the right thing with text. As of Python 3, that's the case. And the default integer type handles arbitrary sized integers (although Py2 went most of the way there by having automatic promotion). It's reasonable to suggest that the default non-integer numeric type should also simply do the right thing. It's a trade-off, though, and for most people, float is sufficient. Is it worth the cost of going decimal everywhere? I want to first see a decimal literal notation (eg 1.0 == float("1.0") and 1.0d == Decimal("1.0"), as has been suggested a few times), and then have a taggable float marker (1.0f == float("1.0")); then there can be consideration of a __future__ directive to change the default, which would let people try it out and see how much performance suffers.Maybe it'd be worth it for the accuracy. Maybe we'd lose less time processing than we save answering the "why does this do weird things sometimes" questions. ChrisA From random832 at fastmail.us Tue Mar 4 02:19:06 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 03 Mar 2014 20:19:06 -0500 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> Message-ID: <1393895946.9103.90200705.2DF58D8B@webmail.messagingengine.com> On Mon, Mar 3, 2014, at 15:12, Paul Moore wrote: > Having said this, I can't actually think of a real-life domain where > math-library functions (by which I assume we basically mean > trancendental functions?) would be used and yet there would be a > genuine need for decimal arithmetic rather than floating point. So > although I'm +1 for a decimal math library in theory, I'm unsure of > its practical value. If we count non-integer powers (though, usually they're still rational), finance. I suspect though there's a better case for an arbitrary-precision library and no reason that library shouldn't be (also) able to work in decimal. From harrismh777 at gmail.com Tue Mar 4 02:37:26 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Mon, 3 Mar 2014 17:37:26 -0800 (PST) Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> Message-ID: <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com> On Monday, March 3, 2014 9:41:40 AM UTC-6, Mark H. Harris wrote: > > > Python3.3 Decimal Library v0.3 is Released here: > > https://code.google.com/p/pythondecimallibrary/ > > hi Oscar, I released my pdeclib module on PyPI this afternoon (it only took four hours) but I'll never forget how to do that again ! So if anyone is interested in the pdeclib module, it may be installed with pip install pdeclib The tarball will install both pdeclib.py (decimal library) & pilib.py (PI Library). The library is beta obviously, and is licensed for contribution on PyPI. Cheers, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/2a6d34c2/attachment.html> From steve at pearwood.info Tue Mar 4 02:55:16 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Mar 2014 12:55:16 +1100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com> Message-ID: <20140304015516.GM28804@ando> On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote: > You could probably make the same performance argument against making > Unicode the default string datatype. I don't think so -- for ASCII strings the performance cost of Unicode is significantly less than the performance hit for Decimal: [steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()" 100000 loops, best of 3: 8.76 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()" 100000 loops, best of 3: 7.05 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6" 1000000 loops, best of 3: 0.344 usec per loop [steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \ > -s "x = Decimal('123.4567')" "x**6" 1000000 loops, best of 3: 1.41 usec per loop That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal. I think that's *fast enough* for all but the most heavy numeric needs, but it's not something we can ignore. > But a stronger argument is that > the default string should be the one that does the right thing with > text. As of Python 3, that's the case. And the default integer type > handles arbitrary sized integers (although Py2 went most of the way > there by having automatic promotion). It's reasonable to suggest that > the default non-integer numeric type should also simply do the right > thing. Define "the right thing" for numbers. > It's a trade-off, though, and for most people, float is sufficient. That's a tricky one. For people doing quote-unquote "serious" numeric work, they'll mostly want to stick to binary floats, even if that means missing out on all the extra IEEE-754 goodies that the decimal module has but floats don't. The momentum of 40+ years of almost entirely binary floating point maths does not shift to decimal overnight. But for everyone else, binary floats are sufficient except when they aren't. Decimal, of course, won't solve all you floating point difficulties -- it's easy to demonstrate that nearly all the common pitfalls of FP maths also occurs with Decimal, with the exception of inexact conversion from decimal strings to numbers. But that one issue alone is a major cause of confusion. My personal feeling is that for Python 4000 I'd vote for the default floating point format to be decimal, with binary floats available with a b suffix. But since that could be a decade away, it's quite premature to spend too much time on this. -- Steven From python at mrabarnett.plus.com Tue Mar 4 03:10:29 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 04 Mar 2014 02:10:29 +0000 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140304015516.GM28804@ando> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com> <20140304015516.GM28804@ando> Message-ID: <53153615.4070304@mrabarnett.plus.com> On 2014-03-04 01:55, Steven D'Aprano wrote: > On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote: > >> You could probably make the same performance argument against making >> Unicode the default string datatype. > > I don't think so -- for ASCII strings the performance cost of Unicode is > significantly less than the performance hit for Decimal: > > [steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()" > 100000 loops, best of 3: 8.76 usec per loop > [steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()" > 100000 loops, best of 3: 7.05 usec per loop > > [steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6" > 1000000 loops, best of 3: 0.344 usec per loop > [steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \ >> -s "x = Decimal('123.4567')" "x**6" > 1000000 loops, best of 3: 1.41 usec per loop > > > That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal. > I think that's *fast enough* for all but the most heavy numeric needs, > but it's not something we can ignore. > > >> But a stronger argument is that >> the default string should be the one that does the right thing with >> text. As of Python 3, that's the case. And the default integer type >> handles arbitrary sized integers (although Py2 went most of the way >> there by having automatic promotion). It's reasonable to suggest that >> the default non-integer numeric type should also simply do the right >> thing. > > Define "the right thing" for numbers. > >> It's a trade-off, though, and for most people, float is sufficient. > > That's a tricky one. For people doing quote-unquote "serious" numeric > work, they'll mostly want to stick to binary floats, even if that means > missing out on all the extra IEEE-754 goodies that the decimal module > has but floats don't. The momentum of 40+ years of almost entirely > binary floating point maths does not shift to decimal overnight. > [snip] Won't people doing quote-unquote "serious" numeric work be using numpy? From mertz at gnosis.cx Tue Mar 4 03:17:18 2014 From: mertz at gnosis.cx (David Mertz) Date: Mon, 3 Mar 2014 18:17:18 -0800 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140304001048.22d6442c@fsol> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> <20140304001048.22d6442c@fsol> Message-ID: <CAEbHw4aHbAku7_1yR_uMNrLZP-zQaEjedhqvkP7jJH4UzsA-eA@mail.gmail.com> On Mon, Mar 3, 2014 at 3:10 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Mon, 3 Mar 2014 15:07:57 -0800 > David Mertz <mertz at gnosis.cx> wrote: > > On Mon, Mar 3, 2014 at 3:01 PM, Antoine Pitrou <solipsis at pitrou.net> > wrote: > >> > For the record, int doesn't have a sqrt() method while Decimal has, so > > > if you wanna take the exact square root of a large integer, you'd > better > > > convert it to a Decimal. > > Well, actually, if you want to take the square root of a large integer, > > most times you'll need an irrational number as a value. > > Well, unless you know by construction that your integer is a perfect > square. > Umm... if you construct your integer as a perfect square, wouldn't it be easier just to store the number it is a perfect square of than to work on optimizing the integer sqrt() function? It does make me wonder--although this is definitely not actually python-ideas--whether there is any technique to determine if a number is a perfect square that takes less work than finding its integral root. Maybe so, I don't know very much number theory. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/799961a0/attachment.html> From musicdenotation at gmail.com Tue Mar 4 04:39:27 2014 From: musicdenotation at gmail.com (Brian Nguyen) Date: Tue, 4 Mar 2014 10:39:27 +0700 Subject: [Python-ideas] Local scope for statement blocks Message-ID: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com> do: nonlocal x k = init for i in list: k = f(i,k) n = f2(n,k) From rosuav at gmail.com Tue Mar 4 04:40:00 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 4 Mar 2014 14:40:00 +1100 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140304015516.GM28804@ando> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <CAPTjJmpob_TJtgueUQSD7DWAXRYGKQELYW8xuyCCojWjTK=9aw@mail.gmail.com> <20140304015516.GM28804@ando> Message-ID: <CAPTjJmp2Y8BQ8z9-CgL9jk1iYXzYwUynC+ORwTCJdjSqC6=Hkw@mail.gmail.com> On Tue, Mar 4, 2014 at 12:55 PM, Steven D'Aprano <steve at pearwood.info> wrote: > On Tue, Mar 04, 2014 at 10:42:57AM +1100, Chris Angelico wrote: > >> You could probably make the same performance argument against making >> Unicode the default string datatype. > > I don't think so -- for ASCII strings the performance cost of Unicode is > significantly less than the performance hit for Decimal: > > [steve at ando ~]$ python3.3 -m timeit -s "s = 'abcdef'*1000" "s.upper()" > 100000 loops, best of 3: 8.76 usec per loop > [steve at ando ~]$ python3.3 -m timeit -s "s = b'abcdef'*1000" "s.upper()" > 100000 loops, best of 3: 7.05 usec per loop > > [steve at ando ~]$ python3.3 -m timeit -s "x = 123.4567" "x**6" > 1000000 loops, best of 3: 0.344 usec per loop > [steve at ando ~]$ python3.3 -m timeit -s "from decimal import Decimal" \ >> -s "x = Decimal('123.4567')" "x**6" > 1000000 loops, best of 3: 1.41 usec per loop > > > That's a factor of 1.2 times slower for Unicode versus 4.1 for Decimal. > I think that's *fast enough* for all but the most heavy numeric needs, > but it's not something we can ignore. There is a difference of degree, yes, but Unicode-strings-as-default has had a few versions to settle in, so the figures mightn't be perfectly fair. But there's still a difference. My point is that Python should be choosing what's right over what's fast, so there's a parallel there. >> It's reasonable to suggest that >> the default non-integer numeric type should also simply do the right >> thing. > > Define "the right thing" for numbers. Yeah, and that's the issue. In this case, since computers don't have infinite computational power, "the right thing" is going to be fairly vague, but I'd define it heuristically as "what the average programmer is most likely to expect". IEEE 754 defines operations on infinity in a way that makes them do exactly what you'd expect. If that's not possible, nan. >>> inf=float("inf") >>> inf+5 inf >>> 5-inf -inf >>> 5/inf 0.0 >>> inf-inf nan A default decimal type would add to the "doing exactly what you expect" operations the obvious one of constructing an object from a series of decimal digits. If you say "0.1", you get the real number 1/10, not 3602879701896397/36028797018963968. > My personal feeling is that for Python 4000 I'd vote for the default > floating point format to be decimal, with binary floats available with a > b suffix. Quite possibly. But changing defaults is a hugely backward-incompatible change, while adding a decimal literal syntax isn't. I'd be in favour of adding decimal literals and using performance and usefulness data from that to guide any discussions about Py4K changing the default. And learn from Py3K and keep both the b and d suffixes supported in the new version :) ChrisA From rosuav at gmail.com Tue Mar 4 04:52:49 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 4 Mar 2014 14:52:49 +1100 Subject: [Python-ideas] Local scope for statement blocks In-Reply-To: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com> References: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com> Message-ID: <CAPTjJmruw7sEXY4prfQKcfRWxp7zBBUiatVoFJCXjWZBUM0piA@mail.gmail.com> On Tue, Mar 4, 2014 at 2:39 PM, Brian Nguyen <musicdenotation at gmail.com> wrote: > do: > nonlocal x > k = init > for i in list: > k = f(i,k) > n = f2(n,k) You may need to elaborate a bit, especially as you have a declaration regarding 'x' and never use x anywhere. But if, as I suspect, you intend for this to have local k/i/n and fetch init/list/f/f2 from its parent scope, then this is something that's been discussed a few times. With the current implementation of CPython, the easiest and cleanest way to do this is with a closure. You could spell 'do' like this: def do(f): f() @do def _(): nonlocal x k = init for i in list: k = f(i,k) n = f2(n,k) Put that inside a function and it'll do most of what I think you're saying. ChrisA From aquavitae69 at gmail.com Tue Mar 4 07:02:46 2014 From: aquavitae69 at gmail.com (David Townshend) Date: Tue, 4 Mar 2014 08:02:46 +0200 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140301034611.GD28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> Message-ID: <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> On 1 Mar 2014 05:46, "Steven D'Aprano" <steve at pearwood.info> wrote: > > On Fri, Feb 28, 2014 at 11:17:18AM -0600, Ron Adam wrote: > > > > Starting new thread because this bike has a different shape and color. > > > > Yesterday I was thinking that just making the keyword lambda assignable > > like True, False, and None, would be enough. > > You can't assign to True, False or None. (You can assign to True and > False in Python 2, but shouldn't.) > > > [...] > > This morning I thought we could have in a functions definition something, > > like "*", and "**", to take an expression. Similar to Nicks idea with =:, > > but more general. > > > > The idea is to have "***" used in def mean to take "any" call expression > > and not evaluate it until *** is used on it. > [...] > > A function call that captures an expression may be tricky to do. Here's one > > approach that requires sugar when a function defined with "***" is called. > > I think it would be useful to have a way to delay execution of an > expression, that is to say, have a way to capture an expression for > later evaluation, something more lightweight than a function, but I > don't think that limiting it to inside function calls is the right > approach. > > Something perhaps like a thunk might be appropriate? We can *almost* do > that now, since Python has a compile function: > > thunk = compile("x + 3", "", "eval") > # much later > eval(thunk) What about a new code literal object, e.g. thunk = c'x + 3' thunk(x=2) The "c" prefix identifies the literal as an code so highlighters can recognise it and it can be parsed. Triple quotes could also be used to support multiline code blocks. It would also be possible to prohibit any direct conversion from a string to avoid the security problems. I'm not sure how positional arguments would be handled though, and scoping still needs to be thought through. > > Some problems with that approach: > > - You have to write the expression as a string, which means you lose any > possibility of syntax highlighting. > > - The temptation is to pass some arbitrary untrusted string, which leads > to serious security implications. The argument here should be limited > to an actual expression, not a string containing an expression which > might have come from who knows where. > > - It should be as lightweight as possible. The actual compilation of the > expression should occur at compile-time, not run-time. That implies some > sort of syntax for making thunks, rather than a function call. > > - Likewise actually evaluating the thunk should be really lightweight, > which may rule out a function call to eval. > > - How should scoping work? I can see use-cases for flat scoping, static > scoping, dynamic scoping, and the ability to optionally provide custom > globals and locals, but I have no idea how practical any of them would > be or what syntax they should use. > > > All of this is pie-in-the-sky at the moment, and not a serious proposal > for Python 3.5. > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/741cac8a/attachment-0001.html> From abarnert at yahoo.com Tue Mar 4 07:51:17 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 3 Mar 2014 22:51:17 -0800 Subject: [Python-ideas] Local scope for statement blocks In-Reply-To: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com> References: <CAL0FH+3KxF-4BKVsZ+nWUWQk3wckXx4TaxVMCk5qsS5wRSXYug@mail.gmail.com> Message-ID: <CF05DD08-F2EA-4D76-833E-3F34BE3FDB71@yahoo.com> On Mar 3, 2014, at 19:39, Brian Nguyen <musicdenotation at gmail.com> wrote: > do: > nonlocal x > k = init > for i in list: > k = f(i,k) > n = f2(n,k) You want _every_ compound statement to be a scope, or you just want to add a new one, a "do" statement, and only that new one is a scope? If the latter, i believe you can do this pretty easily with a MacroPy macro (which will expand to a def and a function call). It might be worth playing with that to get some real-life examples of using it. From abarnert at yahoo.com Tue Mar 4 08:03:03 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 3 Mar 2014 23:03:03 -0800 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> Message-ID: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote: > What about a new code literal object, e.g. > > thunk = c'x + 3' > thunk(x=2) > Why does it need to be built from/look like a string? I think it would be just as simple for the parser, and simpler for editors, and less misleading for readers, if it used a different marker. Not that I'm seriously suggesting backticks here, but... thunk = `x + 3` Most existing editors already treat that as an expression (since in 2.x it means repr(x + 3)). No human reader is going to be misled into thinking this means there's a way to create code objects from string objects without eval (c.f. all those questions on StackOverflow about how to make a raw string from a string...). And as far as the parser is concerned, it's trivial: '`' expr '`' is a Code whose value is whatever expr parses to. Meanwhile, if this gives you a code object, what do you do with it? Using eval on something that looks like a string but isn't may be correct to anyone who understands Python deeply, but I think it could be very misleading to anyone else. And creating a FunctionType from a code object is pretty ugly for something that deserved literal support... > Triple quotes could also be used to support multiline code blocks. > That is one advantage of reusing strings. In fact, that gives us a way to embed statements in the middle of expressions. Which I'm not sure is a good thing. > I'm not sure how positional arguments would be handled though, and scoping still needs to be thought through. > If you have a code object, scoping is up to whoever evaluates it or builds a function out of it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/553d0281/attachment.html> From bruce at leapyear.org Tue Mar 4 08:26:13 2014 From: bruce at leapyear.org (Bruce Leban) Date: Mon, 3 Mar 2014 23:26:13 -0800 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> Message-ID: <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com> On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote: > > What about a new code literal object, e.g. > > thunk = c'x + 3' > thunk(x=2) > > This already exists. thunk = lambda: x + 3 thunk(x=2) Your original post in this thread wanted to allow functions to magically capture their parameters as code (~ call by name) and it's been pointed out that that is incompatible with the way the language is designed and function calls are implemented. Yes, there are languages where the called function gets to tell the caller how to package arguments but Python isn't one of them. Now you seem to have veered in a different direction. I have no idea what problem you are trying to solve. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140303/98627761/attachment.html> From p.f.moore at gmail.com Tue Mar 4 09:01:44 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 4 Mar 2014 08:01:44 +0000 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com> Message-ID: <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com> On 4 March 2014 01:37, Mark H. Harris <harrismh777 at gmail.com> wrote: > I released my pdeclib module on PyPI this afternoon (it only took four > hours) but I'll never forget how to do that again ! So if anyone is > interested > in the pdeclib module, it may be installed with pip install pdeclib Regardless of anything else that comes out of this thread, thanks for doing that. Paul From aquavitae69 at gmail.com Tue Mar 4 10:07:28 2014 From: aquavitae69 at gmail.com (David Townshend) Date: Tue, 4 Mar 2014 11:07:28 +0200 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <CAGu0Anssx-HHx=S=b6TCChFSDXyToa05E7TCHQf+wJFSvUqxnA@mail.gmail.com> Message-ID: <CAEgL-fcWYrGP=NKEziZzS4pT5b+Q-dyKO4W7EcqFjVOkp649Rg@mail.gmail.com> On Tue, Mar 4, 2014 at 9:26 AM, Bruce Leban <bruce at leapyear.org> wrote: > > > On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote: >> >> What about a new code literal object, e.g. >> >> thunk = c'x + 3' >> thunk(x=2) >> >> This already exists. > > thunk = lambda: x + 3 > thunk(x=2) > > Your original post in this thread wanted to allow functions to magically > capture their parameters as code (~ call by name) and it's been pointed out > that that is incompatible with the way the language is designed and > function calls are implemented. Yes, there are languages where the called > function gets to tell the caller how to package arguments but Python isn't > one of them. > > Now you seem to have veered in a different direction. I have no idea what > problem you are trying to solve. > > This is my first post in this thread - I don't recall who the OP was, but it wasn't me. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/d05a0fe2/attachment-0001.html> From aquavitae69 at gmail.com Tue Mar 4 10:03:35 2014 From: aquavitae69 at gmail.com (David Townshend) Date: Tue, 4 Mar 2014 11:03:35 +0200 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> Message-ID: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com> On Tue, Mar 4, 2014 at 9:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote: > On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote: > > What about a new code literal object, e.g. > > thunk = c'x + 3' > thunk(x=2) > > Why does it need to be built from/look like a string? I think it would be > just as simple for the parser, and simpler for editors, and less misleading > for readers, if it used a different marker. > > Not that I'm seriously suggesting backticks here, but... > > thunk = `x + 3` > > The only real reason for it looking like a string is a shortage of symbols. Backticks are off limits, and most other symbols are just plain ugly (e.g. @x + 3@ or $x + 3$) or already in use in a way that could lead to ambiguity (e.g. |x + 3|). String-like quotes seem like a better option than the alternatives. > Most existing editors already treat that as an expression (since in 2.x it > means repr(x + 3)). No human reader is going to be misled into thinking > this means there's a way to create code objects from string objects without > eval (c.f. all those questions on StackOverflow about how to make a raw > string from a string...). And as far as the parser is concerned, it's > trivial: '`' expr '`' is a Code whose value is whatever expr parses to. > > Meanwhile, if this gives you a code object, what do you do with it? Using > eval on something that looks like a string but isn't may be correct to > anyone who understands Python deeply, but I think it could be very > misleading to anyone else. And creating a FunctionType from a code object > is pretty ugly for something that deserved literal support... > My suggestion was to use c'x + 3' as an effective replacement for compile('x + 3', '', 'eval'). The use of a literal rather than compile would solve a few of the issues raised in Steven's original post. I don't see why it would be any more misleading to eval a code literal than a compiled string. I did suggest making it callable (i.e. thunk(x=3), but on second thoughts I'm not sure that's a good idea. It adds complexity without and real benefit over eval. > Triple quotes could also be used to support multiline code blocks. > > That is one advantage of reusing strings. In fact, that gives us a way to > embed statements in the middle of expressions. Which I'm not sure is a good > thing. > > I'm not sure how positional arguments would be handled though, and scoping > still needs to be thought through. > > If you have a code object, scoping is up to whoever evaluates it or builds > a function out of it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/9c2a877f/attachment.html> From greg.ewing at canterbury.ac.nz Tue Mar 4 10:18:13 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 04 Mar 2014 22:18:13 +1300 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <20140304001048.22d6442c@fsol> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> <20140304001048.22d6442c@fsol> Message-ID: <53159A55.9070905@canterbury.ac.nz> Antoine Pitrou wrote: > Well, unless you know by construction that your integer is a perfect > square. If you've constructed it that way, you probably already know its square root. -- Greg From abarnert at yahoo.com Tue Mar 4 11:57:57 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 4 Mar 2014 02:57:57 -0800 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <53159A55.9070905@canterbury.ac.nz> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz> Message-ID: <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com> On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Antoine Pitrou wrote: >> Well, unless you know by construction that your integer is a perfect >> square. > > If you've constructed it that way, you probably already > know its square root. Usually, but not always. For example, there are algorithms to generate squares of Pythagorean triples without generating the triples themselves. Of course there are simpler and more efficient algorithms to just generate the triples (can't get much simpler than Euclid's formula...), but there might be some reason you're using one of the square algorithms. And then, to test it, you'd need to verify that a2+b2==c2 and that all three of them are perfect squares. From steve at pearwood.info Tue Mar 4 12:09:21 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Mar 2014 22:09:21 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> Message-ID: <20140304110921.GP28804@ando> On Mon, Mar 03, 2014 at 11:03:03PM -0800, Andrew Barnert wrote: > On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com> wrote: > > What about a new code literal object, e.g. > > > > thunk = c'x + 3' Off the top of my head, I think that looks alright. > > thunk(x=2) I'm not entirely sure about that. That basically makes these thunks just a function. I'm still not quite sure how this should actually be used in practice, so as far as I'm concerned this is just pie in the sky thinking aloud. If thunks are just functions, why not make them functions? There needs to be something extra (different scoping rules, faster/more lightweight, *something*) to make the idea worthwhile. I think I need to learn more about Algol and other languages that use call-by-name and thunks. I may be completely on a wild-goose chase here, but I'm surely not the only person who has needed to delay evaluation of an expression. http://en.wikipedia.org/wiki/Thunk > Why does it need to be built from/look like a string? I think it would > be just as simple for the parser, and simpler for editors, and less > misleading for readers, if it used a different marker. > > Not that I'm seriously suggesting backticks here, but... > > thunk = `x + 3` Actually I'd be happy with backticks, but Guido has said No Backticks Ever Again. So until the Glorious Revolution, we're stuck. > Meanwhile, if this gives you a code object, what do you do with it? > Using eval on something that looks like a string but isn't may be > correct to anyone who understands Python deeply, but I think it could > be very misleading to anyone else. And creating a FunctionType from a > code object is pretty ugly for something that deserved literal > support... Agreed. What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol. Again, just thinking aloud, perhaps we do this: thunk = `some_expression` # delays evaluation a = [0, 1 + thunk] # evaluates thunk in the current scope b = [0, `1 + thunk`] # delays evaluation and creates a thunk object # equivalent to `1 + some_expression` c = b[1] # now evaluates the thunk object d = f(2, thunk) # evaluates thunk in f's scope e = g(3, `thunk`) # passes the un-evaluated thunk object to g Consider this just a sketch, and in no way fully thought out. (This will most definitely need a PEP.) > > Triple quotes could also be used to support multiline code blocks. > > > That is one advantage of reusing strings. In fact, that gives us a way > to embed statements in the middle of expressions. Which I'm not sure > is a good thing. I have little interest in allowing thunks to be statements. If you want a delayed statement, use compile and eval. Or def. (But maybe that applies to expressions too?) Did I mention this needs a PEP? -- Steven From steve at pearwood.info Tue Mar 4 12:23:20 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Mar 2014 22:23:20 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com> Message-ID: <20140304112320.GQ28804@ando> On Tue, Mar 04, 2014 at 11:03:35AM +0200, David Townshend wrote: > My suggestion was to use c'x + 3' as an effective replacement for > compile('x + 3', '', 'eval'). The use of a literal rather than compile > would solve a few of the issues raised in Steven's original post. I don't > see why it would be any more misleading to eval a code literal than a > compiled string. Agreed. > I did suggest making it callable (i.e. thunk(x=3), but on > second thoughts I'm not sure that's a good idea. It adds complexity > without and real benefit over eval. I don't think calling the thunk is the right API. If we have this sort of functionality, I want calling a thunk to be treated like any other expression, thunk[0] or thunk+1. I don't want it to mean "evaluate this thunk". An example: thunk = `lambda x: x + y` def func(y, obj): return obj + 1 result = func(50, thunk(100)) (This is not a use-case, just an illustration.) In this example, the expression "thunk(1000)" is delayed until inside the func scope. Inside that scope, we evaluate this expression: (lambda x: x + y)(100) where y has the value 50, and bind the result of this to the name obj. The lambda part returns a closure, and calling that closure with argument 100 returns 150, so this is equivalent to binding obj=150. Then func continues, evaluates "obj + 1", and returns 151. -- Steven From steve at pearwood.info Tue Mar 4 12:25:35 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 4 Mar 2014 22:25:35 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140304110921.GP28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> Message-ID: <20140304112535.GR28804@ando> On Tue, Mar 04, 2014 at 10:09:21PM +1100, Steven D'Aprano wrote: > Agreed. What I have in my head is some vague concept that the Python > evaluation rules will somehow know when to evaluate the thunk and when > to treat it as an object, which is (as I understand it) what happens in > Algol. Since Algol is not an OOP language, that would be "value", not "object". -- Steven From rosuav at gmail.com Tue Mar 4 13:07:46 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 4 Mar 2014 23:07:46 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140304110921.GP28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> Message-ID: <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> On Tue, Mar 4, 2014 at 10:09 PM, Steven D'Aprano <steve at pearwood.info> wrote: > Agreed. What I have in my head is some vague concept that the Python > evaluation rules will somehow know when to evaluate the thunk and when > to treat it as an object, which is (as I understand it) what happens in > Algol. Again, just thinking aloud, perhaps we do this: > > thunk = `some_expression` # delays evaluation > a = [0, 1 + thunk] # evaluates thunk in the current scope > b = [0, `1 + thunk`] # delays evaluation and creates a thunk object > # equivalent to `1 + some_expression` > c = b[1] # now evaluates the thunk object > d = f(2, thunk) # evaluates thunk in f's scope > e = g(3, `thunk`) # passes the un-evaluated thunk object to g > > Consider this just a sketch, and in no way fully thought out. > > (This will most definitely need a PEP.) PEP can come later. First, let's get some solid use-cases, and start looking at implications. The way it's described here, there's effectively magic when you try to look at an object of this type, which will break a lot of assumptions. Most people expect that: foo = bar assert foo is bar to be a safe assumption, but if bar is a thunk, then it's getting evaluated separately in each of those, and that's potentially going to create different objects and/or even have side effects. That's going to surprise people. On the flip side, that's something that could be dealt with with a naming convention for thunks. We have _private, __mangled, __magic__, anticollision_, CONSTANT, Class... maybe we could have thunk__ or something. It's most confusable with magic and anticollision, but since both of those are connected with specific keywords, it's reasonably likely there'll be no actual confusion. Specific downside: There's no way to actually pass an unevaluated thunk around. Technically, `thunk__` will create a new wrapper thunk and pass that along. That'll often have the same effect, but it won't be quite identical (same as there's a difference between f(g) and f(lambda x: g(x)) in that the second one lazily looks up g), so that might cause confusion. But you could always special-case it: writing `thunk__` will be guaranteed to transmit the thunk unchanged, and if you actually mean to add another wrapper layer, use `(thunk__)` or something instead. The biggest thing to figure out is scoping. Does a thunk take a snapshot of its enclosing scope (a closure), or is it an expression that gets evaluated in the target namespace? The latter could be extremely confusing, but the former's just what a nested function does, so this'd just be a new lambda syntax. Python's existing lambda syntax has its flaws and its detractors, but it has one huge benefit over thunking: It exists. :) Thunking has to get over that difference. I'd like to see some proposals. ChrisA From masklinn at masklinn.net Tue Mar 4 13:50:19 2014 From: masklinn at masklinn.net (Masklinn) Date: Tue, 4 Mar 2014 13:50:19 +0100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> Message-ID: <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote: > On Tue, Mar 4, 2014 at 10:09 PM, Steven D'Aprano <steve at pearwood.info> wrote: >> Agreed. What I have in my head is some vague concept that the Python >> evaluation rules will somehow know when to evaluate the thunk and when >> to treat it as an object, which is (as I understand it) what happens in >> Algol. Again, just thinking aloud, perhaps we do this: >> >> thunk = `some_expression` # delays evaluation >> a = [0, 1 + thunk] # evaluates thunk in the current scope >> b = [0, `1 + thunk`] # delays evaluation and creates a thunk object >> # equivalent to `1 + some_expression` >> c = b[1] # now evaluates the thunk object >> d = f(2, thunk) # evaluates thunk in f's scope >> e = g(3, `thunk`) # passes the un-evaluated thunk object to g >> >> Consider this just a sketch, and in no way fully thought out. >> >> (This will most definitely need a PEP.) > > PEP can come later. First, let's get some solid use-cases, and start > looking at implications. The way it's described here, there's > effectively magic when you try to look at an object of this type, > which will break a lot of assumptions. Most people expect that: > > foo = bar > assert foo is bar > > to be a safe assumption, but if bar is a thunk, then it's getting > evaluated separately in each of those Why? Either it's forced during assignment or both names map to the same thunk, and are both forced when any of them is. That could be during the identity check, but since both names refer to the same thunk they can only yield the same value, so the identity check needs not force the thunk. An equality test would likely force the thunk. > so that > might cause confusion. But you could always special-case it: writing > `thunk__` will be guaranteed to transmit the thunk unchanged, and if > you actually mean to add another wrapper layer, use `(thunk__)` or > something instead. Is there a use case for actually thunking a thunk? > The biggest thing to figure out is scoping. Does a thunk take a > snapshot of its enclosing scope (a closure), or is it an expression > that gets evaluated in the target namespace? The latter could be > extremely confusing, but the former's just what a nested function > does, so this'd just be a new lambda syntax. That is essentially what a thunk is, at least in my experience: it is conceptually a nullary memoized function, forced (evaluated/called) if an actual value/object is ever needed but potentially thrown out during reduction. The difference, under the proposed semantics, is that the forcing of the thunk would be implicit where that of a function is explicit (not sure that's a good idea in a strict language). From python at mrabarnett.plus.com Tue Mar 4 14:30:32 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 04 Mar 2014 13:30:32 +0000 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <CAEgL-feFoF=n3AzUDQ1KLX6tn7Y1T3w8AWskE625YnK3jkyd7A@mail.gmail.com> Message-ID: <5315D578.3040803@mrabarnett.plus.com> On 2014-03-04 09:03, David Townshend wrote: > > > > On Tue, Mar 4, 2014 at 9:03 AM, Andrew Barnert <abarnert at yahoo.com > <mailto:abarnert at yahoo.com>> wrote: > > On Mar 3, 2014, at 22:02, David Townshend <aquavitae69 at gmail.com > <mailto:aquavitae69 at gmail.com>> wrote: >> >> What about a new code literal object, e.g. >> >> thunk = c'x + 3' >> thunk(x=2) >> > Why does it need to be built from/look like a string? I think it > would be just as simple for the parser, and simpler for editors, and > less misleading for readers, if it used a different marker. > > Not that I'm seriously suggesting backticks here, but... > > thunk = `x + 3` > > > The only real reason for it looking like a string is a shortage of > symbols. Backticks are off limits, and most other symbols are just > plain ugly (e.g. @x + 3@ or $x + 3$) or already in use in a way that > could lead to ambiguity (e.g. |x + 3|). String-like quotes seem like a > better option than the alternatives. > [snip] I'm sure that Unicode could provide some characters/codepoints. For example: thunk = ?x + 3? From rosuav at gmail.com Tue Mar 4 15:14:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 01:14:09 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> Message-ID: <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com> On Tue, Mar 4, 2014 at 11:50 PM, Masklinn <masklinn at masklinn.net> wrote: > On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote: >> Most people expect that: >> >> foo = bar >> assert foo is bar >> >> to be a safe assumption, but if bar is a thunk, then it's getting >> evaluated separately in each of those > > Why? Either it's forced during assignment or both names map to > the same thunk, and are both forced when any of them is. > > That could be during the identity check, but since both names refer to > the same thunk they can only yield the same value, so the identity check > needs not force the thunk. An equality test would likely force the > thunk. Most certainly not. Try this: bar = `[1,2,3]` foo = bar spam = bar assert foo is spam Here's the evaluated version: foo = [1,2,3] spam = [1,2,3] assert foo is spam You can try this one out directly. They're not going to be the same object - they'll be two separate lists. They will be equal, in this case, but there's no guarantee of that either: bar = `random.random()` Separate evaluation of the same expression isn't guaranteed to have the same result, AND it might have unexpected side effects: bar = `whiskey.pop().drink()` and you might find yourself underneath the bar before you know it. ChrisA From masklinn at masklinn.net Tue Mar 4 15:55:37 2014 From: masklinn at masklinn.net (Masklinn) Date: Tue, 4 Mar 2014 15:55:37 +0100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com> Message-ID: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net> On 2014-03-04, at 15:14 , Chris Angelico <rosuav at gmail.com> wrote: > On Tue, Mar 4, 2014 at 11:50 PM, Masklinn <masklinn at masklinn.net> wrote: >> On 2014-03-04, at 13:07 , Chris Angelico <rosuav at gmail.com> wrote: >>> Most people expect that: >>> >>> foo = bar >>> assert foo is bar >>> >>> to be a safe assumption, but if bar is a thunk, then it's getting >>> evaluated separately in each of those >> >> Why? Either it's forced during assignment or both names map to >> the same thunk, and are both forced when any of them is. >> >> That could be during the identity check, but since both names refer to >> the same thunk they can only yield the same value, so the identity check >> needs not force the thunk. An equality test would likely force the >> thunk. > > Most certainly not. Try this: > > bar = `[1,2,3]` > foo = bar > spam = bar > assert foo is spam > > Here's the evaluated version: > > foo = [1,2,3] > spam = [1,2,3] > assert foo is spam I don't agree with this, again why would the thunk be evaluated twice? If thunks are added to *delay* expression evaluation (which is what I understood from Steven's messages) surely something akin to Haskell's semantics is simpler to understand and implement. That is, instead of thunks being sugar for: bar = lambda: expr they're sugar for bar = memoize(lambda: expr) > You can try this one out directly. They're not going to be the same > object - they'll be two separate lists. They will be equal, in this > case, but there's no guarantee of that either: > > bar = `random.random()` > > Separate evaluation of the same expression isn't guaranteed to have > the same result Obviously, but why would repeated evaluation of the expression be desirable? > AND it might have unexpected side effects: > > bar = `whiskey.pop().drink()` > > and you might find yourself underneath the bar before you know it. From rosuav at gmail.com Tue Mar 4 16:11:42 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 02:11:42 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com> <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net> Message-ID: <CAPTjJmqO7_h7k0qoLtcq59YKR5u6x9g1tWfccWW+OoZimoE=FA@mail.gmail.com> On Wed, Mar 5, 2014 at 1:55 AM, Masklinn <masklinn at masklinn.net> wrote: > I don't agree with this, again why would the thunk be evaluated twice? > If thunks are added to *delay* expression evaluation (which is what I > understood from Steven's messages) surely something akin to Haskell's > semantics is simpler to understand and implement. That is, instead > of thunks being sugar for: > > bar = lambda: expr > > they're sugar for > > bar = memoize(lambda: expr) Okay. That's an interesting point, and a distinction from lambda. Effectively, once a thunk is evaluated once, it devolves to its value. That would be *extremely* interesting in the case of lazy evaluation - you could actually make a ternary-if function: value = true_expr if cond else false_expr def if_(cond, true_thunk, false_thunk): if cond: return true_thunk return false_thunk value = if_(cond, `true_expr`, `false_expr`) There's still the questions of scoping, but now you have the distinction between a thunk and a function. And if it's at all possible, the thunk could even replace itself in memory with its result - that would be massively implementation-dependent, but since you can't look at the identity of the thunk itself anyway, it wouldn't break anything. Effectively, as soon as you evaluate bar, it assigns to bar whatever the expression evaluates to. (Which is simpler and cleaner to explain than the memoization.) ChrisA From steve at pearwood.info Tue Mar 4 16:19:59 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 02:19:59 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <CAPTjJmpDF4VEqMmOKHJRMgz1RfLWyPDtxbtG-JyG+GgNh6UGuQ@mail.gmail.com> <931BAA6E-74DB-4997-8307-401D24B1CADF@masklinn.net> <CAPTjJmo+FiZp_4JOxCURi4dpGoo_PgztUwjJ2=sOpcae_mgQ2Q@mail.gmail.com> <41D7DF2D-67B2-438D-A064-97003C595E7D@masklinn.net> Message-ID: <20140304151958.GS28804@ando> On Tue, Mar 04, 2014 at 03:55:37PM +0100, Masklinn wrote: > I don't agree with this, again why would the thunk be evaluated twice? > If thunks are added to *delay* expression evaluation (which is what I > understood from Steven's messages) surely something akin to Haskell's > semantics is simpler to understand and implement. That is, instead > of thunks being sugar for: > > bar = lambda: expr > > they're sugar for > > bar = memoize(lambda: expr) +1 -- Steven From guido at python.org Tue Mar 4 18:20:20 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 4 Mar 2014 09:20:20 -0800 Subject: [Python-ideas] PEP-257: drop recommendation for extra line at end of multi-line docstring In-Reply-To: <2d6d1d67-bf89-4b33-acfe-4db2bd5e28e3@googlegroups.com> References: <30E13F8A-EE63-4E0F-A29A-F35B5906791C@rackspace.com> <CAGu0AnuW++aU5OAm2LTOvu3Nh7jieQfC4=RCgJtjj10-RG8UkA@mail.gmail.com> <1FD9DEBB-1E68-4BB0-9349-B3FA2598D627@rackspace.com> <CAP7+vJJ+SwXDWFvMgk-6BMdV=U6x80EF+8_HRTz-u=moT088aw@mail.gmail.com> <C22E0B99-4FEC-4AB3-B251-80805D8F2C37@rackspace.com> <CAP7+vJLKh1hi8aQZyKmiTb87PDWeTWOvj+hgfgLtTYAA4VnvsA@mail.gmail.com> <991C04FE-35F2-4641-B240-A353AD31A9E3@rackspace.com> <2d6d1d67-bf89-4b33-acfe-4db2bd5e28e3@googlegroups.com> Message-ID: <CAP7+vJLF65FLjJ8j7M-ThDQVOSvKhNrKsY0pvv554io19cuBRA@mail.gmail.com> Removed. On Tue, Mar 4, 2014 at 8:51 AM, Simon Kennedy <sffjunkie at gmail.com> wrote: > On Monday, 3 March 2014 18:55:17 UTC, Ziad Sawalha wrote: >> >> Thanks, Guido. >> >> I'll follow up with updates to common tools as I come across them (ex. >> pep257: https://github.com/GreenSteam/pep257/pull/64). >> >> Ziad >> >> > The footnote's still in the PEP text. > > --Simon > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/44bbb0e4/attachment.html> From greg.ewing at canterbury.ac.nz Tue Mar 4 23:17:17 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2014 11:17:17 +1300 Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz> <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com> Message-ID: <531650ED.1050206@canterbury.ac.nz> Andrew Barnert wrote: > On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > >> If you've constructed it that way, you probably already know its square >> root. > > Usually, but not always. In any case, the fact remains that any algorithm for calculating the square root of a perfect square will work just as well in any base. There's nothing special about decimal in that regard. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 4 23:31:41 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2014 11:31:41 +1300 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140304110921.GP28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> Message-ID: <5316544D.7050302@canterbury.ac.nz> Steven D'Aprano wrote: > What I have in my head is some vague concept that the Python > evaluation rules will somehow know when to evaluate the thunk and when > to treat it as an object, which is (as I understand it) what happens in > Algol. But Algol has the benefit of static typing -- the procedure being called explicitly declares whether the argument is to be passed by name or value. Python has no idea about that at compile time. > b = [0, `1 + thunk`] # delays evaluation and creates a thunk object > # equivalent to `1 + some_expression` > c = b[1] # now evaluates the thunk object > d = f(2, thunk) # evaluates thunk in f's scope > e = g(3, `thunk`) # passes the un-evaluated thunk object to g When exactly does implicit evaluation of a thunk object occur? Does `b[1]` give you an unevaluated thunk object? What if b is a custom sequence type implemented in Python -- how does its __getitem__ method avoid evaluating the thunk object prematurely? None of these problems occur in Algol, because its thunks are not first-class values (you can't store them in arrays, etc.) and it uses static type information to tell when to create and evaluate them. -- Greg From rosuav at gmail.com Tue Mar 4 23:40:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 09:40:09 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <5316544D.7050302@canterbury.ac.nz> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> Message-ID: <CAPTjJmrvTxA0rhHM=0H78KNr6-gtkP54vvgJpLDX+08w-JiG-Q@mail.gmail.com> On Wed, Mar 5, 2014 at 9:31 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Does `b[1]` give you an unevaluated thunk object? I would say that `b[1]` should create a brand new thunk object of that expression. On evaluation of that, it'll look up b, subscript it, and touch the thunk. When it does, it Schrodingers itself into a value and that's that. In my opinion, __getitem__ isn't a problem, but __setitem__ is. Somewhere along the way, you have to be able to "hold" a thunk as a thunk. I've no idea how that works when you pass it around from function to function. ChrisA From solipsis at pitrou.net Wed Mar 5 00:08:12 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 5 Mar 2014 00:08:12 +0100 Subject: [Python-ideas] Python3.3 Decimal Library Released References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <20140303175429.GL28804@ando> <CAKJDb-NBnf8X5yJ-LmeFV=CfoTEOxBHHfahVcd6Jy=eX+5u2-A@mail.gmail.com> <CACac1F8PaFjnz5Wkh6z7WWzpzT=wOGd6GCySyYa2K5Ogp68_Gw@mail.gmail.com> <CAEbHw4YVKmRmvBFe1wwzh_4SGHQkPwWw+e-YMe8_EcOw-Kz2Kw@mail.gmail.com> <c31406e7-0d47-4885-99b1-7beb074cc4c2@googlegroups.com> <53150810.20507@canterbury.ac.nz> <20140304000122.4f61e287@fsol> <CAEbHw4YpVFeBPR3fQod1O15PLYRh9Qf3+pu5bENxJ8qRUaXw=g@mail.gmail.com> <20140304001048.22d6442c@fsol> <53159A55.9070905@canterbury.ac.nz> <3E61C95D-A73F-4E88-8E2D-5054D9D58935@yahoo.com> <531650ED.1050206@canterbury.ac.nz> Message-ID: <20140305000812.7676feca@fsol> On Wed, 05 Mar 2014 11:17:17 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Andrew Barnert wrote: > > On Mar 4, 2014, at 1:18, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > > > >> If you've constructed it that way, you probably already know its square > >> root. > > > > Usually, but not always. > > In any case, the fact remains that any algorithm for > calculating the square root of a perfect square will > work just as well in any base. There's nothing special > about decimal in that regard. The point was not decimal vs. binary, but arbitrarily high precision vs. fixed low precision. (i.e. if you have a 1000-digit integer, just calling N ** 0.5 will give you a very low-precision result compared to the integer's precision). Regards Antoine. From carl.input at gmail.com Wed Mar 5 00:09:02 2014 From: carl.input at gmail.com (Carl Smith) Date: Tue, 4 Mar 2014 23:09:02 +0000 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality Message-ID: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> The tentatively proposed idea here is using dollar signed expressions to define 'bills'. A bill object is essentially an expression which can be evaluated any number of times, potentially in different scopes. The following expression [a bill literal] would be pointless, but would define a bill that always evaluates to 1. $1 So, eval($1)==1. Some better examples... * assign a bill to `a` so that `a` will evaluate to the value of the name `foo` any time that `a` is evaluated, in the scope of that evaluation a = $foo * as above, but always plus one a = $foo + 1 * make `a` a bill that evaluates to the value of the name `foo` at the time that `a` is evaluated, in that scope, plus the value of `bar` **at the time and in the scope of the assignment to `a`** a = $foo + bar Note. Similarly to mixing floats with ints, any expression that contains a bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill too. Passing a bill to eval should be the obvious way to get the value. The point? It allows functions to accept bills to use internally. The function would specify any names the bill can reference in the function's API, like keywords. def f(b): # the bill arg `b` can reference `item` for item in something: if eval(b): return True f($item < 0) You could also use a function call, for example `$foo()` would evaluate to a bill that evaluates to a call to `foo` in the scope and at the time of any evaluation of the bill. I've no idea if this is even possible in Python, and have no hope of implementing it, but thought I'd share :) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/f5479771/attachment.html> From rymg19 at gmail.com Wed Mar 5 00:57:31 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Tue, 4 Mar 2014 17:57:31 -0600 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> Message-ID: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com> Only problem is that it looks a tad perlish... On Tue, Mar 4, 2014 at 5:09 PM, Carl Smith <carl.input at gmail.com> wrote: > The tentatively proposed idea here is using dollar signed expressions to > define 'bills'. A bill object is essentially an expression which can be > evaluated any number of times, potentially in different scopes. > > The following expression [a bill literal] would be pointless, but would > define a bill that always evaluates to 1. > > $1 > > So, eval($1)==1. > > Some better examples... > > * assign a bill to `a` so that `a` will evaluate to the value of the name > `foo` any time that `a` is evaluated, in the scope of that evaluation > > a = $foo > > * as above, but always plus one > > a = $foo + 1 > > * make `a` a bill that evaluates to the value of the name `foo` at the > time that `a` is evaluated, in that scope, plus the value of `bar` **at the > time and in the scope of the assignment to `a`** > > a = $foo + bar > > Note. Similarly to mixing floats with ints, any expression that contains a > bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill > too. Passing a bill to eval should be the obvious way to get the value. > > The point? It allows functions to accept bills to use internally. The > function would specify any names the bill can reference in the function's > API, like keywords. > > def f(b): # the bill arg `b` can reference `item` > for item in something: > if eval(b): return True > > f($item < 0) > > You could also use a function call, for example `$foo()` would evaluate to > a bill that evaluates to a call to `foo` in the scope and at the time of > any evaluation of the bill. > > I've no idea if this is even possible in Python, and have no hope of > implementing it, but thought I'd share :) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/e7c9b5d9/attachment-0001.html> From cs at zip.com.au Wed Mar 5 01:06:10 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 5 Mar 2014 11:06:10 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com> References: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com> Message-ID: <20140305000610.GA3326@cskk.homeip.net> On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote: > Only problem is that it looks a tad perlish... Of shellish. But worse, does nothing even remotely loke what that does in perl or shell (or basic or...). I was going to remark that in my mind I would always see: a = $foo + 1 and (1) mentally bind the "$" to "foo" alone, not treat it as a prefix to the whole expression and (2) expect "a" to evaluate right now, regardess. And then follow up that remark with a strong preference for forming the who expression inside a clear construct so you know at the outself that it is something for evaluation later. And then realised that is already spelt "lambda". I do take the point that this "bill" idea is meant to grant access to the local scope when used. Cheers, -- Cameron Simpson <cs at zip.com.au> Take a perfectly parallel uniform plank of constant density. Balance it exactly at its centre on a frictionnless pivot. Place the hog at a known distance from this pivot. Now add stones on the other end at exactly the same distance until the plank is perfectly horizontal as measured by a precision tiltmeter. Now guess the weight of the stones. - Bob Newhart on weighing a hog From jbvsmo at gmail.com Wed Mar 5 02:05:57 2014 From: jbvsmo at gmail.com (=?ISO-8859-1?Q?Jo=E3o_Bernardo?=) Date: Tue, 4 Mar 2014 22:05:57 -0300 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> Message-ID: <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com> > > * assign a bill to `a` so that `a` will evaluate to the value of the name > `foo` any time that `a` is evaluated, in the scope of that evaluation > > a = $foo > > * as above, but always plus one > > a = $foo + 1 > > * make `a` a bill that evaluates to the value of the name `foo` at the > time that `a` is evaluated, in that scope, plus the value of `bar` **at the > time and in the scope of the assignment to `a`** > > a = $foo + bar > > You can do all that programmatically. No need for magic operators. Check this module as a proof of concept: https://github.com/jbvsmo/funcbuilder foo = FuncBuilder() a = foo + 1 # Then evaluate a with a function call For it to do exactly what you want, you just need override the evaluation function (Funcbuilder.__call__) to load names from current frame locals or something. BTW, I don't recommend using that on real code because it is extremely experimental (unless you're a hipster... In this case, carry on). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/48623a83/attachment.html> From abarnert at yahoo.com Wed Mar 5 02:05:39 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 4 Mar 2014 17:05:39 -0800 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <5316544D.7050302@canterbury.ac.nz> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> Message-ID: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Steven D'Aprano wrote: >> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol. > > But Algol has the benefit of static typing -- the procedure > being called explicitly declares whether the argument is to > be passed by name or value. Python has no idea about that at > compile time. This is the main reason I think it's more productive to think of this in terms of Lisp-style quoting than Algol-style thunks. The fact that quoting/thunking gives you a code object instead of an AST (sexpr) is not significant here* (unless we're also considering adding macros). The fact that it gives you a first-class value, and that we can't use the "implicit casting" syntax that comes with static typing to magically evaluate the object at the right time, is critical. This is basically the same problem I described trying to implement Boost-style auto lambdas in Python without C++-style implicit cast from lambda to function. * It is significant _elsewhere_. In an AST, "b" is just a reference to a name; in a code object, we have to have already determined whether it's a reference to a fast, closure, or global name--unless you want to do something like automatically compile `b[1]` as if it were something like vars('b')[1], or to do something equivalent like add a LOAD_DYNAMIC opcode. >> b = [0, `1 + thunk`] # delays evaluation and creates a thunk object >> # equivalent to `1 + some_expression` >> c = b[1] # now evaluates the thunk object >> d = f(2, thunk) # evaluates thunk in f's scope >> e = g(3, `thunk`) # passes the un-evaluated thunk object to g There's really no way to cleanly distinguish the c, d, and e cases. There are really only two possibilities: Magic evaluation everywhere (so d is passing in eval(thunk) to f), or explicit evaluation (whether via eval or otherwise) only (so c is just binding the unevaluated thunk). I think either of those is potentially viable, but I don't think there's anything in between. Note that you can always quote the implicit evaluation (if you go the first way) or explicitly eval anywhere you want (the second), so neither one really limits what you can do. > When exactly does implicit evaluation of a thunk object occur? > Does `b[1]` give you an unevaluated thunk object? What if b is > a custom sequence type implemented in Python -- how does its > __getitem__ method avoid evaluating the thunk object prematurely? You're creating a new thunk/quoting a new expression inside the backticks, so the issue doesn't arise. You're creating a new code object that looks up b and indexes it. The __getitem__ call doesn't happen until that new code object is evaluated. > None of these problems occur in Algol, because its thunks are > not first-class values (you can't store them in arrays, etc.) > and it uses static type information to tell when to create > and evaluate them. From abarnert at yahoo.com Wed Mar 5 02:16:38 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 4 Mar 2014 17:16:38 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com> References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> <CAOyAWghmi-eYzhcwXyge-pZO+1-UoYuuimtJXtjsnHNfdVpAFw@mail.gmail.com> Message-ID: <8B8381FA-AC52-4D5A-A01C-285B21E2B5E9@yahoo.com> On Mar 4, 2014, at 17:05, Jo?o Bernardo <jbvsmo at gmail.com> wrote: >> * assign a bill to `a` so that `a` will evaluate to the value of the name `foo` any time that `a` is evaluated, in the scope of that evaluation >> >> a = $foo >> >> * as above, but always plus one >> >> a = $foo + 1 >> >> * make `a` a bill that evaluates to the value of the name `foo` at the time that `a` is evaluated, in that scope, plus the value of `bar` **at the time and in the scope of the assignment to `a`** >> >> a = $foo + bar > > > You can do all that programmatically. No need for magic operators. > Check this module as a proof of concept: https://github.com/jbvsmo/funcbuilder > > foo = FuncBuilder() > a = foo + 1 > # Then evaluate a with a function call > > For it to do exactly what you want, you just need override the evaluation function (Funcbuilder.__call__) to load names from current frame locals or something. > BTW, I don't recommend using that on real code because it is extremely experimental (unless you're a hipster... In this case, carry on). I posted something similar in the first of the many improving-lambda threads this month, but it had a lot of little problems and one big one. The big one is that not every expression can be caught by operator overloading (foo[i] is fine, but lst[foo] isn't). And call expressions are one of the most important kinds of expression, but you can't catch that--or, rather, if you _do_ catch it, then you have no way to call the resulting object. The same idea in C++ doesn't have that last problem because C++ is statically typed, and you can use an implicit cast from autolambda object to function object to get from an object whose call operator builds a call expression to one whose call operator evaluates the expression. But that doesn't work in Python. I wrote a blog post that gets into this further, but I can't find the link from my phone. I included it in an earlier message in the thread. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/07f6bb2f/attachment-0001.html> From mertz at gnosis.cx Wed Mar 5 02:23:13 2014 From: mertz at gnosis.cx (David Mertz) Date: Tue, 4 Mar 2014 17:23:13 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305000610.GA3326@cskk.homeip.net> References: <CAO41-mN=50Fi8=eLz7cxCzCLfaVx5Cg+A26VAmjqW9s=oD1v_g@mail.gmail.com> <20140305000610.GA3326@cskk.homeip.net> Message-ID: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote: > On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote: > > Only problem is that it looks a tad perlish...Of shellish. > > But worse, does nothing even remotely loke what that does in perl > or shell (or basic or...). > > I was going to remark that in my mind I would always see: > > a = $foo + 1 > > and (1) mentally bind the "$" to "foo" alone, not treat it as a > prefix to the whole expression and (2) expect "a" to evaluate right > now, regardess. > I think I could be +0 for a bit different spelling that *is* actually shell-ish. I.e. as a way to handle snippets, this doesn't seem so bad: foo = 1 a = $(foo + 1) b = eval(a) * 6 c = $(foo + a/2) print(c, eval(c)) # <thunk object at NNNN> 2 print(a + 1) # TypeError, incompatible types for '+' operation, thunk and int That is, the whole "thunk" or really just "code-snippet" could be a shorthand way to refer to a segment of (dynamically scoped, basically) code we might use in different places. It's a lot like Ruby blocks, of course, but the difference is that it's not only in the final position of a call, but anywhere anything might be used or passed. On the other hand, this isn't *really* different from just putting snippets in strings like we can do now. The one difference I envision is that maybe these "thunks" would be completely resolved by an eval(), unlike a string. That is, 'c' is a thunk, but it's a thunk that contains another thunk. So the eval() function becomes a recursive-unthunk. Well, maybe I'm -0 rather than +0. I'm definitely -1 on the "thunks taint all expressions" approach suggested initially, which seems far too magical and implicit. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/a8d49d75/attachment.html> From cs at zip.com.au Wed Mar 5 02:46:11 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 5 Mar 2014 12:46:11 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> Message-ID: <20140305014611.GA4396@cskk.homeip.net> On 04Mar2014 17:23, David Mertz <mertz at gnosis.cx> wrote: > On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote: > > On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote: > > > Only problem is that it looks a tad perlish...Of shellish. > > > > But worse, does nothing even remotely loke what that does in perl > > or shell (or basic or...). [...] > I think I could be +0 for a bit different spelling that *is* actually > shell-ish. I.e. as a way to handle snippets, this doesn't seem so bad: > > foo = 1 > a = $(foo + 1) Definitely nicer. Still irrationally uncomfortable about the "$" though. A thought, though it could break existing code (and nested tuples, alas): a = (( foo + 1 )) Still, what does this mean? a = 3 + (( foo + 1 )) I think that would need to be a syntax error, because I can't see it being anything except nonsense otherwise. That makes me think the (()) or $() is in the wrong place. Maybe: a := foo + 1 Hmm. hasn't that one already come up? > b = eval(a) * 6 This makes me unhappy. Eval parses a string and runs it, currently. I would _want_ to read that as "compute a, then eval() the result". I would prefer the occurence of "a" in an expression to cause it to be evaluated then, much as happens already. It does mean that there would be "silent" evaluation bombs lurking in expressions. For example, given a doubly linked list (to make the "reference" stuff glaring): node.right = node2 node2.left = node If node or node2 were thunks, this would be surprising to the reader. Of course, we can surprise the reader already with __getattr__ etc. But I think it would be better if: b = a just made "b" a reference to the same thunk as "a". Since I'm cavilling about "eval", how about just treating thunks like other functions (which is how we spell code to be run later presently) and have them be callables. So your "b = eval(a) * 6" would just be: b = a() * 6 That way it is glaringly obvious that "a" happens now, rather than earlier. Cheers, -- Cameron Simpson <cs at zip.com.au> Disease and deprivation stalk the land like two giant.... stalking things. - Rowan Atkinson, _Black Adder the Third_ From mertz at gnosis.cx Wed Mar 5 03:36:40 2014 From: mertz at gnosis.cx (David Mertz) Date: Tue, 4 Mar 2014 18:36:40 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305014611.GA4396@cskk.homeip.net> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> <20140305014611.GA4396@cskk.homeip.net> Message-ID: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote: > > foo = 1 > > a = $(foo + 1) > Definitely nicer. Still irrationally uncomfortable about the "$" though. > A thought, though it could break existing code (and nested tuples, alas): > > a = (( foo + 1 )) > That looks like unresolvable ambiguity to me. I confess that I am more comfortable with '$(...)' because I'm one of those folks who actually likes bash, and uses that spelling often over there (where the meaning isn't the *same* as this, but is enough similar for the meaning to carry over) > Still, what does this mean? > a = 3 + (( foo + 1 )) > I think that would need to be a syntax error, because I can't see it being > anything except nonsense otherwise. > In my example I made it a TypeError on the grounds that a thunk and and int aren't things that can be added together. That makes more sense to me than a SyntaxError, since a thunk is a perfectly good expression (or would be if the feature is added). Besides, you really need to allow thunks in expressions, since sometimes they make sense as values to operate on, e.g. (as I wrote before): print(foo, 'foo', $(foo)) # Using my bash-like syntax # 1 foo <thunk object ...> You could also, for example, have a type of object that knew how to operate on a thunk with operators: class Thunker(object): def __radd__(self, other): if isinstance(other, thunk): ... return something sensible ... thunker = Thunker() a = thunker + $(foo + bar) > b = eval(a) * 6 > This makes me unhappy. Eval parses a string and runs it, currently. > I would _want_ to read that as "compute a, then eval() the result". > b = a() * 6 I don't think I would want a thunk to be *exactly* a callable. That feels wrong to me. But I can see that overloading the meaning of eval() to operate on either a string or a thunk might feel odd. *Except* that eval() is *already* overloaded in a similar manner: >>> exp = compile("1+2", "<string>", "eval") >>> exp, eval(exp) (<code object <module> at 0x100730ae0, file "<string>", line 1>, 3) However, what if it was spelled differently from eval(), e.g.: b = unthunk(a) * 6 OK, I don't actually like that name, but you see what I mean that other names are perfectly possible. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/46581697/attachment.html> From harrismh777 at gmail.com Wed Mar 5 03:36:17 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Tue, 4 Mar 2014 18:36:17 -0800 (PST) Subject: [Python-ideas] Python3.3 Decimal Library Released In-Reply-To: <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com> References: <CAFq9Lp=QPOO53=Z_oCEwUj+gLPnutfMEmFNN9FfhtLLX1GbDhQ@mail.gmail.com> <53591b02-4790-4746-b6a9-10b43a864eaf@googlegroups.com> <CACac1F8FYemK6+pVrcw_zsR4AFAimaZS-r7FC-VwygA0A76SiQ@mail.gmail.com> Message-ID: <9858a1ce-2062-4460-9b8e-b13ee9f1c388@googlegroups.com> On Tuesday, March 4, 2014 2:01:44 AM UTC-6, Paul Moore wrote: > > Regardless of anything else that comes out of this thread, thanks for > doing that. > Paul > > Thank you, you are most welcome. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/196a4a13/attachment-0001.html> From cs at zip.com.au Wed Mar 5 04:49:45 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 5 Mar 2014 14:49:45 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> Message-ID: <20140305034945.GA67087@cskk.homeip.net> On 04Mar2014 18:36, David Mertz <mertz at gnosis.cx> wrote: > On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote: > > > foo = 1 > > > a = $(foo + 1) > > Definitely nicer. Still irrationally uncomfortable about the "$" though. > > A thought, though it could break existing code (and nested tuples, alas): > > > > a = (( foo + 1 )) > > That looks like unresolvable ambiguity to me. Me too. > I confess that I am more > comfortable with '$(...)' because I'm one of those folks who actually likes > bash, and uses that spelling often over there (where the meaning isn't the > *same* as this, but is enough similar for the meaning to carry over) I write a lot of shell scripts too. > > Still, what does this mean? > > a = 3 + (( foo + 1 )) > > I think that would need to be a syntax error, because I can't see it being > > anything except nonsense otherwise. > > In my example I made it a TypeError on the grounds that a thunk and and int > aren't things that can be added together. That makes more sense to me than > a SyntaxError, since a thunk is a perfectly good expression (or would be if > the feature is added). > > Besides, you really need to allow thunks in expressions, since sometimes > they make sense as values to operate on, e.g. (as I wrote before): > > print(foo, 'foo', $(foo)) # Using my bash-like syntax > # 1 foo <thunk object ...> > > You could also, for example, have a type of object that knew how to operate > on a thunk with operators: > > class Thunker(object): > def __radd__(self, other): > if isinstance(other, thunk): > ... return something sensible ... > > thunker = Thunker() > a = thunker + $(foo + bar) Hmm, yes. Ok. We should get TypeErrors for free except where some operator has been designed. > > b = eval(a) * 6 > > This makes me unhappy. Eval parses a string and runs it, currently. > > I would _want_ to read that as "compute a, then eval() the result". > > > b = a() * 6 > > I don't think I would want a thunk to be *exactly* a callable. That feels > wrong to me. Can you be more precise? It seems like exactly what's going on, semanticly. Except that there's no notion of parameters. > But I can see that overloading the meaning of eval() to > operate on either a string or a thunk might feel odd. *Except* that eval() > is *already* overloaded in a similar manner: > > >>> exp = compile("1+2", "<string>", "eval") > >>> exp, eval(exp) > (<code object <module> at 0x100730ae0, file "<string>", line 1>, 3) Hmm. The lack of params makes eval a better match then. > However, what if it was spelled differently from eval(), e.g.: > > b = unthunk(a) * 6 > > OK, I don't actually like that name, but you see what I mean that other > names are perfectly possible. How about: a.eval() Don't make a new public function, give thunks a method. Are we still intending thunks to be effectively a calling-scope closure? That seems subject to being fragile: you can define a thunk far from where it is called/evaled and therefore keeping it in sync with the user's scope is less solid. Cheers, -- Cameron Simpson <cs at zip.com.au> Steve is going for the pink ball - and for those of you who are watching in black and white, the pink is next to the green. - Snooker commentator 'Whispering' Ted Lowe From harrismh777 at gmail.com Wed Mar 5 04:42:28 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Tue, 4 Mar 2014 19:42:28 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System Message-ID: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> Greetings, The purpose of this idea is to expand a bit on the decimal default idea which I submitted previously. In this idea I want to suggest the human idea of a *python number.* The concept is very simple, yet may have far reaching implications not only for future python(s) but also for the wider adaptation of python in the greater academic and professional communities. The idea of *python number* means that there are no types, no limits, no constraints, and that all *python numbers *are dynamic. The upper level syntax is also very simple; all* python numbers *are simply human. My influence for this preference is rooted in the Rexx programming language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx programming language is dynamic, and has no types. To put it more succinctly for those of you who have not used Rexx, the only data type is a string of characters (that's it). *Rexx numbers* are simply those strings of characters that may be interpreted as a valid *Rexx number.* http://books.google.com/books?id=cNiVqFmPs8AC&pg=PA100&lpg=PA100&dq=rexx+numbers&source=bl&ots=SNv00ARBqU&sig=cbRb2pqCsZpYhIXtGUXTDxmtqkw&hl=en&sa=X&ei=OpQWU-XuPMT70gHhooG4BA&ved=0CDcQ6AEwAQ#v=onepage&q=rexx%20numbers&f=false The Python language might be changed to adopt the *python number* concept for *all math processing*, unless explicitly modified. This goes somewhat beyond using decimal floating point as a default numerical type. It means using human numeric expressions that meet human expectation for numeric processing by default. Under the covers (whatever we mean by that) processing is handled by decimal.Decimal, unless explicitly modified. What does this mean for python users in general? Well, no more worrying about types at all... no ints, no floats, no rationals, no irrationals, no fractions, and certainly no binaries. In short, if its a human number, its a *python number.* I am expecting that (just as in Rexx numbers) defining very clearly what is a *python number* will be key for wide adaptation of the concept. But there should be no surprises for users, particularly average users. Anyone with a middle school expectation of a numeric format should be able to use *python numbers *without surprises. However, for advanced users the full interface should be available (as is the context for Decimal) through coding based on knowledge and experience, yet the default context for Decimal should be based on average users in most environments and use cases. It is my belief that Python should lead the way in the twenty-first century for advanced computation for academic, professional, business, and scientific communities. There is a 40 year momentum for embedded binary floats & doubles, also numeric types generally, but it is time to move forward. The technology is ready, and the need is great. Let's do it. {shameless plug} pdeclib https://pypi.python.org/pypi/pdeclib Thank you for your consideration. Good evening. Mark H Harris marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/635e25ea/attachment.html> From python at mrabarnett.plus.com Wed Mar 5 05:06:38 2014 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 05 Mar 2014 04:06:38 +0000 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305014611.GA4396@cskk.homeip.net> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> <20140305014611.GA4396@cskk.homeip.net> Message-ID: <5316A2CE.2040509@mrabarnett.plus.com> On 2014-03-05 01:46, Cameron Simpson wrote: > On 04Mar2014 17:23, David Mertz <mertz at gnosis.cx> wrote: >> On Tue, Mar 4, 2014 at 4:06 PM, Cameron Simpson <cs at zip.com.au> wrote: >> > On 04Mar2014 17:57, Ryan Gonzalez <rymg19 at gmail.com> wrote: >> > > Only problem is that it looks a tad perlish...Of shellish. >> > >> > But worse, does nothing even remotely loke what that does in perl >> > or shell (or basic or...). > [...] >> I think I could be +0 for a bit different spelling that *is* actually >> shell-ish. I.e. as a way to handle snippets, this doesn't seem so bad: >> >> foo = 1 >> a = $(foo + 1) > > Definitely nicer. Still irrationally uncomfortable about the "$" though. > > A thought, though it could break existing code (and nested tuples, alas): > > a = (( foo + 1 )) > That's not a tuple. It's equivalent to: a = foo + 1 [snip] From cs at zip.com.au Wed Mar 5 06:15:34 2014 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 5 Mar 2014 16:15:34 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <5316A2CE.2040509@mrabarnett.plus.com> References: <5316A2CE.2040509@mrabarnett.plus.com> Message-ID: <20140305051534.GA75905@cskk.homeip.net> On 05Mar2014 04:06, MRAB <python at mrabarnett.plus.com> wrote: > On 2014-03-05 01:46, Cameron Simpson wrote: > >> a = $(foo + 1) > > > >Definitely nicer. Still irrationally uncomfortable about the "$" though. > >A thought, though it could break existing code (and nested tuples, alas): > > a = (( foo + 1 )) > > > That's not a tuple. It's equivalent to: > a = foo + 1 I know that. I should have said: though the below looks nice, in more complicated forms it fights with tuples, eg: ((1,2,3),(4,5,6)) (((1,2,3),(4,5,6))) :-( -- Cameron Simpson <cs at zip.com.au> Theoretical Physicist,N.:A physicist whose existence is postulated, to make the numbers balance but who is never actually observed in the laboratory. From ron3200 at gmail.com Wed Mar 5 06:25:42 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 04 Mar 2014 23:25:42 -0600 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> Message-ID: <lf6cg9$rpp$1@ger.gmane.org> On 03/04/2014 07:05 PM, Andrew Barnert wrote: > On Mar 4, 2014, at 14:31, Greg > Ewing<greg.ewing at canterbury.ac.nz> wrote: >>> Steven D'Aprano wrote: >>>>> What I have in my head is some vague concept that the Python >>>>> evaluation rules will somehow know when to evaluate the thunk >>>>> and when to treat it as an object, which is (as I understand it) >>>>> what happens in Algol. >>> But Algol has the benefit of static typing -- the procedure being >>> called explicitly declares whether the argument is to be passed by >>> name or value. Python has no idea about that at compile time. > This is the main reason I think it's more productive to think of this in > terms of Lisp-style quoting than Algol-style thunks. The fact that > quoting/thunking gives you a code object instead of an AST (sexpr) is > not significant here* (unless we're also considering adding macros). The > fact that it gives you a first-class value, and that we can't use the > "implicit casting" syntax that comes with static typing to magically > evaluate the object at the right time, is critical. > This is basically the same problem I described trying to implement > Boost-style auto lambdas in Python without C++-style implicit cast from > lambda to function. In a experimental language I'm writing, I use a concept I call "Context Resolution" to resolve objects to expected kinds of objects. The expected type/kind is determined in the context of how things are used together rather than by how they are defined. (That allows everything to be objects). Keywords, Names, Expressions, and CodeBlocks, etc... In python it would probably depend on AttributeError instead of the type. If an object doesn't have the needed attribute, then it could try calling a different method, possibly __resolve__. Then retry the attribute lookup again on the result. If there's no __resolve__ attribute, then the AttributeError would be raised as usual. The chained __resolve__ resolution attempts would also give a useful exception backtrace. Cheers, Ron From mertz at gnosis.cx Wed Mar 5 07:54:30 2014 From: mertz at gnosis.cx (David Mertz) Date: Tue, 4 Mar 2014 22:54:30 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305034945.GA67087@cskk.homeip.net> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <20140305034945.GA67087@cskk.homeip.net> Message-ID: <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> On Tue, Mar 4, 2014 at 7:49 PM, Cameron Simpson <cs at zip.com.au> wrote: > > > b = eval(a) * 6 > > > This makes me unhappy. Eval parses a string and runs it, currently. > > > I would _want_ to read that as "compute a, then eval() the result". > > > > > b = a() * 6 > > > > I don't think I would want a thunk to be *exactly* a callable. That feels > > wrong to me. > > Can you be more precise? It seems like exactly what's going on, semanticly. > Except that there's no notion of parameters. > As *I* am thinking of it, a "thunk" is really just like a C macro. Not even really like a Lisp macro. So it's not like a function in that it doesn't define a scope, doesn't have a call stack, etc. It's just "pretend I typed these other literal symbols here." Maybe I should stop calling the idea a thunk, and just call it a macro. However, I'm pulled in several directions here. On the one hand, if it really is essentially the same thing as a code object, I'm not convinced we actually need syntax for it. That is, what's really the point of having a = $(expr) # or `expr`, or `(expr), or c"expr" If it's simply a slightly shorter way of spelling: a = compile(expr, "<string>", "eval") One thing about a code object is that it is NOT callable. Which is to say, that it's not usable as a callback. If this other thing (thunk, macro, whatever) is callable, it's usable as a callback; albeit a callback with zero arguments, which again is of limited purpose, and not really that much shorter than lambda, e.g.: b = Button(text="click me", command=lambda: print("Clicked!")) Under the callable-thunk spelling, we might have: b = Button(text="click me", command=$(print("Clicked!"))) All of this is moving me towards the -0, or even -0.5 on my own idea (even though I like my spelling in principle, but I'm having trouble seeing why I actually need it). a.eval() > Don't make a new public function, give thunks a method. > Sure, that's a nice enough spelling, but what do we really get over code objects there? Actually, here's some short and trivial code in existing Python that seems to already get *everything* we've discussed in this thread: >>> class Thunk(object): ... def __init__(self, s): ... self.code = compile(s, "<thunk>", "eval") ... def __call__(self, **kws): ... return eval(self.code, globals(), kws) ... eval = __call__ ... >>> a = Thunk("foo + bar") >>> for foo, bar in enumerate(range(5,8)): ... a() ... 5 7 9 >>> for foo, bar in enumerate(range(5,8)): ... a.eval() ... 5 7 9 >>> a(foo=10, bar=20) 30 Other than using the '$' which I'm mixed about anyway, and saving two quote symbols, my Thunk class seems to be exactly what I had proposed (with Cameron's variations). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/8e680b50/attachment.html> From rosuav at gmail.com Wed Mar 5 07:57:10 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 17:57:10 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> Message-ID: <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com> On Wed, Mar 5, 2014 at 2:42 PM, Mark H. Harris <harrismh777 at gmail.com> wrote: > The idea of python number means that there are no types, no limits, no > constraints, and that all python numbers are dynamic. The upper level > syntax is also very simple; all python numbers are simply human. > > My influence for this preference is rooted in the Rexx programming > language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx > programming language is dynamic, and has no types. To put it more > succinctly for those of you who have not used Rexx, the only data type > is a string of characters (that's it). Rexx numbers are simply those > strings of characters that may be interpreted as a valid Rexx number. I've used REXX extensively. (It's the native scripting language of OS/2, which I first met in the early 1990s and am still using - albeit usually in a VM under Linux these days - as it's still a good thing, just a little left-behind.) There are two huge downsides to that kind of proposal. 1) Performance. Huge huge performance hit. I can take Pike to OS/2, an unoptimized build on an unsupported platform, and absolutely cream a REXX program at any sort of computational work. By specifically working with integers rather than "numbers", I can run code blazingly fast. 2) Accuracy. With REXX, all arithmetic is governed by a single setting of NUMERIC DIGITS, which specifies how many digits of accuracy you want to preserve. (Python can improve granularity with separate contexts, but the same issue will still apply - contexts just let you use different NUMERIC DIGITS settings simultaneously in one program.) Set it too high and performance suffers because the system has to calculate more than you actually care about. Set it too low and accuracy suffers. Even when you're working with integers, going past DIGITS means it goes exponential. 3) Not so huge a downside, but also worth considering: REXX doesn't have complex number support at all. You have to simulate it with two variables. There's no way for the "Python number" system to intrinsically handle complexes. So really, what you're looking at is unifying int/float/Decimal into a single data type, which is basically Decimal. I'd say merging int into that is a bad idea, but you can get most of what you want to achieve simply by encouraging the use of Decimal everywhere. Maybe in some distant future version, the Python literal 1.234 will represent a Decimal rather than a float. (And then there'd be other consequences, like what you get when you divide one integer by another.) But until then, all you can really do is encourage people to explicitly use Decimal. ChrisA From harrismh777 at gmail.com Wed Mar 5 08:34:15 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Tue, 4 Mar 2014 23:34:15 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com> Message-ID: <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com> On Wednesday, March 5, 2014 12:57:10 AM UTC-6, Chris Angelico wrote: > 1) Performance. Huge huge performance hit. I can take Pike to OS/2, an > unoptimized build on an unsupported platform, and absolutely cream a > REXX program at any sort of computational work. By specifically > working with integers rather than "numbers", I can run code blazingly > fast. > > 2) Accuracy. With REXX, all arithmetic is governed by a single setting > of NUMERIC DIGITS, which specifies how many digits of accuracy you > want to preserve. (Python can improve granularity with separate > contexts, but the same issue will still apply - contexts just let you > use different NUMERIC DIGITS settings simultaneously in one program.) > hi Chris, good to hear from another Rexx programmer! Back in the day (during my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370 systems, and OS/2. I too still have OS/2 running. whoohoo! Thanks for your comments. I would normally have agreed with you on this but not in this case. I do not think you are wrong, I just think you may have underestimated the power of decimal.Decimal /its phenomenal--really--! Don't take the Rexx reference too much to heart, I just wanted folks to know where I got my influence on this thinking. Rexx was too slow... so was Decimal until 3.3, and then, boy has it taken off so-to-speak! Implementation details are way off in the future of course, but I am thinking AI processing on this idea. The Python system will sense and optimize the decimal contexts (local and otherwise) so that things are balanced between speed and accuracy. And I would note, that speed is not as important for most python related aspects as user clarity and modern policy and account- ability. But, I'm not the least concerned for performance being a problem, because I have thoroughly tested decimal.Decimal and it flat screams, unlike our old friend Rexx. Thanks again for your feedback on the idea. Please give it some more thought, and let me know what you think of AI for load/speed/accuracy balancing. PS Have you considered the way things might have been if Palmisano had played ball with Bill Gates and Steve Ballmer back in the day... OS/2 would rule, gnu/linux might never have been invented period, and you and I might not be having this conversation today! ha! Go Big Blue. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140304/f2801e41/attachment-0001.html> From abarnert at yahoo.com Wed Mar 5 09:10:18 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 00:10:18 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> Message-ID: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> My previous reply got google-grouped, so let me try again. From: Mark H. Harris <harrismh777 at gmail.com> Sent: Tuesday, March 4, 2014 7:42 PM >? ?The Python language might be changed to adopt the python number >concept for all math?processing, unless explicitly modified. This goes? >somewhat beyond using decimal floating point as a default numerical? >type. ?It means using human numeric expressions that meet human? >expectation for numeric processing by default. Different humans have different expectations in different situations. Trying to pick a single type that can handle all such expectations is an impossible dream.? >? ?Under the covers (whatever we mean by that) processing is handled >by decimal.Decimal, unless explicitly modified. What does this mean for >python users in general? ?Well, no more worrying about types at all... no ints, >no floats, no rationals, no irrationals, no fractions, and certainly no binaries. >In short, if its a human number, its a python number. This is not really a meaningful concept. The types you dislike are not programming concepts that get in the way of human mathematics, they are human mathematical concepts. The integers, the rationals, and the reals all behave differently. And they're not the only types of numbers?Python handles complex numbers natively, and it's very easy to extend it with, say, quaternions or even arbitrary matrices that act like numbers. The types that _are_ programming concepts?decimal and binary floats?are necessary, because you simply can't store real numbers in finite storage. And the very fact that they are inexact approximations means you can't ignore the types. For some uses, IEEE binary floats are best; for others, decimal floats are best; for others, fractions are best; for others, you even want to handle symbolic numbers like pi/2 exactly. >? ?I am expecting that (just as in Rexx numbers) defining very clearly what >is a python number? will be key for wide adaptation of the concept. But there >should be no surprises for users, particularly average users. Anyone with? >a middle school expectation of a numeric format should be able to use? >python numbers without surprises. Anyone with a middle school expectation will expect 1/3 to be a fraction?or, at least, something they can multiply by 3 to get exactly 1. Using an?inexact decimal float instead of an inexact binary float is no improvement at all. Sure, it's an improvement in some _other_ cases, like 0.123, but if you want to deal with 1/3, today you can do so explicitly by using, say, Fraction(1, 3), while in your world it will no longer be possible. And if you take the obvious way around that, you run into the same problem with 2 ** 0.5. Normal humans who write that would expect to be able to square it and get back 2, not an approximation to 2. >However, for advanced users the full >interface should be available (as is the context for Decimal) through coding >based on knowledge and experience, yet the default context for Decimal? >should be based on average users in most environments and use cases. Is there a problem with the current default context? I think just using Decimal as it is in Python today gets you everything you're asking for. Sure, you might want a nicer way to type them and repr them, which goes back to the previous thread, but why do you need to get rid of all of the other types as well? From masklinn at masklinn.net Wed Mar 5 10:18:56 2014 From: masklinn at masklinn.net (Masklinn) Date: Wed, 5 Mar 2014 10:18:56 +0100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <5316544D.7050302@canterbury.ac.nz> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> Message-ID: <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Steven D'Aprano wrote: >> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol. > > But Algol has the benefit of static typing -- the procedure > being called explicitly declares whether the argument is to > be passed by name or value. Python has no idea about that at > compile time. That's not really a blocker though, Haskell thunks are implicit and not type-encoded. A name may correspond to a (unforced) thunk or to a strict value (an already forced thunk, whether through a previous implicit forcing, through an explicit forcing ? a strict annotation ? or through a decision of the strictness analyser). >> b = [0, `1 + thunk`] # delays evaluation and creates a thunk object >> # equivalent to `1 + some_expression` >> c = b[1] # now evaluates the thunk object >> d = f(2, thunk) # evaluates thunk in f's scope >> e = g(3, `thunk`) # passes the un-evaluated thunk object to g > > When exactly does implicit evaluation of a thunk object occur? > Does `b[1]` give you an unevaluated thunk object? What if b is > a custom sequence type implemented in Python -- how does its > __getitem__ method avoid evaluating the thunk object prematurely? > > None of these problems occur in Algol, because its thunks are > not first-class values (you can't store them in arrays, etc.) > and it uses static type information to tell when to create > and evaluate them. There are definitely difficulties in deciding how the decision to force a thunk comes about. From rosuav at gmail.com Wed Mar 5 10:36:42 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 20:36:42 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <CAPTjJmpgwFs9EYVS31qVRZJ5dDMkwkkBwci6zxUGtCTTDNnweg@mail.gmail.com> <95fffdcc-b375-4811-a102-3ca4bee6efcb@googlegroups.com> Message-ID: <CAPTjJmrV_b1Ez33sKONrso3N53psNd7=MjKsPMsu4vMSxQS49w@mail.gmail.com> On Wed, Mar 5, 2014 at 6:34 PM, Mark H. Harris <harrismh777 at gmail.com> wrote: > hi Chris, good to hear from another Rexx programmer! Back in the day > (during > my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370 > systems, and OS/2. I too still have OS/2 running. whoohoo! > > Don't take the Rexx reference too much to heart, I just wanted folks to know > where I got my influence on this thinking. Rexx was too slow... so was > Decimal > until 3.3, and then, boy has it taken off so-to-speak! Yes, that's true. But I've just done some performance testing. REXX on our OS/2 VM accounting server "Stanley" calculates 10000! in 14 seconds; Pike on the same hardware calculates 100000! in 4 seconds. (I didn't bother calculating 100000! in REXX, didn't feel like waiting.) In each case it was by this naive algorithm: REXX: n=1; do i=1 to 10000; n=n*i; end Pike: int n=1; for (int i=1;i<=100000;++i) n*=i; Those were running on the same hardware, and going to a target an order of magnitude higher took a fraction of the time. That's what integer performance is like. (BTW, I set 'numeric digits' to something just a bit more than the target. REXX was pretty fast if I left digits low, and I could then look at the exponent to see what digits setting I needed. In Pike's case, the int type supports arbitrary precision anyway, so the net result is a fair comparison; I could write the digits out to a file and get the exact same result.) >From here on, I've switched computers to Sikorsky, who has more languages and more power than Stanley has. Timings here aren't technically comparable to the above ones, but they seem similar. I guess VirtualBox is doing a decent job of staying out of the way. Pike on Sikorsky: > gauge {int n=1; for (int i=1;i<=100000;++i) n*=i;}; (3) Result: 3.394805453 Okay. Now, Python. Python 3.4.0rc1+ (default:a124b981a7a3, Mar 3 2014, 01:30:30) [GCC 4.7.2] on linux >>> def fac(top): t=time.time() n=1 for i in range(1,top+1): n*=i return time.time()-t >>> fac(100000) 7.814447641372681 Roughly double Pike's time. (I suspect this may be largely because Pike has an optimization for machine-word integers. The difference is far starker if the algorithm used is less naive.) Now watch the result of a switch of data type. def fac_d_maxprec(top): from decimal import Decimal, getcontext, MAX_PREC getcontext().prec=MAX_PREC t=time.time() n=Decimal("1") for i in range(1,top+1): n*=i return time.time()-t (Note that the multiplication is still with ints. Doing all the multiplication with Decimals would mean a whole lot more calling of the constructor, which wouldn't be a fair comparison. If all of Python used Decimal, then range() would be returning Decimal.) >>> fac_d_maxprec(100000) 20.71300506591797 I also wrote an alternative version of the above which pre-calculates at low precision, then sets the precision to just enough, and recalculates. The timings were statistically identical to the above. So there you are: A three to one difference. That's pretty amazing, actually - the fact that it's not *far far* worse is a tribute to the folks who gave us this highly optimized decimal.Decimal implementation! But it's still dramatically slower than integer calculations, and as you can see from the Pike version, the integer figures can be cut a long way too. I'm not sure that I want to give that up. > But, I'm not the least concerned for performance being a problem, > because I have thoroughly tested decimal.Decimal and it flat screams, unlike > our old friend Rexx. Yes, that's true (although frankly, when I first met REXX on OS/2, it screamed compared to doing the job myself in C - sure, I could do integer calculations quicker in C, but if I wanted to go above the size of a long - 1<<32 - that was pretty hard), but the screaming isn't so impressive compared to a modern integer engine. > PS Have you considered the way things might have been if Palmisano > had played ball with Bill Gates and Steve Ballmer back in the day... OS/2 > would rule, gnu/linux might never have been invented period, and you and > I might not be having this conversation today! ha! Go Big Blue. Hmm, I don't think so. Linux had a rather different purpose, back then. But yeah, OS/2 was brilliant tech built in a hostile environment... I'd like to see some of its best parts lifted onto a Linux kernel, though (like the WorkPlace Shell - should be possible to run that on top of X11). Alas, most of OS/2 now is in the "was great in the 90s, but other things do the same job now" basket, and a closed-source system that boasts little above a GNU/Linux stack isn't really going to go anywhere much. Still, it's hanging around. It's the best way I have for running 16-bit Windows programs, believe it or not. (And yes, I did try running Pastel Accounting under Wine. Win-OS/2 still beats Wine hands-down, probably because Wine developers don't see any reason to bother with supporting anything so ancient.) ChrisA From shai at platonix.com Wed Mar 5 11:16:12 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 12:16:12 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight Message-ID: <201403051216.12392.shai@platonix.com> Hi all, This is my first post here, following a recommendation from Alexander Belopolsky to use this list, to try to convince the Python developers to reopen a ticket. I am a long-time Python user, and a Django committer. http://bugs.python.org/issue13936 is a complaint about the fact that midnight -- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was closed as invalid, under the claim that """It is odd, but really no odder than "zero values" of other types evaluating to false in Boolean contexts""". I would like to ask for this to be reconsidered; since the ticket was closed, two main arguments were given for this: 1) The practical argument (paraphrasing Danilo Bergen and Andreas Pelme): The current semantics is surprising and useless; users do not expect valid times to be falsey, and do not normally want code to have special behavior on midnight. Users who ask for Boolean evaluation of a variable that's supposed to hold a time value, usually write "if var" as shorthand for "if var is not None". 2) The principled argument (which I think is at the root of the practical argument); quoting myself from the ticket: """Midnight is not a "zero value", it is just a value. It does not have any special qualities analogous to those of 0, "", or the empty set. ... Midnight evaluting to false makes as much sense as date(1,1,1) -- the minimal valid date value -- evaluating to false""". Thanks, Shai. From p.f.moore at gmail.com Wed Mar 5 11:23:31 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 10:23:31 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051216.12392.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> Message-ID: <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> On 5 March 2014 10:16, Shai Berger <shai at platonix.com> wrote: > http://bugs.python.org/issue13936 is a complaint about the fact that midnight > -- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was > closed as invalid, under the claim that """It is odd, but really no odder than > "zero values" of other types evaluating to false in Boolean contexts""". > > I would like to ask for this to be reconsidered; since the ticket was closed, > two main arguments were given for this: Why on earth would anyone check the boolean value of a time??? Before looking at whether midnight should be false or true, I'd like to be clear on why any real life code would care one way or the other. It seems to me that the correct answer would be to change the code to simply not rely on the boolean value of a time. Paul From masklinn at masklinn.net Wed Mar 5 11:31:08 2014 From: masklinn at masklinn.net (Masklinn) Date: Wed, 5 Mar 2014 11:31:08 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> Message-ID: <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> On 2014-03-05, at 11:23 , Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 10:16, Shai Berger <shai at platonix.com> wrote: >> http://bugs.python.org/issue13936 is a complaint about the fact that midnight >> -- datetime.time(0,0,0) -- is evaluated as False in Boolean contexts. It was >> closed as invalid, under the claim that """It is odd, but really no odder than >> "zero values" of other types evaluating to false in Boolean contexts""". >> >> I would like to ask for this to be reconsidered; since the ticket was closed, >> two main arguments were given for this: > > Why on earth would anyone check the boolean value of a time??? Side-effect of e.g. an object with an optional (nullable) time field, the developer didn't realise a time could be false-ish (because seriously?) and then it turns out code along the lines of if event.start_time: # stuff unexpectedly fails if start_time is midnight. I've actually done something close recently, didn't expect more_itertool's peekable iterator could be false-ish[0] because I'd expect empty sequences to be falsy but not empty iterators. It turned out to be useful and to simplify my code, but I can see developers not considering such edge cases unless there's a clear warning (as in ElementTree which IIRC clearly warns that an element with no children is false-ish) especially for objects for which false-ish-ness makes so little sense as a time. [0] even though the behavior is documented. From p.f.moore at gmail.com Wed Mar 5 11:44:07 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 10:44:07 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> Message-ID: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> On 5 March 2014 10:31, Masklinn <masklinn at masklinn.net> wrote: > Side-effect of e.g. an object with an optional (nullable) time field, > the developer didn't realise a time could be false-ish (because > seriously?) and then it turns out code along the lines of > > if event.start_time: > # stuff > > unexpectedly fails if start_time is midnight. Yeah, I was classing that as "application bug" and it's easy to fix with an "is not None". Agreed that the odd behaviour of time in a boolean context is why this whole class of bugs exists, but it's only a subclass if the wider problem that people shouldn't truth-test values without thinking - an explicit test is always better (explicit is better than implicit and all that). I actually find the idea of truth-testing a value that's expected to have a type that is always true *more* unintuitive than an explicit "is not None", so I'd fix the above code regardless of what the truth value of midnight is. Paul From shai at platonix.com Wed Mar 5 11:59:40 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 12:59:40 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> Message-ID: <201403051259.40289.shai@platonix.com> On Wednesday 05 March 2014 12:44:07 Paul Moore wrote: > > I actually find the idea of truth-testing a value that's expected to > have a type that is always true *more* unintuitive than an explicit > "is not None", so I'd fix the above code regardless of what the truth > value of midnight is. > I can appreciate this point of view, but I think it is not the position generally taken by Python (if it were, Boolean evaluation of an object of a type that is always true would have raised an exception). Shai. From steve at pearwood.info Wed Mar 5 12:18:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 22:18:23 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> References: <CAP-uhDcjReczx8b1M95UD4_7tKBnJmeayrLmojN7oCmTsXgpng@mail.gmail.com> Message-ID: <20140305111823.GU28804@ando> On Tue, Mar 04, 2014 at 11:09:02PM +0000, Carl Smith wrote: > The tentatively proposed idea here is using dollar signed expressions to > define 'bills'. A bill object is essentially an expression which can be > evaluated any number of times, potentially in different scopes. Please don't invent "cute" names for things which already have names. What you are describing could be considered a thunk, or a macro, or a lazily-evaluated expression, depending on the precise details of how it works. But calling it a "bill" just because it starts with a $ just makes me cringe. > The following expression [a bill literal] would be pointless, but would > define a bill that always evaluates to 1. > > $1 > > So, eval($1)==1. My personal feeling here is that if you have to explicitly call eval on the "bill" to evaluate it, it's not worth doing. If we were happy with that, we've already got compile(), or we have functions. Or just eval a string directly. My feeling is that evaluating the expression needs to be implicit, performed at need rather than up front, rather in the same way that short-circuiting operators don't actually evaluate the expression unless needed: x or expr # expr is only evaluated if x is a falsey value Yes, I know, the Zen of Python says that "explicit is better than implicit", but the Zen is intended as guidelines, not thought-blockers. We have perfectly good explicit idioms for delaying computations (eval and functions), to make the thunk/macro/whatever worth doing it has to offer something different. > Some better examples... > > * assign a bill to `a` so that `a` will evaluate to the value of the name > `foo` any time that `a` is evaluated, in the scope of that evaluation > > a = $foo > > * as above, but always plus one > > a = $foo + 1 > > * make `a` a bill that evaluates to the value of the name `foo` at the time > that `a` is evaluated, in that scope, plus the value of `bar` **at the time > and in the scope of the assignment to `a`** > > a = $foo + bar Hmmm. That implies that if you want an entire expression to have delayed evaluation, you have to tag everything with a sigil: a = $spam + $eggs - $spam*$eggs + $cheese*$spam I think it is better to have syntax with delimiters, to turn delayed evaluation ON and OFF, rather than having to tag each and every sub-expression: a = `spam + eggs - spam*eggs + cheese*spam` (disclaimer: using ` ` is just for the sake of illustration.) > Note. Similarly to mixing floats with ints, any expression that contains a > bill evaluates to a bill, so if `a` is a bill, `b=a+1` makes `b` a bill > too. Passing a bill to eval should be the obvious way to get the value. > > The point? It allows functions to accept bills to use internally. The > function would specify any names the bill can reference in the function's > API, like keywords. Well, this contradicts your previous point. If any expression that contains a "bill" is a "bill", then so is func(bill). So given that, your example here: > def f(b): # the bill arg `b` can reference `item` > for item in something: > if eval(b): return True > > f($item < 0) would actually need to be written as: eval(f($item < 0)) -- Steven From steve at pearwood.info Wed Mar 5 12:25:32 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 22:25:32 +1100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> Message-ID: <20140305112532.GV28804@ando> On Tue, Mar 04, 2014 at 05:05:39PM -0800, Andrew Barnert wrote: > On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > > > Steven D'Aprano wrote: > >> What I have in my head is some vague concept that the Python evaluation rules will somehow know when to evaluate the thunk and when to treat it as an object, which is (as I understand it) what happens in Algol. > > > > But Algol has the benefit of static typing -- the procedure > > being called explicitly declares whether the argument is to > > be passed by name or value. Python has no idea about that at > > compile time. I'm not convinced that this really matters, but for the sake of the argument let's say it does. > This is the main reason I think it's more productive to think of this > in terms of Lisp-style quoting than Algol-style thunks. Can you give more detail please? -- Steven From oscar.j.benjamin at gmail.com Wed Mar 5 12:29:10 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Mar 2014 11:29:10 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> On 5 March 2014 08:10, Andrew Barnert <abarnert at yahoo.com> wrote: > From: Mark H. Harris <harrismh777 at gmail.com> >> >> I am expecting that (just as in Rexx numbers) defining very clearly what >>is a python number will be key for wide adaptation of the concept. But there >>should be no surprises for users, particularly average users. Anyone with >>a middle school expectation of a numeric format should be able to use >>python numbers without surprises. > > Anyone with a middle school expectation will expect 1/3 to be a fraction--or, at least, something they can multiply by 3 to get exactly 1. Using an inexact decimal float instead of an inexact binary float is no improvement at all. I actually think that it is an improvement. Most people are surprised by the fact that just writing x = 0.1 causes a rounding error. Python only has dedicated syntax for specifying binary floats in terms of decimal digits meaning that there is no syntax for exactly specifying non integer numbers. I would say that that is clearly a sub-optimal situation. I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d, and I would also use Fraction literals if available e.g. 1/3F. Oscar From steve at pearwood.info Wed Mar 5 12:32:33 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 22:32:33 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305014611.GA4396@cskk.homeip.net> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> <20140305014611.GA4396@cskk.homeip.net> Message-ID: <20140305113233.GW28804@ando> On Wed, Mar 05, 2014 at 12:46:11PM +1100, Cameron Simpson wrote: > That makes me think the (()) or $() is in the wrong place. Maybe: > > a := foo + 1 No good. That implies that there is no way to have an anonymous thunk. You can do this: my_list = [x + 1, y + 2, # eager evaluation $(z + 3), # lazy evaluation ... ] but would have to write: temp := z + 3 my_list = [x + 1, y + 2, temp, ...] Similarly for passing a thunk into a function. > Since I'm cavilling about "eval", how about just treating thunks > like other functions (which is how we spell code to be run later > presently) and have them be callables. In which case, what's the point? What makes them different from regular functions? -- Steven From steve at pearwood.info Wed Mar 5 12:43:44 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 22:43:44 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <20140305034945.GA67087@cskk.homeip.net> <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> Message-ID: <20140305114344.GX28804@ando> On Tue, Mar 04, 2014 at 10:54:30PM -0800, David Mertz wrote: > As *I* am thinking of it, a "thunk" is really just like a C macro. Not > even really like a Lisp macro. So it's not like a function in that it > doesn't define a scope, doesn't have a call stack, etc. It's just "pretend > I typed these other literal symbols here." Maybe I should stop calling the > idea a thunk, and just call it a macro. I'm intrigued by this suggestion. I think that what I'm really after is two things: lazy evaluation, and dynamic scoping. I think a macro would give us both. But hasn't Guido said No Macros at some point? > However, I'm pulled in several directions here. On the one hand, if it > really is essentially the same thing as a code object, I'm not convinced we > actually need syntax for it. That is, what's really the point of having > > a = $(expr) # or `expr`, or `(expr), or c"expr" > > If it's simply a slightly shorter way of spelling: > > a = compile(expr, "<string>", "eval") You can't write it like that. You have to wrap the expression is quotes and turn it into a string: a = compile("expr", "<string>", "eval") which means you lose syntax highlighting. Perhaps the ability to get syntax highlighting is not sufficient to justify this idea. Another disadvantage: the ever-present temptation to pass a user-generated string to compile: a = compile(template % some_string, "<string>", "eval") If some_string came from an untrusted source, you are now the proud owner of a brand new code injection vulnerability. But with syntax, you cannot generate a thunk/macro except from code you write yourself. It's still code written by you, it's just that evaluation is delayed. -- Steven From cfkaran2 at gmail.com Wed Mar 5 12:53:10 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Wed, 5 Mar 2014 06:53:10 -0500 Subject: [Python-ideas] Suggestion for standardized annotations Message-ID: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> Hi all, I wanted to make a suggestion for a convention on how to write annotations. I know that PEP 8 says that the standard library won't use them in order to avoid setting a convention, but that could lead to confusion and conflicts in how they are used in non-standard libraries. The reason I'm thinking about this is because of how docstrings are currently used. If you use PLY (http://www.dabeaz.com/ply/) and also want to use something like Sphinx (http://sphinx-doc.org/), you're going to have a problem; PLY stores its rules in the docstring, while Sphinx parses it for documentation. I want to prevent this problem for annotations. My thought is that all annotations should be dictionaries. The keys should all be unicode strings that are uppercase UUIDs (basically, what 'uuid.uuid1().hex.upper()' returns), and the values can be anything the programmer wants. Each project can generate one (or more) UUIDs to put into the dictionary, and publicly document what the UUID's meaning is (preferably somewhere where a search engine can find it). The advantage is that since UUIDs are unique, the number of false positives you'll get while searching for it should be low; I've tested this on a different mailing list I'm on, and the UUID I generated for it has 0 false positives, while picking up the complete discussion involving it. As an example, if project A had chosen the UUID B3D2AFE8A45A11E3AE24D49A20C52EF2 and project B chose the UUID C02D7C64A45A11E39DAFD49A20C52EF2, we might annotate a function as follows: def foo(first : { B3D2AFE8A45A11E3AE24D49A20C52EF2 : {"doc" : "English documentation"}, C02D7C64A45A11E39DAFD49A20C52EF2 : {"doc" : "expression : MINUS expression %prec UMINUS"} }, second) -> {B3D2AFE8A45A11E3AE24D49A20C52EF2: {"type" : int, "doc" : "Typechecking for a linter"}}: pass You can already see the downside of this approach; it's really, really verbose. However, at least it avoids outright conflicts in usage that prevent usage of certain tools/projects together. Note that I did consider using names as keys directly (e.g. 'doc'). However, that requires a strong, universal convention on what each key means. Since we can't seem to figure that out for the docstring, I don't see why we should expect to be able to figure it out for any of proposed keys. Moreover, the set of keys would need to be documented somewhere, the documentation kept up to date, etc. It becomes a management nightmare. UUIDs have the advantage the we just tell everyone how to generate their own, and let them go at it. If someone wants to use a given project's docstring for their own purposes, it is up to them to keep the meaning the same. Thoughts/suggestions? Thanks, Cem Karan From steve at pearwood.info Wed Mar 5 13:04:10 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 5 Mar 2014 23:04:10 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> Message-ID: <20140305120410.GY28804@ando> On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote: > I don't agree with Mark's proposal in this thread but I would like to > have decimal literals e.g. 1.23d, +1 on that. Although that will depend on how big a slowdown it causes to Python's startup time. > and I would also use Fraction > literals if available e.g. 1/3F. Out of curiosity, why F rather than f? (By the way, I think it is somewhat amusing that Python not only has a built-in complex type, but also *syntax* for creating complex numbers, but no built-in support for exact rationals.) -- Steven From oscar.j.benjamin at gmail.com Wed Mar 5 13:30:19 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Mar 2014 12:30:19 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305120410.GY28804@ando> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> Message-ID: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> On 5 March 2014 12:04, Steven D'Aprano <steve at pearwood.info> wrote: > On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote: > >> I don't agree with Mark's proposal in this thread but I would like to >> have decimal literals e.g. 1.23d, > > +1 on that. Although that will depend on how big a slowdown it causes to > Python's startup time. If it is a significant slowdown then it can be a delayed import that only occurs when a decimal literal is first encountered. Applications that don't need decimal won't be slowed down. Applications that do would have had to import it anyway. >> and I would also use Fraction >> literals if available e.g. 1/3F. > > Out of curiosity, why F rather than f? In C and some other languages the f suffix indicates a numeric literal that has type "float" e.g. "6.0f". You can use upper case there as well but the convention is lower case and to include the .0 when it's an integer. I just though that 3F looks sufficiently distinct from the way it's typically done in C. Oscar From rosuav at gmail.com Wed Mar 5 13:33:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 23:33:11 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305051534.GA75905@cskk.homeip.net> References: <5316A2CE.2040509@mrabarnett.plus.com> <20140305051534.GA75905@cskk.homeip.net> Message-ID: <CAPTjJmpf1XQA+PCi3=5KiT_=DE+TAVxMNqmHV_+keUxS+CLeHg@mail.gmail.com> On Wed, Mar 5, 2014 at 4:15 PM, Cameron Simpson <cs at zip.com.au> wrote: > On 05Mar2014 04:06, MRAB <python at mrabarnett.plus.com> wrote: >> On 2014-03-05 01:46, Cameron Simpson wrote: >> >> a = $(foo + 1) >> > >> >Definitely nicer. Still irrationally uncomfortable about the "$" though. >> >A thought, though it could break existing code (and nested tuples, alas): >> > a = (( foo + 1 )) >> > >> That's not a tuple. It's equivalent to: >> a = foo + 1 > > I know that. I should have said: though the below looks nice, in more complicated forms it fights with tuples, eg: > > ((1,2,3),(4,5,6)) > (((1,2,3),(4,5,6))) Bikeshedding the syntax without expressing an opinion on the feature: Using {{ }} would be safer. The inner one might be interpreted as a dict or a set, but neither of those can go into a set, so it'd be illogical. Whether the similarity with C-style block delimiters is a good thing or a bad thing remains to be seen :) ChrisA From oscar.j.benjamin at gmail.com Wed Mar 5 13:34:09 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Mar 2014 12:34:09 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051259.40289.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <201403051259.40289.shai@platonix.com> Message-ID: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> On 5 March 2014 10:59, Shai Berger <shai at platonix.com> wrote: > On Wednesday 05 March 2014 12:44:07 Paul Moore wrote: >> >> I actually find the idea of truth-testing a value that's expected to >> have a type that is always true *more* unintuitive than an explicit >> "is not None", so I'd fix the above code regardless of what the truth >> value of midnight is. > > I can appreciate this point of view, but I think it is not the position > generally taken by Python (if it were, Boolean evaluation of an object of a > type that is always true would have raised an exception). I think it's clear that if this were new code being added to the stdlib then the consensus would be that having midnight evaluate as False is ridiculous. The question is surely whether the issue is worth a backwards compatibility break not whether the current behaviour is a good idea (it clearly isn't). Oscar From carl.input at gmail.com Wed Mar 5 13:41:10 2014 From: carl.input at gmail.com (Carl Smith) Date: Wed, 5 Mar 2014 12:41:10 +0000 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality Message-ID: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> Just for the record, I never called them bills just because they contain a dollar sign! I was thinking of a short word which describes an expression who's evaluation changes depending on the local circumstances. Like a legal bill, not a dollar bill. The name macro doesn't really work, as it's only an expression, it's not a code block. --- The idea was never to start passing complex expressions around the place. It was just to allow a function to take an expression containing about one or two names, and evaluate the expression internally, so this... func(lambda foo: foo > 0) ...can become... func($foo > 0) Other uses are there, and might be helpful from time to time, but it's mainly about cleaning up APIs. I personally work with user facing Python APIs all day, and users are generally programming interactively, so perhaps my take is atypical. --- What the hell is a thunk anyway? It's a horrible name. --- On the jQuery like syntax, $(foo), that might work and might do away with the issue of expressions containing bills always evaluating to bills. I took that too far. Good point. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5fa7acf1/attachment.html> From rosuav at gmail.com Wed Mar 5 13:42:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 5 Mar 2014 23:42:14 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> Message-ID: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > I just though that 3F looks sufficiently distinct from the > way it's typically done in C. 3F would be about -16C wouldn't it? (Not painting any bike sheds when it's that cold, thanks!) I'm not sure I like the idea of tagging the end of the expression. Currently, the nearest Python has to that is the complex notation: >>> 1+2j (1+2j) And that works just fine when considered to be addition: >>> a=1 >>> b=2j >>> a+b (1+2j) So really, what Python has is a notation for a j-suffixed float, meaning a complex number with no real part. You can't do that with Fraction: >>> a=2 >>> b=3F >>> a/b What I'd prefer would be some modification of the division operator. We currently have two ways to divide an integer by an integer: >>> 1234/10 123.4 >>> 1234//10 123 Adding a third operator, defined only between two integers, would be cleaner than tagging one of the integer literals. Exactly what that operator would be is hard to say, but I'm sure there's a combination that would look right and not be ambiguous. >>> 1234-/-10 Fraction(617, 5) I can't think of anything good, though. ChrisA From p.f.moore at gmail.com Wed Mar 5 13:50:14 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 12:50:14 +0000 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> Message-ID: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote: > Thoughts/suggestions? I think the core/stdlib position is that agreeing conventions would be better done once some real world experience of the practical issues and benefits of annotations has been established. So while a proposal like this is not without merit, it needs to be considered in the light of how projects actually use annotations. Personally, I'm not aware of any libraries that make significant use of annotations, so a good first step would be to survey existing use, and summarise it here. That would allow you to clarify your proposal in terms of exactly how existing projects would need to modify their current code. Of course, there's likely a chicken and egg problem here - projects may be holding off using annotations through fear of issues caused by clashes. But I'm not sure that a UUID-based proposal like the above (which as you admit is very verbose, and not particularly user friendly) would be more likely to encourage use. If I were developing a library that would benefit from annotations, at this point in time I'd probably just choose whatever conventions suited me and go with those - likely marking the feature as "subject to change" initially. Then, when people raised bug reports or feature requests that asked for better interoperability, I'd look at how to achieve that in conjunction with the other project(s) that clashed with me. Paul From p.f.moore at gmail.com Wed Mar 5 13:54:27 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 12:54:27 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <201403051259.40289.shai@platonix.com> <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> Message-ID: <CACac1F8oCRHWSo777p6NvkXv4i4wE8C+Nj2cSFWBdhZSnNNuBw@mail.gmail.com> On 5 March 2014 12:34, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > On 5 March 2014 10:59, Shai Berger <shai at platonix.com> wrote: >> On Wednesday 05 March 2014 12:44:07 Paul Moore wrote: >>> >>> I actually find the idea of truth-testing a value that's expected to >>> have a type that is always true *more* unintuitive than an explicit >>> "is not None", so I'd fix the above code regardless of what the truth >>> value of midnight is. >> >> I can appreciate this point of view, but I think it is not the position >> generally taken by Python (if it were, Boolean evaluation of an object of a >> type that is always true would have raised an exception). > > I think it's clear that if this were new code being added to the > stdlib then the consensus would be that having midnight evaluate as > False is ridiculous. > > The question is surely whether the issue is worth a backwards > compatibility break not whether the current behaviour is a good idea > (it clearly isn't). Precisely. Sorry I wasn't clear enough. And FWIW, my opinion is that the problem is not worth a compatibility break, because it's so easily solved (and the fixed code is probably an improvement in any case). Paul From oscar.j.benjamin at gmail.com Wed Mar 5 13:56:46 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Mar 2014 12:56:46 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> Message-ID: <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com> On 5 March 2014 12:42, Chris Angelico <rosuav at gmail.com> wrote: > On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin > <oscar.j.benjamin at gmail.com> wrote: >> I just though that 3F looks sufficiently distinct from the >> way it's typically done in C. > > 3F would be about -16C wouldn't it? > > (Not painting any bike sheds when it's that cold, thanks!) > > I'm not sure I like the idea of tagging the end of the expression. > Currently, the nearest Python has to that is the complex notation: > >>>> 1+2j > (1+2j) > > And that works just fine when considered to be addition: > >>>> a=1 >>>> b=2j >>>> a+b > (1+2j) > > So really, what Python has is a notation for a j-suffixed float, > meaning a complex number with no real part. You can't do that with > Fraction: > >>>> a=2 >>>> b=3F >>>> a/b >From a syntactic perspective Python doesn't have syntax for general complex literals. There is only syntax for creating imaginary literals and it is trivial to create a complex number by adding a real and an imaginary one. 3F could create a Fraction with the integer value 3 so that a/b gives a rational number: >>> from fractions import Fraction as F >>> a = 2 >>> b = F(3) >>> a/b Fraction(2, 3) I don't understand why you say that can't be done. Oscar From kaiser.yann at gmail.com Wed Mar 5 14:09:58 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Wed, 5 Mar 2014 14:09:58 +0100 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> Message-ID: <CANUJvPXt7gbWt0btRe6hGesO2A7LC5_fnAqVkR771teLF_PZ6g@mail.gmail.com> You could use a slight modification of sigtools.modifiers.annotate [1] to create different objects with different function annotations. That way programmers who only use one library have no change to do, and those who use more only have to add a few lines devoid of the verbosity of UUIDs. [1] http://sigtools.readthedocs.org/en/latest/#sigtools.modifiers.annotate On 5 March 2014 13:50, Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote: >> Thoughts/suggestions? > > I think the core/stdlib position is that agreeing conventions would be > better done once some real world experience of the practical issues > and benefits of annotations has been established. So while a proposal > like this is not without merit, it needs to be considered in the light > of how projects actually use annotations. Personally, I'm not aware of > any libraries that make significant use of annotations, so a good > first step would be to survey existing use, and summarise it here. > That would allow you to clarify your proposal in terms of exactly how > existing projects would need to modify their current code. > > Of course, there's likely a chicken and egg problem here - projects > may be holding off using annotations through fear of issues caused by > clashes. But I'm not sure that a UUID-based proposal like the above > (which as you admit is very verbose, and not particularly user > friendly) would be more likely to encourage use. > > If I were developing a library that would benefit from annotations, at > this point in time I'd probably just choose whatever conventions > suited me and go with those - likely marking the feature as "subject > to change" initially. Then, when people raised bug reports or feature > requests that asked for better interoperability, I'd look at how to > achieve that in conjunction with the other project(s) that clashed > with me. > > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From shai at platonix.com Wed Mar 5 14:10:54 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 15:10:54 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051259.40289.shai@platonix.com> <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> Message-ID: <201403051510.54769.shai@platonix.com> On Wednesday 05 March 2014 14:34:09 Oscar Benjamin wrote: > > I think it's clear that if this were new code being added to the > stdlib then the consensus would be that having midnight evaluate as > False is ridiculous. > > The question is surely whether the issue is worth a backwards > compatibility break not whether the current behaviour is a good idea > (it clearly isn't). > If this is the case, then we should be asking a variant of Paul's question: Why would anyone check if a time is equal to midnight, by Boolean evaluation rather than comparison to midnight? Or, rather, how many people do this? We have several reports of the current behavior causing surprising, hard-to- detect bugs (a co-worker spent a couple of hours on one such bug yesterday; frankly, I'm impressed he found it that fast). Unless we have reason to think people are using this on purpose, the effect of the change will probably be to fix many more bugs than it causes. I suggest to handle the backwards-compatibility issue with a deprecation cycle: Python 3.4 is already feature frozen, so make 3.5 raise a PendingDeprecationWarning when datetime.time.__nonzero__ returns False; 3.6 a DeprecationWarning; and 3.7 return True. That's what we would do in Django. On Wednesday 05 March 2014 14:54:27 Paul Moore wrote: > > And FWIW, my opinion is that the problem is not worth a compatibility > break, because it's so easily solved (and the fixed code is probably > an improvement in any case). Ah, but it is only easily solved once it is detected; the behavior causes bugs that surface once in a blue moon (in production, and then they cannot be reproduced by developers who, usually, do not work at midnight). So, on a second thought -- an alternative solution is to make any Boolean evaluation of time objects warn the user about midnight. Thanks for your attention, Shai. From steve at pearwood.info Wed Mar 5 14:21:03 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 00:21:03 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <20140305132103.GZ28804@ando> On Wed, Mar 05, 2014 at 12:10:18AM -0800, Andrew Barnert wrote: > Anyone with a middle school expectation will expect 1/3 to be a > fraction?or, at least, something they can multiply by 3 to get exactly > 1. Not a very good example -- that happens to work for Python floats (which are C doubles under the hood): py> (1/3)*3 == 1 True py> (1/3 + 1/3 + 1/3) == 1 True But it *does not work* with Decimal, at least not with the default precision: py> from decimal import Decimal py> (Decimal(1)/3)*3 Decimal('0.9999999999999999999999999999') Decimal is not a panacea! It does not eliminate floating point issues. Between 1 and 100, there are 32 Decimal numbers that fail the test that (1/n)*n == 1, and only two floats. Between 1 and 100, there are only four floats where 1/(1/n) does not equal n: 49 93 98 and 99. In comparison, there are 46 such failing Decimals, including 6 7 and 9. -- Steven From steve at pearwood.info Wed Mar 5 14:23:33 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 00:23:33 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> Message-ID: <20140305132333.GA28804@ando> On Wed, Mar 05, 2014 at 11:42:14PM +1100, Chris Angelico wrote: > I'm not sure I like the idea of tagging the end of the expression. > Currently, the nearest Python has to that is the complex notation: > > >>> 1+2j > (1+2j) > > And that works just fine when considered to be addition: > > >>> a=1 > >>> b=2j > >>> a+b > (1+2j) > > So really, what Python has is a notation for a j-suffixed float, > meaning a complex number with no real part. You can't do that with > Fraction: > > >>> a=2 > >>> b=3F > >>> a/b Why not? If 3d is a Decimal with the value of 3, why couldn't 3F be a Fraction with the value of 3? -- Steven From p.f.moore at gmail.com Wed Mar 5 14:26:38 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 13:26:38 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051510.54769.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051259.40289.shai@platonix.com> <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> <201403051510.54769.shai@platonix.com> Message-ID: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote: > On Wednesday 05 March 2014 14:54:27 Paul Moore wrote: >> >> And FWIW, my opinion is that the problem is not worth a compatibility >> break, because it's so easily solved (and the fixed code is probably >> an improvement in any case). > > Ah, but it is only easily solved once it is detected; the behavior causes bugs > that surface once in a blue moon (in production, and then they cannot be > reproduced by developers who, usually, do not work at midnight). That's a fair point. But your deprecation suggestion doesn't seem to help this, unless you expect (in a realistic timescale) to be able to drop support for Python <3.7. Your users (on say 2.7 or 3.3) will still get bugs, and now your developers (using 3.7) won't even be able to reproduce the issue by staying up till midnight. And once they do find the bug they will still have to fix it by an explicit test. Again, I'm not saying the current behaviour is sensible, but I doubt the work to fix it will benefit anyone in practice. Paul. From jsbueno at python.org.br Wed Mar 5 14:40:45 2014 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Wed, 5 Mar 2014 10:40:45 -0300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051259.40289.shai@platonix.com> <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> <201403051510.54769.shai@platonix.com> <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> Message-ID: <CAH0mxTRVn5-RnuS9N+kYfWQnyeyyP-JdZwR2EKKraCx=yv3+uA@mail.gmail.com> I agree this change would cause far more good than harm - I see that it implies an incompatible change, but lets think on "Although practicality beats purity" terms here: the only instance I see this could possibly be used by purpose is in a code to that would increase a "day counter" if the time.now() would evaluate to false. Such a piece of code would be broken in many other levels to start with. A critical session in any piece of code that has to check if "at this second now we are at midnight" should be using one of the specialized timers to take care of borderline cases, such as start and ending of D.S.T. and such. It would have to be orders of magnitude better written than "if not current_time: ... " And even in the improbable case of such a piece of code exists, the changing of behavior would make it break in the first 24 hours, and the problem would not be that hard to find out. In this light, I think having to wait for 3 major release cycles until that is gone is a lot of overkill. On 5 March 2014 10:26, Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote: >> On Wednesday 05 March 2014 14:54:27 Paul Moore wrote: >>> >>> And FWIW, my opinion is that the problem is not worth a compatibility >>> break, because it's so easily solved (and the fixed code is probably >>> an improvement in any case). >> >> Ah, but it is only easily solved once it is detected; the behavior causes bugs >> that surface once in a blue moon (in production, and then they cannot be >> reproduced by developers who, usually, do not work at midnight). > > That's a fair point. But your deprecation suggestion doesn't seem to > help this, unless you expect (in a realistic timescale) to be able to > drop support for Python <3.7. Your users (on say 2.7 or 3.3) will > still get bugs, and now your developers (using 3.7) won't even be able > to reproduce the issue by staying up till midnight. And once they do > find the bug they will still have to fix it by an explicit test. > > Again, I'm not saying the current behaviour is sensible, but I doubt > the work to fix it will benefit anyone in practice. > > Paul. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From stefan at bytereef.org Wed Mar 5 14:39:42 2014 From: stefan at bytereef.org (Stefan Krah) Date: Wed, 5 Mar 2014 13:39:42 +0000 (UTC) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> Message-ID: <loom.20140305T143713-529@post.gmane.org> Steven D'Aprano: > > I don't agree with Mark's proposal in this thread but I would like to > > have decimal literals e.g. 1.23d, > > +1 on that. Although that will depend on how big a slowdown it causes to > Python's startup time. Startup time should not be a problem once http://bugs.python.org/issue19232 is dealt with. Stefan Krah From ncoghlan at gmail.com Wed Mar 5 14:45:33 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 5 Mar 2014 23:45:33 +1000 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> Message-ID: <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> On 5 Mar 2014 22:51, "Paul Moore" <p.f.moore at gmail.com> wrote: > > On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote: > > Thoughts/suggestions? > > I think the core/stdlib position is that agreeing conventions would be > better done once some real world experience of the practical issues > and benefits of annotations has been established. MyPy uses function annotations for optional static typing, which is pretty much the use case Guido originally had in mind and the main reason that PEP 8 doesn't make combining annotations with an associated decorator mandatory: http://www.mypy-lang.org/ You do still have to import a particular module to indicate that all annotations in the importing module are to be interpreted as type annotations. Beyond that, the guidance in PEP 8 stands: """It is recommended that third party experiments with annotations use an associated decorator to indicate how the annotation should be interpreted.""" Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/50045387/attachment.html> From steve at pearwood.info Wed Mar 5 14:56:46 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 00:56:46 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> Message-ID: <20140305135646.GB28804@ando> On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote: > The idea of *python number* means that there are no types, no limits, no > constraints, and that all *python numbers *are dynamic. The upper level > syntax is also very simple; all* python numbers *are simply human. What makes this a "python number"? In what way are they "dynamic"? > My influence for this preference is rooted in the Rexx programming > language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx > programming language is dynamic, and has no types. To put it more > succinctly for those of you who have not used Rexx, the only data type > is a string of characters (that's it). *Rexx numbers* are simply those > strings of characters that may be interpreted as a valid *Rexx number.* I haven't used Rexx, but I have used Hypertalk, which worked the same way. If you don't care about performance, it can work quite well. > The Python language might be changed to adopt the *python number* > concept for *all math processing*, unless explicitly modified. Well, there's a bit of a problem here. Numbers in Python are not just used for maths processing. They're also used for indexing into lists, as keys in dicts, for bitwise operations, for compatibility with external libraries that have to interface with other languages, as flags, etc. For some of these purposes, we *really do* want to distinguish between ints and floats that happen to have the same value: mylist[3] # allowed mylist[3.0] # not allowed Now, you might argue that this distinction is unnecessary, but it runs quite deep in Python. You'd need to change that philosophy for this idea to work. > This goes > somewhat beyond using decimal floating point as a default numerical > type. It means using human numeric expressions that meet human > expectation for numeric processing by default. I don't understand what that means, unless it means that you want Python to somehow, magically, make all the unintuitive issues with floating point to disappear. Good luck with that one. If you want that, Decimal is not the answer. It would have to be a Rational type, like Fraction, although even that doesn't support surds. Fractions have their own problems too. Compare the status quo: py> 3**0.5 1.7320508075688772 with a hypothetical version that treats all numbers as exact fractions: py> 3**0.5 Fraction(3900231685776981, 2251799813685248) Which do you think the average person using Python as a calculator would prefer to see? And another issue: before he invented Python, Guido spent a lot of time working with ABC, which used Fractions as the native number type. The experience soured him on the idea for nearly two decades. Although Guido has softened his stance enough to allow the fractions module into the standard library, I doubt he would allow Fractions to become the underlying implementation of numbers in Python. The problem is that fractions can be unbounded in memory, and some simple operations become extremely inefficient. For example, without converting to float, which is bigger? Fraction(296, 701) Fraction(355, 594) For many purposes, the fact that floats (whether binary or decimal) have finite precision and hence introduce rounding error is actually a good thing. Compare: py> 1e-300 + 1e300 1e+300 versus fractions: py> from fractions import Fraction as F py> F(10)**-300 + F(10)**300 Fraction(100000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000001, 100000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000) There are *very few* applications where such precision is needed or wanted. The performance hit in calculating such excessively precise numbers when the user doesn't need it will be painful. The same applies to Decimal, although to a lesser extent since Decimals do have a finite precision. Unlike fractions, they cannot grow without limit: py> Decimal("1e-300") + Decimal("1e300") Decimal('1.000000000000000000000000000E+300') -- Steven From shai at platonix.com Wed Mar 5 14:53:19 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 15:53:19 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051510.54769.shai@platonix.com> <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> Message-ID: <201403051553.19311.shai@platonix.com> On Wednesday 05 March 2014 15:26:38 Paul Moore wrote: > On 5 March 2014 13:10, Shai Berger <shai at platonix.com> wrote: > > On Wednesday 05 March 2014 14:54:27 Paul Moore wrote: > >> And FWIW, my opinion is that the problem is not worth a compatibility > >> break, because it's so easily solved (and the fixed code is probably > >> an improvement in any case). > > > > Ah, but it is only easily solved once it is detected; the behavior causes > > bugs that surface once in a blue moon (in production, and then they > > cannot be reproduced by developers who, usually, do not work at > > midnight). > > That's a fair point. But your deprecation suggestion doesn't seem to > help this, unless you expect (in a realistic timescale) to be able to > drop support for Python <3.7. I expect to drop support for Python<3.7 sometime... > Your users (on say 2.7 or 3.3) will > still get bugs, and now your developers (using 3.7) won't even be able > to reproduce the issue by staying up till midnight. If my developers are using 3.7 to debug problems that user encounter with 3.3, they deserve to stay up till midnight and beyond. > And once they do > find the bug they will still have to fix it by an explicit test. The process I want to see is, that Boolean evaluations of time objects is slowly, but more-or-less completely removed from code that targets Python<3.7; then, in a few years, people can go back to writing "if event.start_time" like they do with dates and datetimes. > Again, I'm not saying the current behaviour is sensible, but I doubt > the work to fix it will benefit anyone in practice. > So, you support my other suggestion -- a warning on every use of bool(time)? Also, the work to fix it is probably less than the work invested in this discussion so far... In any case, if we all agree that http://bugs.python.org/issue13936 is a valid problem -- can we re-open it, and then discuss how (or if) we solve it? Thanks, Shai. From skip at pobox.com Wed Mar 5 15:34:16 2014 From: skip at pobox.com (Skip Montanaro) Date: Wed, 5 Mar 2014 08:34:16 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> Message-ID: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> On Wed, Mar 5, 2014 at 4:44 AM, Paul Moore <p.f.moore at gmail.com> wrote: >> if event.start_time: >> # stuff >> >> unexpectedly fails if start_time is midnight. > > Yeah, I was classing that as "application bug" and it's easy to fix > with an "is not None". +1, +1, +1. This is nothing more than an application bug. Issue 13936 should not be reopened. This is a very common mistake when None is used as a sentinel value (though people normally get away with it), especially by people coming from languages having NULL pointers which shall not be named. After all, "None" and "NULL" both start with the letter "N". Surely, they must represent the same fundamental concept, right? <wink> Let's change the problem slightly. Instead of time objects, let's consider the domain of possible values to be integers. Clearly, None is not a member of that set. You could thus use it as a sentinel. So, tell me, OP. Would this usage be correct? if event.some_id: # stuff I would argue, "no." I would not argue that 0 should not compare as False, however. I would fix my code: if event.some_id is not None: # stuff In contexts where None is an element of the set of possible values (and thus can't be used as a sentinel value itself), it's common to explicitly create such a (unique) value, e.g.: SENTINEL = ["xyz"] # or [None] or {} or set([sys]) or ..., but not () or 1 or True! When you check for the sentinel you have to be explicit (and you MUST check for it using the "is" operator, to drag in another recent thread): if something is SENTINEL: mix_the_special_sauce() Using None as a sentinel value is no different, it just happens to a) already exist, and b) be unique, saving you an exceedingly small amount of typing. If you were trying to be terribly clever, there are lots of other values in Python (at least in CPython) which happen to be unique (the empty tuple, small integers, True and False, ...), and could thus theoretically be used as a sentinel. Someone would eventually slap your wrist for that sort of thing though (one would hope). Two final points. 1. If you wanted to modify the language (or the standard library) in some way, perhaps the more theoretically reasonable request would be to ask that None not be usable in a boolean context. Making that change would certainly break a lot of code though, so it's too late to do that. 2. The more I think about it, the more I think None should not be used as a sentinel. Skip From p.f.moore at gmail.com Wed Mar 5 16:01:33 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 15:01:33 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051553.19311.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051510.54769.shai@platonix.com> <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> <201403051553.19311.shai@platonix.com> Message-ID: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote: > then, in a few years, people can go back to writing "if event.start_time" like > they do with dates and datetimes. Why on earth would they do that? It's still bad practice. If you're using None as a sentinel, you should test for it explicitly. Nobody has yet suggested any other use case where this matters. >> Again, I'm not saying the current behaviour is sensible, but I doubt >> the work to fix it will benefit anyone in practice. >> > > So, you support my other suggestion -- a warning on every use of bool(time)? No. Good programming practice should cover that. We don't warn in other cases where programmers make silly coding errors. Look at Skip's message - should we also warn on uses of bool(int) because people can write bad code that fails to work properly with zero, as well? > Also, the work to fix it is probably less than the work invested in this > discussion so far... That is not obvious (given documentation, release management, etc, costs). > In any case, if we all agree that http://bugs.python.org/issue13936 is a valid > problem -- can we re-open it, and then discuss how (or if) we solve it? We don't. Can we agree that it's not a bug and abandon this fruitless discussion? Paul From kaiser.yann at gmail.com Wed Mar 5 16:08:02 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Wed, 5 Mar 2014 16:08:02 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> Message-ID: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> On 5 March 2014 15:34, Skip Montanaro <skip at pobox.com> wrote: > On Wed, Mar 5, 2014 at 4:44 AM, Paul Moore <p.f.moore at gmail.com> wrote: >>> if event.start_time: >>> # stuff >>> >>> unexpectedly fails if start_time is midnight. >> >> Yeah, I was classing that as "application bug" and it's easy to fix >> with an "is not None". > > +1, +1, +1. This is nothing more than an application bug. Issue 13936 > should not be reopened. > > This is a very common mistake when None is used as a sentinel value > (though people normally get away with it), especially by people coming > from languages having NULL pointers which shall not be named. After > all, "None" and "NULL" both start with the letter "N". Surely, they > must represent the same fundamental concept, right? <wink> > > Let's change the problem slightly. Instead of time objects, let's > consider the domain of possible values to be integers. Clearly, None > is not a member of that set. You could thus use it as a sentinel. So, > tell me, OP. Would this usage be correct? > > if event.some_id: > # stuff > > I would argue, "no." I would not argue that 0 should not compare as > False, however. You come up with an instance where 0 clearly does not apply as something to incur a different branch, which is very apparent in just reading the line, so very easy to spot as a mistake. But this 0 isn't a real zero. It's just a unique identifier that happens to be implemented as an integer(because it's convenient for your database, iterators and whatnot.) Let me show you an actual zero: if event.num_attendants: prepare_cake() else: cancel_event() When no one is coming to your party, is is clearly a different condition than if any number of people are coming to your event. When you read "if num_attendants", you can clearly tell this is going to do something depending on if people are coming or not. Let's read this line: if not event.date: How would you read it, knowing nothing about date objects? I would read it to mean "has no date been set?" and would naturally continue the code as such: if event.date: schedule_event() I cannot fathom one example where it could read as "does the party start at midnight?". Can you? From breamoreboy at yahoo.co.uk Wed Mar 5 16:10:28 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 15:10:28 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051510.54769.shai@platonix.com> <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> Message-ID: <lf7eou$bf6$1@ger.gmane.org> On 05/03/2014 15:01, Paul Moore wrote: > On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote: >> then, in a few years, people can go back to writing "if event.start_time" like >> they do with dates and datetimes. > > Why on earth would they do that? It's still bad practice. If you're > using None as a sentinel, you should test for it explicitly. Nobody > has yet suggested any other use case where this matters. > >>> Again, I'm not saying the current behaviour is sensible, but I doubt >>> the work to fix it will benefit anyone in practice. >>> >> >> So, you support my other suggestion -- a warning on every use of bool(time)? > > No. Good programming practice should cover that. We don't warn in > other cases where programmers make silly coding errors. Look at Skip's > message - should we also warn on uses of bool(int) because people can > write bad code that fails to work properly with zero, as well? > >> Also, the work to fix it is probably less than the work invested in this >> discussion so far... > > That is not obvious (given documentation, release management, etc, costs). > >> In any case, if we all agree that http://bugs.python.org/issue13936 is a valid >> problem -- can we re-open it, and then discuss how (or if) we solve it? > > We don't. Can we agree that it's not a bug and abandon this fruitless > discussion? > > Paul I wouldn't agree that this has been a fruitless discussion. I would agree that this is not a bug, that the code needs changing, and that at least one new test needs to be added to the unit test suite(s) for the buggy application(s). -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From shai at platonix.com Wed Mar 5 16:19:09 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 17:19:09 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> Message-ID: <201403051719.09276.shai@platonix.com> On Wednesday 05 March 2014 17:01:33 you wrote: > On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote: > > >> Again, I'm not saying the current behaviour is sensible, but I doubt > >> the work to fix it will benefit anyone in practice. > > [...] > > In any case, if we all agree that http://bugs.python.org/issue13936 is a > > valid problem > > We don't. Can we agree that it's not a bug and abandon this fruitless > discussion? > What, other than "bug", do you call behavior that isn't sensible? From shai at platonix.com Wed Mar 5 16:15:29 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 17:15:29 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> Message-ID: <201403051715.29773.shai@platonix.com> On Wednesday 05 March 2014 16:34:16 Skip Montanaro wrote: > > Let's change the problem slightly. Instead of time objects, let's > consider the domain of possible values to be integers. Clearly, None > is not a member of that set. You could thus use it as a sentinel. So, > tell me, OP. Would this usage be correct? > > if event.some_id: > # stuff > > I would argue, "no." I would not argue that 0 should not compare as > False, however. What Yann said. Also, when opening this thread, I argued that 0 is special, but midnight is not -- which you completely ignore. Thanks, Shai. From rosuav at gmail.com Wed Mar 5 16:23:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Mar 2014 02:23:19 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> <CAPTjJmrySz_1Y_Q5+RjCOW0sHFdHrhxKgD1FNDFjuSi+7WvH2g@mail.gmail.com> <CAHVvXxRtDJ7HepkGUn+JX1KKBdbHXfy35hUF686Y_0yA_ZVOxQ@mail.gmail.com> Message-ID: <CAPTjJmqywp2NgeKA7MXQD=cCinm1W+qjzAfafdYieGLwx2m5WQ@mail.gmail.com> On Wed, Mar 5, 2014 at 11:56 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > 3F could create a Fraction with the integer value 3 so that a/b gives > a rational number: > >>>> from fractions import Fraction as F >>>> a = 2 >>>> b = F(3) >>>> a/b > Fraction(2, 3) > > I don't understand why you say that can't be done. (Also Steven who said the same thing.) Uhh... brown-paper-bag moment. When I wrote up that post, I somehow blanked out the obvious fact that Fraction can happily represent an integer. Whoops... As Julia Jellicoe said, my objection falls to the ground. Very well! ChrisA From steve at pearwood.info Wed Mar 5 16:35:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 02:35:29 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> Message-ID: <20140305153529.GC28804@ando> On Wed, Mar 05, 2014 at 12:41:10PM +0000, Carl Smith wrote: > Just for the record, I never called them bills just because they contain a > dollar sign! I was thinking of a short word which describes an expression > who's evaluation changes depending on the local circumstances. Like a legal > bill, not a dollar bill. Hmmm. Well, I don't really get the connection between legal bills and delayed evaluation, and I still really dislike the name. > The name macro doesn't really work, as it's only an expression, it's not a > code block. There is nothing about macros that *require* them to accept a full block of code. In fact, there are multiple different meanings for macro, although they are all related they do have significant differences: http://en.wikipedia.org/wiki/Macro_%28computer_science%29 Macros in C are not the same kind of thing as macros in Lisp. > --- > > The idea was never to start passing complex expressions around the place. What you consider a complex expression somebody else may consider a simple expression. Unless you want to demand some arbitrary hard limit on complexity (and how do you measure complexity?) this "bill" system would have to allow the exact same types of expressions that are legal elsewhere in Python. [...] > What the hell is a thunk anyway? It's a horrible name. Wikipedia is your friend: http://en.wikipedia.org/wiki/Thunk Also, more here: http://c2.com/cgi/wiki?CallByName The meaning of thunk I'm referring to comes from the call-by-name argument passing model. Suppose we pass an expression to a function: function(arg=x-1) and the body of the function looks like this: if arg > 1: y = arg + 1 else: y = arg - 1 return y*arg In the "call-by-name" model, the function body executes like this: if (x+1) > 1: y = (x+1) + 1 else: y = (x+1) - 1 return y*(x+1) which not only duplicates the "x+1" part four times, but may require evaluating it three times. To avoid this wasteful duplication, the compiler creates a special thunk value, which is something like a function with no arguments: def thunk(): if cache is None: cache = x-1 return cache (This should be considered pseudo-code, not the exact way Algol or Haskell work -- real thunks also allow you to assign a value back to the "name". Also, technically the presence of a cache makes it call-by-need rather than call-by-name). The body of the function then becomes: if thunk() > 1: y = thunk() + 1 else: y = thunk() - 1 return y*thunk() So as you can see, what I've been calling a thunk is not precisely like Algol thunks. One thing which is missing is the dynamic scoping: in the thunk evaluation, the x comes from the caller's scope. But the delayed evaluation part is quite similar, enough that I think the name may be appropriate. Another name for this might be a "promise", as in the promise to execute a computation later. -- Steven From p.f.moore at gmail.com Wed Mar 5 16:37:14 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 15:37:14 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051719.09276.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> Message-ID: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote: > What, other than "bug", do you call behavior that isn't sensible? "Unfortunate". I'm not arguing that the behaviour is useful. Or sensible. What I'm arguing, and you're missing, is that the behaviour is as documented, and so can't be changed without backward compatibility implications. Specifically, see http://docs.python.org/3.4/library/datetime.html#time-objects: """in Boolean contexts, a time object is considered to be true if and only if, after converting it to minutes and subtracting utcoffset() (or 0 if that's None), the result is non-zero.""" Your code is ignoring documented behaviour. Worse, your code can clearly be improved by being explicit in your test, and yet you'd rather argue that the Python developers implement a backward compatibility break which you won't even be able to take advantage of until you drop support of versions of Python before 3.7. Paul. From ron3200 at gmail.com Wed Mar 5 16:38:27 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 05 Mar 2014 09:38:27 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305120410.GY28804@ando> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> Message-ID: <lf7gd6$1eg$1@ger.gmane.org> On 03/05/2014 06:04 AM, Steven D'Aprano wrote: > (By the way, I think it is somewhat amusing that Python not only has a > built-in complex type, but also*syntax* for creating complex numbers, > but no built-in support for exact rationals.) That is interesting. I think Mark is correct in unifying numbers. And also adding in decimal features. The way it should actually be done is another thing. But having an up to date package that can be used is a very good start. (Thanks Mark!) Mark describes an AI approach,, which I think he means having a internal representation that may change as needed depending on how a number can best be stored and calculated while still keeping it's accuracy. Weather or not that approach is called Decimals is another thing. It might be called "Unified Numbers".. or just Numbers. The point is for the internal representation to be an implementation detail the user doesn't need to worry about. And have Decimal features available by default. If things are decided by use case, then I can't even think of a good enough argument against having decimal features available by default. The financial use cases are that overwhelming. -Ron From p.f.moore at gmail.com Wed Mar 5 16:42:32 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 15:42:32 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> Message-ID: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote: > Let me show you an actual zero: > > if event.num_attendants: > prepare_cake() > else: > cancel_event() > > When no one is coming to your party, is is clearly a different > condition than if any number of people are coming to your event. When > you read "if num_attendants", you can clearly tell this is going to do > something depending on if people are coming or not. I'm sorry, are you really trying to say that the above code is better than if event.num_attendants != 0: prepare_cake() else: cancel_event() ? (Personally, I'd actually prefer something like "if event.num_attendants > 0" or switch the order of clauses and test for being equal to 0, but that's a minor issue. The major point is I'd prefer any of these to your version). Paul From shai at platonix.com Wed Mar 5 17:05:15 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 18:05:15 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> Message-ID: <201403051805.15363.shai@platonix.com> On Wednesday 05 March 2014 17:37:14 Paul Moore wrote: > On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote: > > What, other than "bug", do you call behavior that isn't sensible? > > "Unfortunate". That's unfortunate. You miss a lot of bugs. > I'm not arguing that the behaviour is useful. Or > sensible. What I'm arguing, and you're missing, is that the behaviour > is as documented, and so can't be changed without backward > compatibility implications. I think I have given ample room to backwards-compatibility considerations. > Specifically, see > http://docs.python.org/3.4/library/datetime.html#time-objects: > > """in Boolean contexts, a time object is considered to be true if and > only if, after converting it to minutes and subtracting utcoffset() > (or 0 if that's None), the result is non-zero.""" > Yes, I know. It's mentioned in the bug. > Your code is ignoring documented behaviour. Worse, your code can > clearly be improved by being explicit in your test, We disagree. I don't see adding "is not None" as an improvement at all. I see it as adding unnecessary ceremony. If you really like that much explicitness, use Java. > and yet you'd > rather argue that the Python developers implement a backward > compatibility break which you won't even be able to take advantage of > until you drop support of versions of Python before 3.7. > What Ryan said: The code in question will not benefit from this change; it is Python 2.7, and will probably be rewritten from scratch before anything comes out of this. This is about fixing my projects a decade from now. Shai. From ryan at ryanhiebert.com Wed Mar 5 16:50:04 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Wed, 5 Mar 2014 09:50:04 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> Message-ID: <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com> On Wed, Mar 5, 2014 at 9:37 AM, Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote: > > What, other than "bug", do you call behavior that isn't sensible? > > "Unfortunate". I'm not arguing that the behaviour is useful. Or > sensible. What I'm arguing, and you're missing, is that the behaviour > is as documented, and so can't be changed without backward > compatibility implications. Specifically, see > http://docs.python.org/3.4/library/datetime.html#time-objects: > > """in Boolean contexts, a time object is considered to be true if and > only if, after converting it to minutes and subtracting utcoffset() > (or 0 if that's None), the result is non-zero.""" > > Your code is ignoring documented behaviour. Worse, your code can > clearly be improved by being explicit in your test, and yet you'd > rather argue that the Python developers implement a backward > compatibility break which you won't even be able to take advantage of > until you drop support of versions of Python before 3.7. > > Or, perhaps, even though he wouldn't be able to use this change right away, he'd just like to see Python be better. I can't see that there's going to come a better time to discuss this issue than the present, thus it seems to me that you're suggesting that this odd and illogical behavior continue forever, simply because it's been documented. The backward compatibility and migration path still need to be addressed, but it seems that we are disagreeing on whether this strange behavior should ever change. I think that, at some point, it should. If we ever agree on that, then we can start thinking about when. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f158fa02/attachment.html> From ryan at ryanhiebert.com Wed Mar 5 16:55:13 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Wed, 5 Mar 2014 09:55:13 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> Message-ID: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> On Wed, Mar 5, 2014 at 9:42 AM, Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote: > > Let me show you an actual zero: > > > > if event.num_attendants: > > prepare_cake() > > else: > > cancel_event() > > > > When no one is coming to your party, is is clearly a different > > condition than if any number of people are coming to your event. When > > you read "if num_attendants", you can clearly tell this is going to do > > something depending on if people are coming or not. > > I'm sorry, are you really trying to say that the above code is better than > > if event.num_attendants != 0: > prepare_cake() > else: > cancel_event() > > ? > > (Personally, I'd actually prefer something like "if > event.num_attendants > 0" or switch the order of clauses and test for > being equal to 0, but that's a minor issue. The major point is I'd > prefer any of these to your version). I suspect you'd like the whole idea of empty collections and zeroes evaluating false to go away, to force this kind of style (one right way to do it and all). Some languages do that. Python isn't one of them. I can see both sides, but I like how Python can use empty lists and zeros as false values, and I often (but not always) write code that takes advantage of it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/3f02e5cf/attachment.html> From breamoreboy at yahoo.co.uk Wed Mar 5 17:19:28 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 16:19:28 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051719.09276.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> Message-ID: <lf7iqa$224$1@ger.gmane.org> On 05/03/2014 15:19, Shai Berger wrote: > On Wednesday 05 March 2014 17:01:33 you wrote: >> On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote: >> >>>> Again, I'm not saying the current behaviour is sensible, but I doubt >>>> the work to fix it will benefit anyone in practice. >>> > [...] >>> In any case, if we all agree that http://bugs.python.org/issue13936 is a >>> valid problem >> >> We don't. Can we agree that it's not a bug and abandon this fruitless >> discussion? >> > > What, other than "bug", do you call behavior that isn't sensible? Documented behaviour. As opposed to insanity: doing the same thing over and over again and expecting different results. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From rosuav at gmail.com Wed Mar 5 17:20:08 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Mar 2014 03:20:08 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051805.15363.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> <201403051805.15363.shai@platonix.com> Message-ID: <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> On Thu, Mar 6, 2014 at 3:05 AM, Shai Berger <shai at platonix.com> wrote: > On Wednesday 05 March 2014 17:37:14 Paul Moore wrote: >> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote: >> > What, other than "bug", do you call behavior that isn't sensible? >> >> "Unfortunate". > > That's unfortunate. You miss a lot of bugs. A bug is something where behaviour differs from documentation or intent. Differing from expectation isn't necessarily a bug (although in the absence of clear docs or statement of intent, it could be called one), partly because expectations vary. When the bulk of programmers expect one thing and the code does another, it's probably a wart or misfeature, but if it's properly documented, it can't really be called a bug. In this case, I would say that it's a wart. The fact that one timestamp happens to be treated as zero is odd, and I wouldn't advocate it, but now that it's there and documented, it's not a bug. ChrisA From p.f.moore at gmail.com Wed Mar 5 17:24:16 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 16:24:16 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> Message-ID: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com> On 5 March 2014 15:55, Ryan Hiebert <ryan at ryanhiebert.com> wrote: > I suspect you'd like the whole idea of empty collections and zeroes > evaluating false to go away, to force this kind of style (one right way to > do it and all). Some languages do that. Python isn't one of them. Not really. I just found the particular example given to be confusing. I tend to use boolean tests when I think of something as a yes/no type of thing, and the variable name implies it. If the example had referred to "event.attendees" I would probably have been fine with the check as written. The attribute in that case could be any one of a boolean flag, a count, or a list of attendees. That's actually a reasonably plausible way of writing the test to allow for later changes to the event class API. The point here is that there's more to whether code is good than just a black and white "does it work" question. But we're way off topic now - I don't even have any assurance that the OP's code is trying to test for a None sentinel and hitting a bug because not all time values evaluate as True (but if that's *not* his code, he really should have pointed that out by now!) All I can say is that *if* that is what his code is, then I think that rewriting it with an explicit test is clearer, and I'm not convinced by his assertion that "fixing" Python is a better solution. > I can see both sides, but I like how Python can use empty lists and zeros as > false values, and I often (but not always) write code that takes advantage > of it. Agreed, but that doesn't mean that code taking advantage of this is never obfuscated or hard to maintain. Paul. PS One final point, and I'll say this one last time and then shut up. I have no objection to making Python better. Nor do I insist that backward compatibility should never be broken. Nor do I wish Python were Java. Nor do I discount the issue of hard to find bugs. But equally, I don't think Python is perfect, nor do I think that this behaviour is ideal. I also don't think that the core developers have infinite time, nor do I think that every wart in Python needs to be fixed. And I trust the core developers to make the right judgements most of the time. All anyone should infer from this discussion is that I don't think that reopening the bug report referred to by the OP is productive. (You may also assume from the fact that I will drop out of this discussion now, that I no longer think that this thread is particularly productive :-() From ryan at ryanhiebert.com Wed Mar 5 16:20:57 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Wed, 5 Mar 2014 09:20:57 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051510.54769.shai@platonix.com> <CACac1F-xrwu-pP6dKMMyCrWe1QsiNFLydpGJCak1bf7osbgVag@mail.gmail.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> Message-ID: <CABpHFHS4gTBAsQYkj7MCANmgTJ8ftNsa4gR_cDqRcTceBDVaaA@mail.gmail.com> On Wed, Mar 5, 2014 at 9:01 AM, Paul Moore <p.f.moore at gmail.com> wrote: > On 5 March 2014 13:53, Shai Berger <shai at platonix.com> wrote: > > We don't. Can we agree that it's not a bug and abandon this fruitless > discussion. Please don't abandon this discussion. The behavior we're discussing is clearly illogical and useless. If we simply removed the __nonzero__ behavior, and it did what user-created types do, then it would perform as I would expect it to. Consider also that this time has no special significance, unlike the special cases of 0, (), {}, etc. This is true also for any arbitrarily zeroed unit, such as temperature (C or F), and including datetime, date, and time. Even zero in those cases (however that is defined) is simply an arbitrary point, and not the same as a nonzero value. This is completely illogical behavior, and, IMO, is a bug. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bb7db7b8/attachment.html> From rosuav at gmail.com Wed Mar 5 17:26:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Mar 2014 03:26:09 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> Message-ID: <CAPTjJmoAn9zpQi==RxW+S2iaiCdqymmgBTLq566nBmkM4+Fngw@mail.gmail.com> On Thu, Mar 6, 2014 at 2:55 AM, Ryan Hiebert <ryan at ryanhiebert.com> wrote: > I suspect you'd like the whole idea of empty collections and zeroes > evaluating false to go away, to force this kind of style (one right way to > do it and all). Some languages do that. Python isn't one of them. > > I can see both sides, but I like how Python can use empty lists and zeros as > false values, and I often (but not always) write code that takes advantage > of it. The broad concept is: A thing is true, a non-thing is false. The question is, though, which things are actually non-things? Since Python doesn't have a concept of not storing anything into a variable, there has to be a thing that represents the state of not being a thing, so None should be false. Python's made the choice that an empty string is a non-thing, but other languages choose instead to have a separate "Nil" value and all strings are true. Same with lists, tuples, etc. Pike's choice is that the integer 0 is false, and anything else is true, so you distinguish between having a string and not-having a string, rather than between having a non-empty string and having an empty string. Simple design choice, and one that each language follows consistently. But if you truly want to abolish the confusion, you want REXX. In REXX, 0 is false and 1 is true, and everything else is error 34, "Logical value not 0 or 1". Is that really an improvement? Not in my opinion. ChrisA From shai at platonix.com Wed Mar 5 17:38:25 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 18:38:25 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> Message-ID: <201403051838.25342.shai@platonix.com> On Wednesday 05 March 2014 18:20:08 Chris Angelico wrote: > On Thu, Mar 6, 2014 at 3:05 AM, Shai Berger <shai at platonix.com> wrote: > > On Wednesday 05 March 2014 17:37:14 Paul Moore wrote: > >> On 5 March 2014 15:19, Shai Berger <shai at platonix.com> wrote: > >> > What, other than "bug", do you call behavior that isn't sensible? > >> > >> "Unfortunate". > > > > That's unfortunate. You miss a lot of bugs. > > A bug is something where behaviour differs from documentation or > intent. Differing from expectation isn't necessarily a bug (although > in the absence of clear docs or statement of intent, it could be > called one), partly because expectations vary. When the bulk of > programmers expect one thing and the code does another, it's probably > a wart or misfeature, but if it's properly documented, it can't really > be called a bug. > > In this case, I would say that it's a wart. Fine. That is, I'm not too interested in the semantics of English, but rather in the semantics of Python. So, I rephrase my request: Would anybody object to a silent warning issued whenever a time-object is evaluated as Boolean? Something along the lines of, "Evaluating time object as Boolean: Note that time(0,0,0) evaluates to False" A silent warning, IIUC, is only printed when requested -- so it won't get in anybody's way, but still provide a tool for avoiding the bug. It can be added into Python 3.5. It could probably be added to earlier versions, but for that we'd need to consider this a bug :) Shai. From rob.cliffe at btinternet.com Wed Mar 5 18:00:40 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 05 Mar 2014 17:00:40 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com> Message-ID: <53175838.6020100@btinternet.com> On 05/03/2014 16:24, Paul Moore wrote: >> I can see both sides, but I like how Python can use empty lists and zeros as >> false values, and I often (but not always) write code that takes advantage >> of it. > Agreed, but that doesn't mean that code taking advantage of this is > never obfuscated or hard to maintain. > > Paul. I'm with Shai and Ryan on this one. There's no such thing as a perfect language, but that doesn't mean we shouldn't strive towards it. > > PS One final point, and I'll say this one last time and then shut up. > I have no objection to making Python better. Nor do I insist that > backward compatibility should never be broken. Nor do I wish Python > were Java. Nor do I discount the issue of hard to find bugs. But > equally, I don't think Python is perfect, nor do I think that this > behaviour is ideal. I also don't think that the core developers have > infinite time, nor do I think that every wart in Python needs to be > fixed. Good. In other words, each wart should be considered on its merits, and some may be worth fixing. Changes have been made to Python where the backward compatibility issue is far greater. In this case any code that would be broken by the change is already flawed, if not broken. And I find it hard to imagine that the development effort in making this change is anything other than minimal, _compared with other comparable changes_ (I'm not trying to minimise the undoubted effort required to make _any_ change no matter how small). Rob Cliffe > And I trust the core developers to make the right judgements > most of the time. All anyone should infer from this discussion is that > I don't think that reopening the bug report referred to by the OP is > productive. (You may also assume from the fact that I will drop out of > this discussion now, that I no longer think that this thread is > particularly productive :-() > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6652 - Release Date: 03/04/14 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a822a83c/attachment-0001.html> From skip at pobox.com Wed Mar 5 18:09:11 2014 From: skip at pobox.com (Skip Montanaro) Date: Wed, 5 Mar 2014 11:09:11 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051715.29773.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> Message-ID: <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote: > Also, when opening this thread, I argued that 0 is special, > but midnight is not -- which you completely ignore. Zero is not different, at least when considered as an element of a set (of integers, in my example). What's important is that we have a set of values (either time objects or integers in these two examples) and need some way to specify a sentinel value. That sentinel must not be in the set of valid values. In both of these cases, None fills the bill. Note that the only reasonable relationship between None and either the set of all time objects or the set of all integers is that it is not a member of either set. Testing that membership by appealing to their value cast to a boolean is simply wrong. I'll push the argument just a bit further. Suppose you have an attribute which corresponds to the setting of a two-valued sensor external to your system. You choose to represent those two values in your program as True and False. You need a way to tell if any sensor inputs have been detected yet, or not, so you initialize the attribute with a sentinel value. I don't care what value you choose out of all the possible sentinel values available to you (None, [], ["xyz"], {}, 7, 0.0, etc), treating any of them as a boolean will be wrong. If you do, you will erroneously always believe that your sensor has produced a value. I have made this sort of mistake many times over the years, and seen other people do it as well. If what you really want to test is membership of a value in a set, be explicit about it. Skip From alexander.belopolsky at gmail.com Wed Mar 5 18:27:21 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 12:27:21 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051838.25342.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> Message-ID: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com> wrote: > So, I rephrase my request: Would anybody object to a silent warning issued > whenever a time-object is evaluated as Boolean? > I would. Users should not be penalized for using a documented behavior. There are legitimate uses for bool(midnight) being False. Midnight is special in many contexts. For example, it is uncertain whether midnight belongs to the previous or next day. If your application wants to group midnight differently from other times - it is perfectly fine to use "if dt.time()" instead of a more verbose "if dt.time() != datetime.time(0, 0)". -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/3f930cd1/attachment.html> From amber.yust at gmail.com Wed Mar 5 18:27:27 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 05 Mar 2014 17:27:27 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> Message-ID: <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> Let's turn this around. Aside from backwards compatibility, what benefit is there from having midnight evaluate as false? On Wednesday, March 5, 2014 9:09:50 AM, Skip Montanaro <skip at pobox.com> wrote: > On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote: > > Also, when opening this thread, I argued that 0 is special, > > but midnight is not -- which you completely ignore. > > Zero is not different, at least when considered as an element of a set > (of integers, in my example). What's important is that we have a set > of values (either time objects or integers in these two examples) and > need some way to specify a sentinel value. That sentinel must not be > in the set of valid values. In both of these cases, None fills the > bill. Note that the only reasonable relationship between None and > either the set of all time objects or the set of all integers is that > it is not a member of either set. Testing that membership by appealing > to their value cast to a boolean is simply wrong. > > I'll push the argument just a bit further. Suppose you have an > attribute which corresponds to the setting of a two-valued sensor > external to your system. You choose to represent those two values in > your program as True and False. You need a way to tell if any sensor > inputs have been detected yet, or not, so you initialize the attribute > with a sentinel value. I don't care what value you choose out of all > the possible sentinel values available to you (None, [], ["xyz"], {}, > 7, 0.0, etc), treating any of them as a boolean will be wrong. If you > do, you will erroneously always believe that your sensor has produced > a value. > > I have made this sort of mistake many times over the years, and seen > other people do it as well. If what you really want to test is > membership of a value in a set, be explicit about it. > > Skip > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/2bb9f055/attachment.html> From shai at platonix.com Wed Mar 5 18:24:04 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 19:24:04 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> Message-ID: <201403051924.05252.shai@platonix.com> On Wednesday 05 March 2014 19:09:11 Skip Montanaro wrote: > On Wed, Mar 5, 2014 at 9:15 AM, Shai Berger <shai at platonix.com> wrote: > > Also, when opening this thread, I argued that 0 is special, > > but midnight is not -- which you completely ignore. > > Zero is not different, at least when considered as an element of a set > (of integers, in my example). What's important is that we have a set > of values (either time objects or integers in these two examples) and > need some way to specify a sentinel value. Your argument has merit, but is besides the point. As noted by many in this thread, the idiom "if obj:" for deciding on the existence of some value is quite common. We all know it doesn't always fit, because Python chose to make some objects falsey; however, when it does fit, it makes for succinct, clear, unceremonious code. If your argument is not that "if obj:" should never be used, except when obj actually is Boolean, please elaborate. At issue here is the fact that the idiom doesn't fit where one would expect that it should. Thanks, Shai. From ryan at ryanhiebert.com Wed Mar 5 17:38:56 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Wed, 5 Mar 2014 10:38:56 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <CABpHFHQFbX0JM5_wusqL9S_PcmLKXakfj-taOc6_U6k7sxS1Jg@mail.gmail.com> <CACac1F_1XjrTMp2t85ZyvJuJwfZ-rVtC3WVH_Gg-vTet-f89pg@mail.gmail.com> Message-ID: <CABpHFHTPBRGgBFj3vfNxmZD3wOBXKrzZGhXBXPsj_oeju=SGpA@mail.gmail.com> On Wed, Mar 5, 2014 at 10:24 AM, Paul Moore <p.f.moore at gmail.com> wrote: > > PS One final point, and I'll say this one last time and then shut up. > I have no objection to making Python better. Nor do I insist that > backward compatibility should never be broken. Nor do I wish Python > were Java. Nor do I discount the issue of hard to find bugs. But > equally, I don't think Python is perfect, nor do I think that this > behaviour is ideal. I also don't think that the core developers have > infinite time, nor do I think that every wart in Python needs to be > fixed. And I trust the core developers to make the right judgements > most of the time. All anyone should infer from this discussion is that > I don't think that reopening the bug report referred to by the OP is > productive. (You may also assume from the fact that I will drop out of > this discussion now, that I no longer think that this thread is > particularly productive :-() > Thanks for bringing this up. Even assuming that this was accepted as a bug, it's an entirely separate discussion to decide whether worth fixing or if it's worth the dev's time. I make no assertion on that, as that's their time and not mine. I hold the developers with highest regard, even in my disagreement. Python is my language of choice because of the effort of these folks, and I thank them. This seems like it could be a simple enough bug that were it allowed, it would be a fix that even someone as inexperienced with Python's internals as myself could fix. If that's not true and the issue is that it's simply too much work to change, then it should be marked "wontfix" rather than "invalid", and that's a decision that I would not contest (at least not without a working patch). I, like you, suspect that I will be abandoning further involvement, as simply arguing won't change people's minds. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5cc69176/attachment-0001.html> From p.f.moore at gmail.com Wed Mar 5 18:58:12 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 5 Mar 2014 17:58:12 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> Message-ID: <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> On 5 March 2014 17:27, Amber Yust <amber.yust at gmail.com> wrote: > Let's turn this around. Aside from backwards compatibility, what benefit is > there from having midnight evaluate as false? Zero cost. Any alternative must justify the cost of making the change. Simple as that. Quantify the cost of making the change (properly!) and describe the benefits in such a way as to justify those costs. Paul From shai at platonix.com Wed Mar 5 18:56:23 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 19:56:23 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> Message-ID: <201403051956.24081.shai@platonix.com> On Wednesday 05 March 2014 19:27:21 Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com> wrote: > > So, I rephrase my request: Would anybody object to a silent warning > > issued whenever a time-object is evaluated as Boolean? > > I would. Users should not be penalized for using a documented behavior. Fair enough. I would also add the warning to the documentation, and call the behavior "deprecated" (even if no steps will be taken to actually remove it). > There are legitimate uses for bool(midnight) being False. Midnight is > special in many contexts. For example, it is uncertain whether midnight > belongs to the previous or next day. If your application wants to group > midnight differently from other times - it is perfectly fine to use "if > dt.time()" instead of a more verbose "if dt.time() != datetime.time(0, 0)". The question is: Of all the Python lines in existence, where a time is evaluated as Boolean, what percentage intends to capture midnight, and what percentage intends to capture an empty field; the latter is a (user) bug, and I suspect it is the much larger part. So even though there are legitimate uses for midnight being false, keeping things as they are is setting a trap for users. It is documented, but highly unintuitive (as evidenced by many messages on this thread). I think it best if the gotcha is removed, but if not, I ask at least to have a way to mitigate the problem. Thanks, Shai. From amber.yust at gmail.com Wed Mar 5 19:12:46 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 5 Mar 2014 10:12:46 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> Message-ID: <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> I think this is the wrong stage to evaluate cost. After all, one of the tenets of open source is that the core devs don't have to be the source of all change. I see a difference between "this could be better, but the core devs aren't going to spend time on it" and "this should not change." It would be nice if those two were not conflated so much. On Mar 5, 2014 9:58 AM, "Paul Moore" <p.f.moore at gmail.com> wrote: > On 5 March 2014 17:27, Amber Yust <amber.yust at gmail.com> wrote: > > Let's turn this around. Aside from backwards compatibility, what benefit > is > > there from having midnight evaluate as false? > > Zero cost. Any alternative must justify the cost of making the change. > Simple as that. Quantify the cost of making the change (properly!) and > describe the benefits in such a way as to justify those costs. > > Paul > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/9ecbb54b/attachment.html> From breamoreboy at yahoo.co.uk Wed Mar 5 19:23:12 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 18:23:12 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> Message-ID: <lf7q2a$vpl$1@ger.gmane.org> On 05/03/2014 18:12, Amber Yust wrote: > I think this is the wrong stage to evaluate cost. After all, one of the > tenets of open source is that the core devs don't have to be the source > of all change. > But the core devs have to review the change and ultimately decide whether or not to commit it. In this case it looks as if a patch would not be accepted, so why keep going on about it, especially when there's a known work around? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From amber.yust at gmail.com Wed Mar 5 19:26:59 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 5 Mar 2014 10:26:59 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> Message-ID: <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com> Where did you see an indication that a patch would not be accepted? On Mar 5, 2014 10:24 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote: > On 05/03/2014 18:12, Amber Yust wrote: > >> I think this is the wrong stage to evaluate cost. After all, one of the >> tenets of open source is that the core devs don't have to be the source >> of all change. >> >> > But the core devs have to review the change and ultimately decide whether > or not to commit it. In this case it looks as if a patch would not be > accepted, so why keep going on about it, especially when there's a known > work around? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask what > you can do for our language. > > Mark Lawrence > > --- > This email is free from viruses and malware because avast! Antivirus > protection is active. > http://www.avast.com > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/abaa2db8/attachment.html> From skip at pobox.com Wed Mar 5 19:33:08 2014 From: skip at pobox.com (Skip Montanaro) Date: Wed, 5 Mar 2014 12:33:08 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051924.05252.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <201403051924.05252.shai@platonix.com> Message-ID: <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> On Wed, Mar 5, 2014 at 11:24 AM, Shai Berger <shai at platonix.com> wrote: > If your argument is not that "if obj:" should never be used, except > when obj actually is Boolean, please elaborate. My approach continues to be to consider the overall set of objects from which you draw values to assign to obj. Call that set V. If this expression is true: len([bool(x) for x in V]) == 1 then using "if obj:" might be valid (if the one "false" value of that set is some sort of distinguishing characteristic, as it often is). If, however, the length of that set is greater than one, you must be explicit in your conditional expressions. It is unfortunate for your purposes that bool(datetime.time(0, 0, 0)) is False, but it's a documented property of that object, and is highly unlikely to change. In your case V is None plus the set of all time objects. That has two values which evaluate to False, so "if obj:" is not appropriate. If, on the other hand, the set of all values is the set of all strings or the set of all integers or the set of all lists, then maybe the falsity of the zero values in those sets ("", 0, []) has some use. My last comment on this topic. I will invoke the Uncle Timmy rule (which I just made up). Never contradict a decision Uncle Timmy made. He had his reasons. They were very good reasons. They were almost certainly better than any reasons you can come up with as counterarguments. Deal with it. :-) Skip From shai at platonix.com Wed Mar 5 19:33:43 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 20:33:43 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> Message-ID: <201403052033.44162.shai@platonix.com> On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote: > On 05/03/2014 18:12, Amber Yust wrote: > > I think this is the wrong stage to evaluate cost. After all, one of the > > tenets of open source is that the core devs don't have to be the source > > of all change. > > But the core devs have to review the change and ultimately decide > whether or not to commit it. In this case it looks as if a patch would > not be accepted, so why keep going on about it, especially when there's > a known work around? As noted before, the workaround is trivial once you detect the problem. Detecting the problem is not easy. Shai. From shai at platonix.com Wed Mar 5 19:44:55 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 20:44:55 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> Message-ID: <201403052044.55720.shai@platonix.com> On Wednesday 05 March 2014 20:33:08 Skip Montanaro wrote: > On Wed, Mar 5, 2014 at 11:24 AM, Shai Berger <shai at platonix.com> wrote: > > If your argument is not that "if obj:" should never be used, except > > when obj actually is Boolean, please elaborate. > > My approach continues to be to consider the overall set of objects > from which you draw values to assign to obj. Call that set V. If this > expression is true: > > len([bool(x) for x in V]) == 1 > > then using "if obj:" might be valid (if the one "false" value of that > set is some sort of distinguishing characteristic, as it often > is). If, however, the length of that set is greater than one, you must > be explicit in your conditional expressions. No argument so far. > It is unfortunate for your purposes that bool(datetime.time(0, 0, 0)) is > False, but it's a documented property of that object, and is highly unlikely > to change. I think this is what's called 'begging the question' -- this discussion was not about the validity of boolean testing in general, but about the validity of this property of time objects. Yes, it's documented. I'm trying to make it change. There is a lot of support for that, and the opposition seems to be centered on "because it's documented" -- with the two exceptions being an edge use-case (actually checking midnight) and your call to authority below. It's a gotcha. Let's remove it. If we don't remove it, let's give ourselves a tool to detect it. > My last comment on this topic. I will invoke the Uncle Timmy rule > (which I just made up). Never contradict a decision Uncle Timmy > made. He had his reasons. They were very good reasons. They were > almost certainly better than any reasons you can come up with as > counterarguments. Deal with it. :-) > The reason he actually gave (on the ticket)? "because it's documented". Shai. From breamoreboy at yahoo.co.uk Wed Mar 5 19:47:02 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 18:47:02 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403052033.44162.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> <201403052033.44162.shai@platonix.com> Message-ID: <lf7rf0$e7m$1@ger.gmane.org> On 05/03/2014 18:33, Shai Berger wrote: > On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote: >> On 05/03/2014 18:12, Amber Yust wrote: >>> I think this is the wrong stage to evaluate cost. After all, one of the >>> tenets of open source is that the core devs don't have to be the source >>> of all change. >> >> But the core devs have to review the change and ultimately decide >> whether or not to commit it. In this case it looks as if a patch would >> not be accepted, so why keep going on about it, especially when there's >> a known work around? > > As noted before, the workaround is trivial once you detect the problem. > Detecting the problem is not easy. > > Shai. Use mocks in unit testing? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From breamoreboy at yahoo.co.uk Wed Mar 5 19:59:37 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 18:59:37 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com> Message-ID: <lf7s6j$rl4$1@ger.gmane.org> On 05/03/2014 18:26, Amber Yust wrote: > Where did you see an indication that a patch would not be accepted? > > On Mar 5, 2014 10:24 AM, "Mark Lawrence" > <breamoreboy at yahoo.co.uk > <mailto:breamoreboy at yahoo.co.uk>> wrote: > > On 05/03/2014 18:12, Amber Yust wrote: > > I think this is the wrong stage to evaluate cost. After all, one > of the > tenets of open source is that the core devs don't have to be the > source > of all change. > > > But the core devs have to review the change and ultimately decide > whether or not to commit it. In this case it looks as if a patch > would not be accepted, so why keep going on about it, especially > when there's a known work around? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence > > --- http://bugs.python.org/issue13936 has been closed as invalid and I don't see a rush of core devs backing this proposal. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From amber.yust at gmail.com Wed Mar 5 20:06:50 2014 From: amber.yust at gmail.com (Amber Yust) Date: Wed, 5 Mar 2014 11:06:50 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf7s6j$rl4$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com> <lf7s6j$rl4$1@ger.gmane.org> Message-ID: <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com> It was closed by one or two individuals, and there has been significant comment since then. Asking to revisit a decision isn't crazy, and there have been reasonable suggestions made. I don't see a lack of core dev "jumping upon" as an active indication that patches would be rejected; merely that core devs are unlikely to produce such a patch themselves. On Mar 5, 2014 11:00 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote: > On 05/03/2014 18:26, Amber Yust wrote: > >> Where did you see an indication that a patch would not be accepted? >> >> On Mar 5, 2014 10:24 AM, "Mark Lawrence" >> <breamoreboy at yahoo.co.uk >> <mailto:breamoreboy at yahoo.co.uk>> wrote: >> >> On 05/03/2014 18:12, Amber Yust wrote: >> >> I think this is the wrong stage to evaluate cost. After all, one >> of the >> tenets of open source is that the core devs don't have to be the >> source >> of all change. >> >> >> But the core devs have to review the change and ultimately decide >> whether or not to commit it. In this case it looks as if a patch >> would not be accepted, so why keep going on about it, especially >> when there's a known work around? >> >> -- >> My fellow Pythonistas, ask not what our language can do for you, ask >> what you can do for our language. >> >> Mark Lawrence >> >> --- >> > > http://bugs.python.org/issue13936 has been closed as invalid and I don't > see a rush of core devs backing this proposal. > > -- > My fellow Pythonistas, ask not what our language can do for you, ask what > you can do for our language. > > Mark Lawrence > > --- > This email is free from viruses and malware because avast! Antivirus > protection is active. > http://www.avast.com > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/007cdeda/attachment-0001.html> From breamoreboy at yahoo.co.uk Wed Mar 5 20:19:11 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 19:19:11 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> <CAE0SK642+qyo6SireYwJkKzC0UN6jB1Apq32vPtcYJOx-QJ5nQ@mail.gmail.com> <lf7s6j$rl4$1@ger.gmane.org> <CAE0SK65-1cg_XxndBmMqUWYjiGun_YMOFA0LTMUX7b0EAa4n0w@mail.gmail.com> Message-ID: <lf7tb9$a8p$1@ger.gmane.org> On 05/03/2014 19:06, Amber Yust wrote: > It was closed by one or two individuals, and there has been significant > comment since then. Asking to revisit a decision isn't crazy, and there > have been reasonable suggestions made. I don't see a lack of core dev > "jumping upon" as an active indication that patches would be rejected; > merely that core devs are unlikely to produce such a patch themselves. > > On Mar 5, 2014 11:00 AM, "Mark Lawrence" > <breamoreboy at yahoo.co.uk > <mailto:breamoreboy at yahoo.co.uk>> wrote: > > On 05/03/2014 18:26, Amber Yust wrote: > > Where did you see an indication that a patch would not be accepted? > > On Mar 5, 2014 10:24 AM, "Mark Lawrence" > <breamoreboy at yahoo.co.uk > <mailto:breamoreboy at yahoo.co.uk> > <mailto:breamoreboy at yahoo.co.__uk > <mailto:breamoreboy at yahoo.co.uk>>> > wrote: > > On 05/03/2014 18:12, Amber Yust wrote: > > I think this is the wrong stage to evaluate cost. After > all, one > of the > tenets of open source is that the core devs don't have > to be the > source > of all change. > > > But the core devs have to review the change and ultimately > decide > whether or not to commit it. In this case it looks as if a > patch > would not be accepted, so why keep going on about it, > especially > when there's a known work around? > > -- > My fellow Pythonistas, ask not what our language can do for > you, ask > what you can do for our language. > > Mark Lawrence > > --- > > http://bugs.python.org/__issue13936 > <http://bugs.python.org/issue13936> has been closed as invalid and I > don't see a rush of core devs backing this proposal. > There can be lots of comments by Her Majesty Queen Elizabeth II or President Obama but they don't really count unless they put forward arguments to pursuade the core devs to change their minds about this issue. I see no evidence here that this is likely to happen, or do the core devs all come to life at the same time as they're all in the same time zone? A patch can be placed on the issue but if a core dev reviews and it and decides not to commit it nothing happens? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From shai at platonix.com Wed Mar 5 20:18:37 2014 From: shai at platonix.com (Shai Berger) Date: Wed, 5 Mar 2014 21:18:37 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf7rf0$e7m$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org> Message-ID: <201403052118.38295.shai@platonix.com> On Wednesday 05 March 2014 20:47:02 Mark Lawrence wrote: > On 05/03/2014 18:33, Shai Berger wrote: > > On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote: > >> On 05/03/2014 18:12, Amber Yust wrote: > >>> I think this is the wrong stage to evaluate cost. After all, one of the > >>> tenets of open source is that the core devs don't have to be the source > >>> of all change. > >> > >> But the core devs have to review the change and ultimately decide > >> whether or not to commit it. In this case it looks as if a patch would > >> not be accepted, so why keep going on about it, especially when there's > >> a known work around? > > > > As noted before, the workaround is trivial once you detect the problem. > > Detecting the problem is not easy. > > > > Shai. > > Use mocks in unit testing? Unless you're aware of the issue, you are not going to unit-test specifically for midnight. If you are aware, you don't need to detect it, you write "is not None". From steve at pearwood.info Wed Mar 5 20:24:19 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 06:24:19 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403052044.55720.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> Message-ID: <20140305192419.GD28804@ando> On Wed, Mar 05, 2014 at 08:44:55PM +0200, Shai Berger wrote: > > It is unfortunate for your purposes that bool(datetime.time(0, 0, 0)) is > > False, but it's a documented property of that object, and is highly unlikely > > to change. > > I think this is what's called 'begging the question' -- this discussion was > not about the validity of boolean testing in general, but about the validity > of this property of time objects. Yes, it's documented. I'm trying to make it > change. There is a lot of support for that, and the opposition seems to be > centered on "because it's documented" -- with the two exceptions being an edge > use-case (actually checking midnight) and your call to authority below. There are some very powerful practical reasons for leaving the status quo, namely backwards-compatibility and code-churn. These are real costs, not for the person who writes the patch, or the people who write the tests, and document the change. But these are potential costs for the millions of people using Python who may be *relying* on midnight being falsey, or nevertheless will have to change their code if this change occurs. There's also the argument from consistency: midnight is the zero point of the day, and zero is falsey. It would be surprising if something which is essentially zero became truthy. Compared to those costs, the benefits are insignificant. For library authors, they cannot take advantage of this new always-truthy time objects for probably eight or ten years, when they drop all support for any prior to 3.6 or 3.7. A tiny benefit, discounted even further because it is so far in the distance, versus significant costs. Application authors, only using a single version of Python, may be able to make use of this rather sooner: perhaps as little as three years from now. > It's a gotcha. Let's remove it. If we don't remove it, let's give ourselves a > tool to detect it. Um, it's easy to detect it. not time_obj returns True if time_obj is midnight and therefore falsey. -- Steven From cfkaran2 at gmail.com Wed Mar 5 20:24:57 2014 From: cfkaran2 at gmail.com (CFK) Date: Wed, 5 Mar 2014 14:24:57 -0500 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> Message-ID: <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> On Wed, Mar 5, 2014 at 8:45 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 5 Mar 2014 22:51, "Paul Moore" <p.f.moore at gmail.com> wrote: > > > > On 5 March 2014 11:53, Cem Karan <cfkaran2 at gmail.com> wrote: > > > Thoughts/suggestions? > > > > I think the core/stdlib position is that agreeing conventions would be > > better done once some real world experience of the practical issues > > and benefits of annotations has been established. > > MyPy uses function annotations for optional static typing, which is pretty > much the use case Guido originally had in mind and the main reason that PEP > 8 doesn't make combining annotations with an associated decorator > mandatory: http://www.mypy-lang.org/ > > You do still have to import a particular module to indicate that all > annotations in the importing module are to be interpreted as type > annotations. > > Beyond that, the guidance in PEP 8 stands: > > """It is recommended that third party experiments with annotations use an > associated decorator to indicate how the annotation should be > interpreted.""" > I like what all of you are suggesting; decorators are the way to go. If a project defines its own annotation decorators as sigtools.modifiers.annotate<http://sigtools.readthedocs.org/en/latest/_modules/sigtools/modifiers.html#annotate>, mypy, <http://www.mypy-lang.org/> or pyanno<http://www.fightingquaker.com/pyanno>do, then it shouldn't be too hard for a project to add its own UUID to the annotation dictionary. I'll spend a little while this weekend seeing if I can come up with some proof-of-concept code to make this work in a portable way. If the signatures looked vaguely like the following: @type_annotation(some_arg, int) @return_type_annotation(int) def function(some_arg): pass Would that be appealing to everyone? One question though, are we allowed to modify a functions __annotation__ dictionary directly? I know that I can do it, I just don't know if it is discouraged. Thanks, Cem Karan <http://www.fightingquaker.com/pyanno> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f8031bdd/attachment.html> From breamoreboy at yahoo.co.uk Wed Mar 5 20:35:29 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 19:35:29 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403052118.38295.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org> <201403052118.38295.shai@platonix.com> Message-ID: <lf7u9q$kdv$1@ger.gmane.org> On 05/03/2014 19:18, Shai Berger wrote: > On Wednesday 05 March 2014 20:47:02 Mark Lawrence wrote: >> On 05/03/2014 18:33, Shai Berger wrote: >>> On Wednesday 05 March 2014 20:23:12 Mark Lawrence wrote: >>>> On 05/03/2014 18:12, Amber Yust wrote: >>>>> I think this is the wrong stage to evaluate cost. After all, one of the >>>>> tenets of open source is that the core devs don't have to be the source >>>>> of all change. >>>> >>>> But the core devs have to review the change and ultimately decide >>>> whether or not to commit it. In this case it looks as if a patch would >>>> not be accepted, so why keep going on about it, especially when there's >>>> a known work around? >>> >>> As noted before, the workaround is trivial once you detect the problem. >>> Detecting the problem is not easy. >>> >>> Shai. >> >> Use mocks in unit testing? > > Unless you're aware of the issue, you are not going to unit-test specifically > for midnight. If you are aware, you don't need to detect it, you write "is not > None". So you have your solution. Good, now can we please move on as this is getting tedious. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From rymg19 at gmail.com Wed Mar 5 20:44:01 2014 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Wed, 5 Mar 2014 13:44:01 -0600 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> <20140305014611.GA4396@cskk.homeip.net> <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> Message-ID: <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com> On Tue, Mar 4, 2014 at 8:36 PM, David Mertz <mertz at gnosis.cx> wrote: > On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote: > >> > foo = 1 >> > a = $(foo + 1) >> Definitely nicer. Still irrationally uncomfortable about the "$" though. >> A thought, though it could break existing code (and nested tuples, alas): >> >> a = (( foo + 1 )) >> > > That looks like unresolvable ambiguity to me. I confess that I am more > comfortable with '$(...)' because I'm one of those folks who actually likes > bash, and uses that spelling often over there (where the meaning isn't the > *same* as this, but is enough similar for the meaning to carry over) > > But this is Python, which is 10x better. And besides, that syntax gives me GNU make nightmares. > Still, what does this mean? >> a = 3 + (( foo + 1 )) >> I think that would need to be a syntax error, because I can't see it being >> anything except nonsense otherwise. > > I see it as: Create an anonymous function object that adds foo to 1. Then, try and add 3 to that resulting object(which obviously would fail). It'd be kind of like: a = 3 + (lambda: foo+1) a.k.a: def myfunc(): return foo+1 a = 3+myfunc or(somewhat clearer in C++): SomeType a = 3 + [&]() { return foo+1; }; It's a bit more obvious of the error in the C++ example(or, at least to me). ... > -- > Keeping medicines from the bloodstreams of the sick; food > from the bellies of the hungry; books from the hands of the > uneducated; technology from the underdeveloped; and putting > advocates of freedom in prisons. Intellectual property is > to the 21st century what the slave trade was to the 16th. > -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated." -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/b26c4ce0/attachment.html> From haoyi.sg at gmail.com Wed Mar 5 20:50:36 2014 From: haoyi.sg at gmail.com (Haoyi Li) Date: Wed, 5 Mar 2014 11:50:36 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com> References: <CAEbHw4YQkC=er0wQsx8TOqz2v3-J6YysxsUjrYTwhK=+b1gtyw@mail.gmail.com> <20140305014611.GA4396@cskk.homeip.net> <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <CAO41-mOt7wEUs+wzYUZRAaHh6_MEnB3+Y2TAoDhNr-Fz5WSkYA@mail.gmail.com> Message-ID: <CALruUQJZkn=QKOAv2-9RrGaiCtJQ2Xfk=r+3rKGnUbH7OLPyDg@mail.gmail.com> > The idea was never to start passing complex expressions around the place. It was just to allow a function to take an expression containing about one or two names, and evaluate the expression internally, so this... > > func(lambda foo: foo > 0) > > ...can become... > > func($foo > 0) >>> from macropy.quick_lambda import macros, f, _ >>> map(f[_ + 1], [1, 2, 3]) [2, 3, 4] >From an external API, it seems exactly the same. From an internal point of view, you have to pass *foo* explicitly to the function you capture, but I'd argue that that's probably a good thing in the majority of cases. There are cases where you'd want to "evaluate the expression internally" to automagically inject identifiers into the scope of the expression, but those are few and far between. On Wed, Mar 5, 2014 at 11:44 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote: > On Tue, Mar 4, 2014 at 8:36 PM, David Mertz <mertz at gnosis.cx> wrote: > >> On Tue, Mar 4, 2014 at 5:46 PM, Cameron Simpson <cs at zip.com.au> wrote: >> >>> > foo = 1 >>> > a = $(foo + 1) >>> Definitely nicer. Still irrationally uncomfortable about the "$" though. >>> A thought, though it could break existing code (and nested tuples, alas): >>> >>> a = (( foo + 1 )) >>> >> >> That looks like unresolvable ambiguity to me. I confess that I am more >> comfortable with '$(...)' because I'm one of those folks who actually likes >> bash, and uses that spelling often over there (where the meaning isn't the >> *same* as this, but is enough similar for the meaning to carry over) >> >> > > But this is Python, which is 10x better. And besides, that syntax gives me > GNU make nightmares. > > >> Still, what does this mean? >>> a = 3 + (( foo + 1 )) >>> I think that would need to be a syntax error, because I can't see it >>> being >>> anything except nonsense otherwise. >> >> > I see it as: > > Create an anonymous function object that adds foo to 1. Then, try and add > 3 to that resulting object(which obviously would fail). It'd be kind of > like: > > a = 3 + (lambda: foo+1) > > a.k.a: > > def myfunc(): return foo+1 > a = 3+myfunc > > or(somewhat clearer in C++): > > SomeType a = 3 + [&]() { return foo+1; }; > > It's a bit more obvious of the error in the C++ example(or, at least to > me). > > ... >> -- >> Keeping medicines from the bloodstreams of the sick; food >> from the bellies of the hungry; books from the hands of the >> uneducated; technology from the underdeveloped; and putting >> advocates of freedom in prisons. Intellectual property is >> to the 21st century what the slave trade was to the 16th. >> > > > > -- > Ryan > If anybody ever asks me why I prefer C++ to C, my answer will be simple: > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was > nul-terminated." > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e9cf69e5/attachment.html> From abarnert at yahoo.com Wed Mar 5 21:30:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 12:30:23 -0800 (PST) Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <20140305112532.GV28804@ando> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <6939FECB-867D-46EC-B5D2-737DA205590D@yahoo.com> <20140305112532.GV28804@ando> Message-ID: <1394051423.82411.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Steven D'Aprano <steve at pearwood.info> Sent: Wednesday, March 5, 2014 3:25 AM > On Tue, Mar 04, 2014 at 05:05:39PM -0800, Andrew Barnert wrote: >> On Mar 4, 2014, at 14:31, Greg Ewing <greg.ewing at canterbury.ac.nz> > wrote: >> >> > Steven D'Aprano wrote: >> >> What I have in my head is some vague concept that the Python > evaluation rules will somehow know when to evaluate the thunk and when to treat > it as an object, which is (as I understand it) what happens in Algol. >> > >> > But Algol has the benefit of static typing -- the procedure >> > being called explicitly declares whether the argument is to >> > be passed by name or value. Python has no idea about that at >> > compile time. > > I'm not convinced that this really matters, but for the sake of the > argument let's say it does. It's not really static typing that's key here; it's that, unless you have explicit syntax for all uses of thunks, you either need implicit type casts, and static typing is necessary for implicit type casts. (However, as I said before, this?doesn't have to be a deal-breaker. You can probably get away with making all references either immediate or delayed, so long as the syntax for doing the other explicitly?isn't too obtrusive and the performance cost isn't too high. For example, if referencing a thunk always evaluates, `thunk` just creates a new thunk that will evaluate the old one, and you could optimize that into not much heavier than just passing the old one around?with a JIT, like PyPy, you could even optimize it into just passing the old one around.) >> This is the main reason I think it's more productive to think of this >> in terms of Lisp-style quoting than Algol-style thunks. > > Can you give more detail please? I think I've already explained it, but let me try again in more detail. A quoted expression and a thunk are similar things: ways to turn a language expression into something that can be evaluated or executed later. But there are two major differences. First, quoted expressions are first-class values, while thunks are not. Second, quoted expressions have an inspectable (or pattern-matchable) structure, while thunks do not. You could relate that second difference to ASTs vs. code objects in Python?but since idiomatic Python does not use macros or any other generation or parsing of ASTs, it really isn't a visible difference, so the first one is more important.* And then there's the historical difference: thunks come from static languages, quoting from dynamic languages. Together with being first-class values,?this means quoted expressions need explicit but not-too-intrusive syntax not just for creating them, but also either for evaluating them, or for continuing to delay them. So, that's why I think quoting is a more apt model for what we're trying to accomplish in this thread. * As I mentioned before, it does actually matter for _implementation_ whether we use ASTs or code objects, at least if we want dynamic or flat in-place scoping, because the former would allow us to look up names at execution time, while the latter would not, unless we either add a LOAD_DYNAMIC opcode or compile name lookup into explicit dynamic lookup code. But as long as we define what the scoping rules are, users won't care how that's implemented. From abarnert at yahoo.com Wed Mar 5 21:32:38 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 12:32:38 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> Message-ID: <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Oscar Benjamin <oscar.j.benjamin at gmail.com> Sent: Wednesday, March 5, 2014 3:29 AM > On 5 March 2014 08:10, Andrew Barnert <abarnert at yahoo.com> wrote: >> From: Mark H. Harris <harrismh777 at gmail.com> >>> >>> ? I am expecting that (just as in Rexx numbers) defining very clearly > what >>> is a python number? will be key for wide adaptation of the concept. But > there >>> should be no surprises for users, particularly average users. Anyone > with >>> a middle school expectation of a numeric format should be able to use >>> python numbers without surprises. >> >> Anyone with a middle school expectation will expect 1/3 to be a > fraction--or, at least, something they can multiply by 3 to get exactly 1. Using > an inexact decimal float instead of an inexact binary float is no improvement at > all. > > I actually think that it is an improvement. Most people are surprised > by the fact that just writing x = 0.1 causes a rounding error. Python > only has dedicated syntax for specifying binary floats in terms of > decimal digits meaning that there is no syntax for exactly specifying > non integer numbers. I would say that that is clearly a sub-optimal > situation. > > I don't agree with Mark's proposal in this thread but I would like to > have decimal literals e.g. 1.23d, and I would also use Fraction > literals if available e.g. 1/3F. I agree with you completely on that. That's kind of my point?your suggestion is almost the _opposite_ of the proposal in this thread. You want to make it easier for people to use the appropriate type for each use case; he wants to eliminate that choice entirely so you never have to make it. The latter would be nice if it were doable, but it's not even possible in principle. So the former is the best choice. From bruce at leapyear.org Wed Mar 5 22:04:56 2014 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 5 Mar 2014 13:04:56 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140305192419.GD28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> Message-ID: <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> On Wed, Mar 5, 2014 at 11:24 AM, Steven D'Aprano <steve at pearwood.info>wrote: > There's also the argument from consistency: midnight is the zero point > of the day, and zero is falsey. It would be surprising if something > which is essentially zero became truthy. > I don't think it's consistent now. In Boolean contexts ... - a timedelta<http://docs.python.org/3.4/library/datetime.html#datetime.timedelta> object is considered to be true if and only if it isn't equal to timedelta(0). - all date<http://docs.python.org/3.4/library/datetime.html#datetime.date> objects are considered to be true. - all datetime<http://docs.python.org/3.4/library/datetime.html#datetime.datetime> objects are considered to be true. - a time <http://docs.python.org/3.4/library/datetime.html#datetime.time> object is considered to be true if and only if, after converting it to minutes and subtracting utcoffset() (or 0 if that's None), the result is non-zero. And the documentation is not exactly clear for understanding when a time object is falsey. In particular, the "converting it to minutes" part seems irrelevant as a time of 1 microsecond from midnight is truthy and timedeltas are not numbers and are not in minutes. Perhaps a reasonable improvement would be to change this to: - a time <http://docs.python.org/3.4/library/datetime.html#datetime.time> object is considered to be true unless it represents exactly midnight local time, that is, it's false if subtracting the utcoffset() from the time produces an all-zero result (or it's all-zero to start with if the utcoffset is None). --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/7412a561/attachment.html> From alex.gaynor at gmail.com Wed Mar 5 22:10:12 2014 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 5 Mar 2014 21:10:12 +0000 (UTC) Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight References: <201403051216.12392.shai@platonix.com> Message-ID: <loom.20140305T220007-633@post.gmane.org> Shai Berger <shai at ...> writes: > > Hi all, > > This is my first post here, following a recommendation from Alexander > Belopolsky to use this list, to try to convince the Python developers to > reopen a ticket. I am a long-time Python user, and a Django committer. > > http://bugs.python.org/issue13936 is a complaint about the fact that midnight I strongly support fixing this issue. This behavior is so fundamentally nonsensical that resolving it is worth the potential compatibility issues. First, I'd like to dispense with the notion that ``if foo`` is somehow a bad practice. This is an EXTREMELY common practice as a shorthand for checking for None, a great many people consider it more idiomatic, this really isn't a place to bikeshed that particlar idiom. Now I'd like to address whether this is correct behavior: midnight is not a special time. It does not differ from other times in magnitutde (as with a container), it is not equivilant to the False value (as with 0). It is simply, sometimes, represented as 00:00:00 (or 12:00:00!). The fact that it's representation is composed of zeros no more makes it false than the point in 3-space at (0, 0, 0) is false. It's still a completely normal time, which in no way distinguishes itself from any other. I don't have a strong perspective on what the exact deprecation or breakage cycle should like, but it's very clear to me that this behavior is nonsensical. Out of a sample of half-a-dozen python programmers I've told about this, the *universal* response was confusion and revulsion. I strongly suspect that the majority of people writing ``if time_object`` have a bug in their code. Alex From ron3200 at gmail.com Wed Mar 5 22:13:29 2014 From: ron3200 at gmail.com (Ron Adam) Date: Wed, 05 Mar 2014 15:13:29 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <lf841b$rt6$1@ger.gmane.org> On 03/05/2014 02:32 PM, Andrew Barnert wrote: >>> I don't agree with Mark's proposal in this thread but I would like >>> to have decimal literals e.g. 1.23d, and I would also use Fraction >>> literals if available e.g. 1/3F. > > I agree with you completely on that. That's kind of my point?your > suggestion is almost the_opposite_ of the proposal in this thread. You > want to make it easier for people to use the appropriate type for each > use case; he wants to eliminate that choice entirely so you never have > to make it. The latter would be nice if it were doable, but it's not > even possible in principle. So the former is the best choice. It's also a matter of how far in the future you are looking. If he waited until later to propose something like this, it most likely wouldn't get in. I'm not sure he's expecting a firm answer now, but probably is hoping for a "hey lets look into this more and see if it's really possible" kind of maybe. For that, it's good to start early. On the near term, adding literal syntax's as you describe here would get all the pieces into python. Then an import from future could enable what Mark is thinking. He really needs to flesh out the details first before we can make any objective opinions about how it would actually work. The way I see it, with a unified number type, we will still need context like api's to get the more specialised behaviours. The difference may be a decorator on a function that specifies some subset of number types to use rather than notating each literal and casting each value. @numbers(int, decimal) def acounting_foo(...): ... # using ints for integers # using decimal for reals ... That just a guess... maybe Mark can give some examples how what he's proposing would work. ? Cheers, Ron From mertz at gnosis.cx Wed Mar 5 22:20:59 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 5 Mar 2014 13:20:59 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305114344.GX28804@ando> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <20140305034945.GA67087@cskk.homeip.net> <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> <20140305114344.GX28804@ando> Message-ID: <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com> On Wed, Mar 5, 2014 at 3:43 AM, Steven D'Aprano <steve at pearwood.info> wrote: > >That is, what's really the point of having > > > > a = $(expr) # or `expr`, or `(expr), or c"expr" > > > > If it's simply a slightly shorter way of spelling: > > > > a = compile(expr, "<string>", "eval") > > You can't write it like that. You have to wrap the expression is quotes > and turn it into a string: > > a = compile("expr", "<string>", "eval") > True, I misspelled my example. Or maybe my 'expr' implicitly meant string_expr :-). But the basic point that the literal for a "thunk" is already essentially available with compile() remains. > which means you lose syntax highlighting. Perhaps the ability to get > syntax highlighting is not sufficient to justify this idea. > There's no reason you even NEED to lose syntax highlighting. A code editor could perfectly well have a highlight rule that strings inside compile() calls get highlighted. Sure, that's a pretty special language mode, but there's nothing that an editor couldn't do in principle. > Another disadvantage: the ever-present temptation to pass a > user-generated string to compile: > > a = compile(template % some_string, "<string>", "eval") > > If some_string came from an untrusted source, you are now the proud > owner of a brand new code injection vulnerability. But with syntax, you > cannot generate a thunk/macro except from code you write yourself. It's > still code written by you, it's just that evaluation is delayed. > The literal hardly saves you from injection attacks. I could write this too under the proposed idea: foo = get_string_from_attacker() a = $(foo) b = a.eval() Now one can say "don't do that!" ... but that advice applies just as well to 'compile(unsafe_string, ...)' -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/d4fd1e42/attachment.html> From donald at stufft.io Wed Mar 5 22:31:36 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 16:31:36 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <loom.20140305T220007-633@post.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> Message-ID: <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> On Mar 5, 2014, at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote: > Shai Berger <shai at ...> writes: > >> >> Hi all, >> >> This is my first post here, following a recommendation from Alexander >> Belopolsky to use this list, to try to convince the Python developers to >> reopen a ticket. I am a long-time Python user, and a Django committer. >> >> http://bugs.python.org/issue13936 is a complaint about the fact that midnight > > > I strongly support fixing this issue. This behavior is so fundamentally > nonsensical that resolving it is worth the potential compatibility issues. > > First, I'd like to dispense with the notion that ``if foo`` is somehow a bad > practice. This is an EXTREMELY common practice as a shorthand for checking for > None, a great many people consider it more idiomatic, this really isn't a place > to bikeshed that particlar idiom. > > Now I'd like to address whether this is correct behavior: midnight is not a > special time. It does not differ from other times in magnitutde (as with a > container), it is not equivilant to the False value (as with 0). It is simply, > sometimes, represented as 00:00:00 (or 12:00:00!). The fact that it's > representation is composed of zeros no more makes it false than the point in > 3-space at (0, 0, 0) is false. It's still a completely normal time, which in > no way distinguishes itself from any other. > > I don't have a strong perspective on what the exact deprecation or breakage > cycle should like, but it's very clear to me that this behavior is nonsensical. > Out of a sample of half-a-dozen python programmers I've told about this, the > *universal* response was confusion and revulsion. I strongly suspect that the > majority of people writing ``if time_object`` have a bug in their code. > > Alex > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ I completely agree with Alex here. This is a ridiculous behavior and it should be fixed. I find the argument that it would only be fixed in 3.something and thus not be really fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python so by that logic we should simply give up on fixing anything in Python ever? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/1a0e9fc6/attachment.sig> From rosuav at gmail.com Wed Mar 5 22:31:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Mar 2014 08:31:47 +1100 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <20140305034945.GA67087@cskk.homeip.net> <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> <20140305114344.GX28804@ando> <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com> Message-ID: <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com> On Thu, Mar 6, 2014 at 8:20 AM, David Mertz <mertz at gnosis.cx> wrote: > The literal hardly saves you from injection attacks. I could write this too > under the proposed idea: > > foo = get_string_from_attacker() > a = $(foo) > b = a.eval() > > Now one can say "don't do that!" ... but that advice applies just as well to > 'compile(unsafe_string, ...)' That'll just be like doing: b = foo So it's still safe. That's the point. ChrisA From alexander.belopolsky at gmail.com Wed Mar 5 22:36:14 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 16:36:14 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <loom.20140305T220007-633@post.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> Message-ID: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> On Wed, Mar 5, 2014 at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote: > I strongly support fixing this issue. This behavior is so fundamentally > nonsensical that resolving it is worth the potential compatibility issues. > Does anyone know a language that (a) has a time-of-day class in stdlib; (b) allows instances of that class to be used in boolean context, and (c) has a midnight represented by an instance that is truthy? Here is my example of an entirely independent of Python language [1] that nevertheless has the same behavior for time-of-day objects: q)`boolean$00:00:00.000 0b q)`boolean$00:00:00.001 1b [1] http://code.kx.com/wiki/Reference/Datatypes -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bdea6e6b/attachment.html> From mertz at gnosis.cx Wed Mar 5 22:40:25 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 5 Mar 2014 13:40:25 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com> References: <CAEbHw4YH+O=_qM4egio4VyuCdnCTFqpFRU-WmdqVQ2hion7KvA@mail.gmail.com> <20140305034945.GA67087@cskk.homeip.net> <CAEbHw4aU44joWeMnHAQeg00A58B9yxpv6cdOUBSLNXZy4JSgug@mail.gmail.com> <20140305114344.GX28804@ando> <CAEbHw4YFisY-SxEEwMNbAQtAmTF+LqFmujsYYMJHNEfNrw2Dtg@mail.gmail.com> <CAPTjJmqG3qMQChUxC-6WnXDtUuk9A3pCKycfoDTvN3aw1JPt_g@mail.gmail.com> Message-ID: <CAEbHw4ZvObizopJTJ=anBqZ7rfPsQyaAJTZtXm6KHDYtcYAang@mail.gmail.com> On Wed, Mar 5, 2014 at 1:31 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Thu, Mar 6, 2014 at 8:20 AM, David Mertz <mertz at gnosis.cx> wrote: > > The literal hardly saves you from injection attacks. I could write this > too > > under the proposed idea: > > > > foo = get_string_from_attacker() > > a = $(foo) > > b = a.eval() > > > > Now one can say "don't do that!" ... but that advice applies just as > well to > > 'compile(unsafe_string, ...)' > > That'll just be like doing: > > b = foo > > So it's still safe. That's the point. > Doh! You are right. The literal does make it somewhat harder to shoot yourself in the foot with code injection, I had a thinko there. Still, advice in the docs not to do 'compile(untrusted_string, ...)' feels like it pretty much does what we actually need. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a1946eea/attachment-0001.html> From rosuav at gmail.com Wed Mar 5 22:50:06 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Mar 2014 08:50:06 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> Message-ID: <CAPTjJmoSvEw+cMuxhs5_Kqo6K2jrzQMZM3OcfaZG1dhmj9XzJw@mail.gmail.com> On Thu, Mar 6, 2014 at 8:36 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > On Wed, Mar 5, 2014 at 4:10 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote: >> >> I strongly support fixing this issue. This behavior is so fundamentally >> nonsensical that resolving it is worth the potential compatibility issues. > > > Does anyone know a language that (a) has a time-of-day class in stdlib; (b) > allows > instances of that class to be used in boolean context, and (c) has a > midnight > represented by an instance that is truthy? Pike fits all three of those, but (c) is trivially true: any object in Pike is truthy. Of course, you can choose to represent time-of-day with an integer, in which case an integer of 0 will be false; but if you use floating-point (for sub-second accuracy), then it's still safe, because any float is truthy (even 0.0 - and if you think that's illogical, consider that expecting 0.0 to be false is effectively making an implicit equality comparison, with all the issues of equality comparisons on floats _plus_ the issues of doing that implicitly). If Python didn't have to worry about backward compatibility (that is, if this were being implemented new now), then I'd say a null time range should be false, but any date, datetime, or time, regardless of what it represents, is true. Maybe there could be a "null date" value that isn't a date, but None should be able to do that. ChrisA From greg.ewing at canterbury.ac.nz Wed Mar 5 22:51:13 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 10:51:13 +1300 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> Message-ID: <53179C51.4020001@canterbury.ac.nz> Masklinn wrote: > On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > >>But Algol has the benefit of static typing > > That's not really a blocker though, Haskell thunks are implicit and not > type-encoded. A name may correspond to a (unforced) thunk or to a strict > value That comes at the expense of making *everything* a thunk until its value is needed. This would be a big, far-reaching change in the way Python works. Also, Haskell has the advantage of knowing that the value won't change, so it can replace the thunk with its value once it's been calculated. Python would have to keep it as a thunk and reevaluate it every time. And Haskell doesn't have to provide a way of passimg the thunk around *without* evaluating it. -- Greg From mal at egenix.com Wed Mar 5 23:04:05 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 05 Mar 2014 23:04:05 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> Message-ID: <53179F55.3080501@egenix.com> I don't know what all the fuzz is about. There are lots of types in Python that evaluate to False in certain corner cases, e.g. "", (), [], {}, 0, 0.0, 0j, None, etc. datetime.time(0,0,0) is just another one :-) BTW: If you want to test for None, well, then do test for None and not for some property of None: if x is None: print ("x is None, which implies it's False") print ("Oh, and do note that the inverse is not necessarily true :-)") In logic: A -> B does not imply B -> A; it only implies (not B) -> (not A). in Python again: if bool(x) is not False: print ("x is not None") -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 05 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From donald at stufft.io Wed Mar 5 23:10:28 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 17:10:28 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53179F55.3080501@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> Message-ID: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote: > I don't know what all the fuzz is about. There are lots of types > in Python that evaluate to False in certain corner cases, e.g. > "", (), [], {}, 0, 0.0, 0j, None, etc. > > datetime.time(0,0,0) is just another one :-) > > BTW: If you want to test for None, well, then do test for None and > not for some property of None: > > if x is None: > print ("x is None, which implies it's False") > print ("Oh, and do note that the inverse is not necessarily true :-)") > > In logic: > > A -> B does not imply B -> A; it only implies (not B) -> (not A). > > in Python again: > > if bool(x) is not False: > print ("x is not None") > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 05 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ Most of those are pretty easy to reason about being ?False?, they all have some concept of empty, nothing, etc. Midnight however is not any more or less a time than one minute past midnight. Even worse, if you consider not explicitly checking for None a bug in that application code, debugging that code is made much more difficult because of the fact that for some reason "midnight" is considered False. That is the real problem here, sure people should be using ``is None`` checks but until people are perfect and never write buggy code, we have to contend with the fact that sometimes people will not write ``is None``. Making that particular error harder to debug is doing nothing but wasting their time and having Python be punitive for what in many cases may be a mistake. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/b5fd660e/attachment.sig> From ben+python at benfinney.id.au Wed Mar 5 23:16:06 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 06 Mar 2014 09:16:06 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> Message-ID: <85siqwcmvt.fsf@benfinney.id.au> "M.-A. Lemburg" <mal at egenix.com> writes: > I don't know what all the fuzz is about. There are lots of types > in Python that evaluate to False in certain corner cases, e.g. > "", (), [], {}, 0, 0.0, 0j, None, etc. > > datetime.time(0,0,0) is just another one :-) The salient difference is: With the former set, they all embody concepts of either ?empty? or ?zero magnitude?. They strongly connote ?not there? within their domain of values, hence map well to Boolean false. With ?datetime.time? there is no equivalent: midnight is not special in the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point on a continuum, and is not a ?not there? value. It should not be Boolean false any more than any other time. -- \ ?What is it that makes a complete stranger dive into an icy | `\ river to save a solid gold baby? Maybe we'll never know.? ?Jack | _o__) Handey | Ben Finney From mal at egenix.com Wed Mar 5 23:27:07 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 05 Mar 2014 23:27:07 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> Message-ID: <5317A4BB.1090303@egenix.com> On 05.03.2014 23:10, Donald Stufft wrote: > > On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote: > >> I don't know what all the fuzz is about. There are lots of types >> in Python that evaluate to False in certain corner cases, e.g. >> "", (), [], {}, 0, 0.0, 0j, None, etc. >> >> datetime.time(0,0,0) is just another one :-) >> > Most of those are pretty easy to reason about being ?False?, they all have some > concept of empty, nothing, etc. Midnight however is not any more or less a time > than one minute past midnight. Even worse, if you consider not explicitly > checking for None a bug in that application code, debugging that code is made > much more difficult because of the fact that for some reason "midnight" is > considered False. The reasoning becomes clear if you regard a time value as number of seconds since midnight (the time of day is a relative measure). This is 0 for time(0,0,0). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 05 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From donald at stufft.io Wed Mar 5 23:28:08 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 17:28:08 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317A4BB.1090303@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> <5317A4BB.1090303@egenix.com> Message-ID: <D25CFB2C-3D48-4D4A-90FC-67E646AE5340@stufft.io> On Mar 5, 2014, at 5:27 PM, M.-A. Lemburg <mal at egenix.com> wrote: > On 05.03.2014 23:10, Donald Stufft wrote: >> >> On Mar 5, 2014, at 5:04 PM, M.-A. Lemburg <mal at egenix.com> wrote: >> >>> I don't know what all the fuzz is about. There are lots of types >>> in Python that evaluate to False in certain corner cases, e.g. >>> "", (), [], {}, 0, 0.0, 0j, None, etc. >>> >>> datetime.time(0,0,0) is just another one :-) >>> >> Most of those are pretty easy to reason about being ?False?, they all have some >> concept of empty, nothing, etc. Midnight however is not any more or less a time >> than one minute past midnight. Even worse, if you consider not explicitly >> checking for None a bug in that application code, debugging that code is made >> much more difficult because of the fact that for some reason "midnight" is >> considered False. > > The reasoning becomes clear if you regard a time value as > number of seconds since midnight (the time of day is a relative > measure). This is 0 for time(0,0,0). Except time isn?t number of seconds since midnight. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 05 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/0f874a63/attachment.sig> From mal at egenix.com Wed Mar 5 23:35:02 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 05 Mar 2014 23:35:02 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> Message-ID: <5317A696.8040605@egenix.com> On 05.03.2014 23:16, Ben Finney wrote: > "M.-A. Lemburg" <mal at egenix.com> writes: > >> I don't know what all the fuzz is about. There are lots of types >> in Python that evaluate to False in certain corner cases, e.g. >> "", (), [], {}, 0, 0.0, 0j, None, etc. >> >> datetime.time(0,0,0) is just another one :-) > > The salient difference is: > > With the former set, they all embody concepts of either ?empty? or ?zero > magnitude?. They strongly connote ?not there? within their domain of > values, hence map well to Boolean false. > > With ?datetime.time? there is no equivalent: midnight is not special in > the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point > on a continuum, and is not a ?not there? value. This is true in a date/time continuum, but not true if you regard time as "time of day". In the latter interpretation, the day starts at midnight, so the day is "not there" yet at midnight. > It should not be Boolean false any more than any other time. Just like with most interpretations, you can just as easily find one which implies the exact opposite. It's all smoke and mirrors anyway :-) Perhaps Tim just wanted to be able to make float(time(0,0,0)) return 0.0 seconds some day. That's what mxDateTime does for time values. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 05 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From greg.ewing at canterbury.ac.nz Wed Mar 5 23:36:09 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 11:36:09 +1300 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305120410.GY28804@ando> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> Message-ID: <5317A6D9.5040704@canterbury.ac.nz> Steven D'Aprano wrote: > (By the way, I think it is somewhat amusing that Python not only has a > built-in complex type, but also *syntax* for creating complex numbers, > but no built-in support for exact rationals.) I gather that Guido's experiences with ABC have led him to believe such a feature would be an attractive nuisance. Rationals have some nasty performance traps if you use them without being fully aware of what you're doing. -- Greg From donald at stufft.io Wed Mar 5 23:38:12 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 17:38:12 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317A696.8040605@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> Message-ID: <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote: > On 05.03.2014 23:16, Ben Finney wrote: >> "M.-A. Lemburg" <mal at egenix.com> writes: >> >>> I don't know what all the fuzz is about. There are lots of types >>> in Python that evaluate to False in certain corner cases, e.g. >>> "", (), [], {}, 0, 0.0, 0j, None, etc. >>> >>> datetime.time(0,0,0) is just another one :-) >> >> The salient difference is: >> >> With the former set, they all embody concepts of either ?empty? or ?zero >> magnitude?. They strongly connote ?not there? within their domain of >> values, hence map well to Boolean false. >> >> With ?datetime.time? there is no equivalent: midnight is not special in >> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point >> on a continuum, and is not a ?not there? value. > > This is true in a date/time continuum, but not true if you regard > time as "time of day". In the latter interpretation, the day starts > at midnight, so the day is "not there" yet at midnight. The day is absolutely here at midnight. Midnight isn?t some void where time ceases to exist. > >> It should not be Boolean false any more than any other time. > > Just like with most interpretations, you can just as easily find > one which implies the exact opposite. > > It's all smoke and mirrors anyway :-) > > Perhaps Tim just wanted to be able to make float(time(0,0,0)) return > 0.0 seconds some day. That's what mxDateTime does for time values. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 05 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/98b1c4db/attachment.sig> From alexander.belopolsky at gmail.com Wed Mar 5 23:38:53 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 17:38:53 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> Message-ID: <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com> On Wed, Mar 5, 2014 at 5:16 PM, Ben Finney <ben+python at benfinney.id.au>wrote: > .. midnight is not special in > the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point > on a continuum, .. > Tell that to people who gather in the Times Square on New Year Eve! It is very common to implement time-of-day in the languages that lack a dedicated class as seconds since midnight or fraction of a 24 hour day or any similar scheme. In all these schemes midnight is false in boolean context. Note that when datetime module was introduced, Python did not have a time class and people were using int or float to denote time-of-day. Compatibility with older schemes could be part of the original motivation. It also made more sense when the dunder method was spelled __nonzero__. Whatever the reason was - this is a well established feature and it won't change. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bcce54e8/attachment-0001.html> From ncoghlan at gmail.com Wed Mar 5 23:43:59 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 6 Mar 2014 08:43:59 +1000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> Message-ID: <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com> On 6 Mar 2014 08:11, "Donald Stufft" <donald at stufft.io> wrote: > > Most of those are pretty easy to reason about being "False", they all have some > concept of empty, nothing, etc. Midnight however is not any more or less a time > than one minute past midnight. Even worse, if you consider not explicitly > checking for None a bug in that application code, debugging that code is made > much more difficult because of the fact that for some reason "midnight" is > considered False. > > That is the real problem here, sure people should be using ``is None`` checks > but until people are perfect and never write buggy code, we have to contend > with the fact that sometimes people will not write ``is None``. Making that > particular error harder to debug is doing nothing but wasting their time and > having Python be punitive for what in many cases may be a mistake. Right, the core problem here is changing an often harmless style problem into a subtle behavioural bug. I changed the title on http://bugs.python.org/issue13936 accordingly, reopened it and described the appropriate next steps for anyone that is motivated enough about the issue to actually do the work. We'll see if the enthusiasm for changing the behaviour translates into anyone being sufficiently interested to work out the details of the deprecation warning and the associated documentation and test updates. Regards, Nick. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/47b44c79/attachment.html> From donald at stufft.io Wed Mar 5 23:45:34 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 17:45:34 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> <CADiSq7e15s47hBCOz6BBfs3kj411cyQ+MQSHDaxDND6EzS+ysg@mail.gmail.com> Message-ID: <6A78ACBF-1628-499D-9A31-F951DDF062D1@stufft.io> On Mar 5, 2014, at 5:43 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > > On 6 Mar 2014 08:11, "Donald Stufft" <donald at stufft.io> wrote: > > > > Most of those are pretty easy to reason about being ?False?, they all have some > > concept of empty, nothing, etc. Midnight however is not any more or less a time > > than one minute past midnight. Even worse, if you consider not explicitly > > checking for None a bug in that application code, debugging that code is made > > much more difficult because of the fact that for some reason "midnight" is > > considered False. > > > > That is the real problem here, sure people should be using ``is None`` checks > > but until people are perfect and never write buggy code, we have to contend > > with the fact that sometimes people will not write ``is None``. Making that > > particular error harder to debug is doing nothing but wasting their time and > > having Python be punitive for what in many cases may be a mistake. > > Right, the core problem here is changing an often harmless style problem into a subtle behavioural bug. I changed the title on http://bugs.python.org/issue13936 accordingly, reopened it and described the appropriate next steps for anyone that is motivated enough about the issue to actually do the work. > > We'll see if the enthusiasm for changing the behaviour translates into anyone being sufficiently interested to work out the details of the deprecation warning and the associated documentation and test updates. > > Regards, > Nick. > > > > > ----------------- > > Donald Stufft > > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ Thanks Nick :) ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/ac1f49fe/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/ac1f49fe/attachment.sig> From breamoreboy at yahoo.co.uk Wed Mar 5 23:47:45 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 22:47:45 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com> Message-ID: <lf89ib$sqs$1@ger.gmane.org> On 05/03/2014 22:38, Alexander Belopolsky wrote: > > On Wed, Mar 5, 2014 at 5:16 PM, Ben Finney <ben+python at benfinney.id.au > <mailto:ben+python at benfinney.id.au>> wrote: > > .. midnight is not special in > the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point > on a continuum, .. > > > Tell that to people who gather in the Times Square on New Year Eve! > > It is very common to implement time-of-day in the languages that lack a > dedicated class as seconds since midnight or fraction of a 24 hour day > or any similar scheme. In all these schemes midnight is false in > boolean context. > > Note that when datetime module was introduced, Python did not have a > time class and people were using int or float to denote time-of-day. > Compatibility with older schemes could be part of the original > motivation. It also made more sense when the dunder method was spelled > __nonzero__. > > Whatever the reason was - this is a well established feature and it > won't change. > Have you looked at the issue recently? :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From greg.ewing at canterbury.ac.nz Wed Mar 5 23:49:25 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 11:49:25 +1300 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> Message-ID: <5317A9F5.8050704@canterbury.ac.nz> Oscar Benjamin wrote: > In C and some other languages the f suffix indicates a numeric literal > that has type "float" e.g. "6.0f". You can use upper case there as > well but the convention is lower case and to include the .0 when it's > an integer. I just though that 3F looks sufficiently distinct from the > way it's typically done in C. I'm not sure it's different enough; I've been looking at Java code recently that uses uppercase F. An alternative would be to use 'r' or 'R' for 'rational', which would eliminate any chance of confusion. -- Greg From alexander.belopolsky at gmail.com Wed Mar 5 23:55:46 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 17:55:46 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf89ib$sqs$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com> <lf89ib$sqs$1@ger.gmane.org> Message-ID: <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com> On Wed, Mar 5, 2014 at 5:47 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote: > Have you looked at the issue recently? :) Yes, and I added one more myself: http://bugs.python.org/issue20855 let's see which one fares better. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/d8a1fee7/attachment.html> From breamoreboy at yahoo.co.uk Thu Mar 6 00:02:50 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 05 Mar 2014 23:02:50 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <CAP7h-xY1fK8VRL8_8zXVCY3+TuyLEFsbHsA-Q3TYE8VGrQdbgQ@mail.gmail.com> <lf89ib$sqs$1@ger.gmane.org> <CAP7h-xaq+LjUFj8spHnjWNdQHg531WN0fOLSHQ+Y1bWY1mqgcQ@mail.gmail.com> Message-ID: <lf8aek$51k$1@ger.gmane.org> On 05/03/2014 22:55, Alexander Belopolsky wrote: > > On Wed, Mar 5, 2014 at 5:47 PM, Mark Lawrence > <breamoreboy at yahoo.co.uk > <mailto:breamoreboy at yahoo.co.uk>> wrote: > > Have you looked at the issue recently? :) > > > Yes, and I added one more myself: > > http://bugs.python.org/issue20855 > > let's see which one fares better. :-) > As 20855 has already been closed, with a comment that I tend to agree with, I think it's time for me to grab my trenching tool, tin hat, camouflage net and flak jacket and start running before the Batley Townswomen Guild get involved. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From masklinn at masklinn.net Thu Mar 6 00:07:06 2014 From: masklinn at masklinn.net (Masklinn) Date: Thu, 6 Mar 2014 00:07:06 +0100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <53179C51.4020001@canterbury.ac.nz> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> <53179C51.4020001@canterbury.ac.nz> Message-ID: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Masklinn wrote: >> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >>> But Algol has the benefit of static typing >> That's not really a blocker though, Haskell thunks are implicit and not >> type-encoded. A name may correspond to a (unforced) thunk or to a strict >> value > > That comes at the expense of making *everything* a thunk > until its value is needed. Of course not, Haskell allows explicitly forcing a thunk and the compiler/runtime pair can use strictness analysis and not create thunks in the first place. > This would be a big, far-reaching > change in the way Python works. Having Haskell-type thunks does not mean using the same default as Haskell. Python would have the opposite one, instead of creating thunks by default it would create them only when requested. > Also, Haskell has the advantage of knowing that the value > won't change I do not see why that would be relevant to thunks. The result of the thunk may be mutable, so what? > so it can replace the thunk with its value > once it's been calculated. Python would have to keep it > as a thunk and reevaluate it every time. See above, I do not see why that would happen. Consider that a thunk is a deferral of an expression's evaluation, why would said expression's evaluation happen more than once? The only thing which changes is *when* the actual evaluation happens. > And Haskell doesn't have to provide a way of passimg the > thunk around *without* evaluating it. Well no, instead it takes the sensible option to force thunks when they need to be forced (or when their forcing is explicitly requested e.g. using ($!) or "bang pattern" extensions, but the need for those is mostly because haskell defaults to creating thunks I would say). Why would passing a reference around force a thunk in the first place? From mal at egenix.com Thu Mar 6 00:08:20 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 00:08:20 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> Message-ID: <5317AE64.90007@egenix.com> On 05.03.2014 23:38, Donald Stufft wrote: > > On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote: > >> On 05.03.2014 23:16, Ben Finney wrote: >>> "M.-A. Lemburg" <mal at egenix.com> writes: >>> >>>> I don't know what all the fuzz is about. There are lots of types >>>> in Python that evaluate to False in certain corner cases, e.g. >>>> "", (), [], {}, 0, 0.0, 0j, None, etc. >>>> >>>> datetime.time(0,0,0) is just another one :-) >>> >>> The salient difference is: >>> >>> With the former set, they all embody concepts of either ?empty? or ?zero >>> magnitude?. They strongly connote ?not there? within their domain of >>> values, hence map well to Boolean false. >>> >>> With ?datetime.time? there is no equivalent: midnight is not special in >>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point >>> on a continuum, and is not a ?not there? value. >> >> This is true in a date/time continuum, but not true if you regard >> time as "time of day". In the latter interpretation, the day starts >> at midnight, so the day is "not there" yet at midnight. > > The day is absolutely here at midnight. Midnight isn?t some void where > time ceases to exist. No, but midnight starts the day, just like 0 starts the positive real numbers (itself not being positive) :-) >>> It should not be Boolean false any more than any other time. >> >> Just like with most interpretations, you can just as easily find >> one which implies the exact opposite. >> >> It's all smoke and mirrors anyway :-) >> >> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return >> 0.0 seconds some day. That's what mxDateTime does for time values. >> >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Source (#1, Mar 05 2014) >>>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >> ________________________________________________________________________ >> 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go >> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: >> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >> Registered at Amtsgericht Duesseldorf: HRB 46611 >> http://www.egenix.com/company/contact/ >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 05 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From donald at stufft.io Thu Mar 6 00:10:12 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 18:10:12 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317AE64.90007@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> Message-ID: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> On Mar 5, 2014, at 6:08 PM, M.-A. Lemburg <mal at egenix.com> wrote: > On 05.03.2014 23:38, Donald Stufft wrote: >> >> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote: >> >>> On 05.03.2014 23:16, Ben Finney wrote: >>>> "M.-A. Lemburg" <mal at egenix.com> writes: >>>> >>>>> I don't know what all the fuzz is about. There are lots of types >>>>> in Python that evaluate to False in certain corner cases, e.g. >>>>> "", (), [], {}, 0, 0.0, 0j, None, etc. >>>>> >>>>> datetime.time(0,0,0) is just another one :-) >>>> >>>> The salient difference is: >>>> >>>> With the former set, they all embody concepts of either ?empty? or ?zero >>>> magnitude?. They strongly connote ?not there? within their domain of >>>> values, hence map well to Boolean false. >>>> >>>> With ?datetime.time? there is no equivalent: midnight is not special in >>>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point >>>> on a continuum, and is not a ?not there? value. >>> >>> This is true in a date/time continuum, but not true if you regard >>> time as "time of day". In the latter interpretation, the day starts >>> at midnight, so the day is "not there" yet at midnight. >> >> The day is absolutely here at midnight. Midnight isn?t some void where >> time ceases to exist. > > No, but midnight starts the day, just like 0 starts the positive > real numbers (itself not being positive) :-) And yet 0 is False not because it starts the positive real numbers (not sure what that has to do with a value in an integer type) but because it signifies an empty or zero magnitude object. > >>>> It should not be Boolean false any more than any other time. >>> >>> Just like with most interpretations, you can just as easily find >>> one which implies the exact opposite. >>> >>> It's all smoke and mirrors anyway :-) >>> >>> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return >>> 0.0 seconds some day. That's what mxDateTime does for time values. >>> >>> -- >>> Marc-Andre Lemburg >>> eGenix.com >>> >>> Professional Python Services directly from the Source (#1, Mar 05 2014) >>>>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >>> ________________________________________________________________________ >>> 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go >>> >>> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: >>> >>> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >>> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >>> Registered at Amtsgericht Duesseldorf: HRB 46611 >>> http://www.egenix.com/company/contact/ >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA >> > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 05 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/50b4e5bf/attachment-0001.sig> From tjreedy at udel.edu Thu Mar 6 00:20:45 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 05 Mar 2014 18:20:45 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <loom.20140305T220007-633@post.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> Message-ID: <lf8bgc$lg2$1@ger.gmane.org> On 3/5/2014 4:10 PM, Alex Gaynor wrote: > First, I'd like to dispense with the notion that ``if foo`` is somehow a bad > practice. It is when one means 'if foo is not None' as it is as fragile, and in the same way, in the same way as using 'a and b or c' as a conditional expression. > This is an EXTREMELY common practice as a shorthand for checking for > None, a great many people consider it more idiomatic, In what universe. I was originally for changing datetime, but seems the issue being used as a vehicle to justify and promote buggy programming has changed my mind. -- Terry Jan Reedy From abarnert at yahoo.com Thu Mar 6 00:20:12 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 15:20:12 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <20140305153529.GC28804@ando> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> <20140305153529.GC28804@ando> Message-ID: <AC1CFBD0-FD40-4005-B120-FA6CC425C711@yahoo.com> On Mar 5, 2014, at 7:35, Steven D'Aprano <steve at pearwood.info> wrote: > So as you can see, what I've been calling a thunk is not precisely like > Algol thunks. One thing which is missing is the dynamic scoping: in the > thunk evaluation, the x comes from the caller's scope. But the delayed > evaluation part is quite similar, enough that I think the name may be > appropriate. Another name for this might be a "promise", as in the > promise to execute a computation later. As I mentioned in one of the previous threads, Alice does lazy evaluation with futures and promises--the same construct it uses for concurrent scheduling, closely related to what Python has in concurrent.futures (Python merges futures and promises into a single object with two different APIs, but the idea is the same). If `expr` evaluated to an object with the same API as a concurrent.futures.Future, that's not a terrible form of explicit evaluation. But if it's doing something like dynamic scoping, it might look pretty misleading. From mal at egenix.com Thu Mar 6 00:22:38 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 00:22:38 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> Message-ID: <5317B1BE.4090804@egenix.com> On 06.03.2014 00:10, Donald Stufft wrote: > > On Mar 5, 2014, at 6:08 PM, M.-A. Lemburg <mal at egenix.com> wrote: > >> On 05.03.2014 23:38, Donald Stufft wrote: >>> >>> On Mar 5, 2014, at 5:35 PM, M.-A. Lemburg <mal at egenix.com> wrote: >>> >>>> On 05.03.2014 23:16, Ben Finney wrote: >>>>> "M.-A. Lemburg" <mal at egenix.com> writes: >>>>> >>>>>> I don't know what all the fuzz is about. There are lots of types >>>>>> in Python that evaluate to False in certain corner cases, e.g. >>>>>> "", (), [], {}, 0, 0.0, 0j, None, etc. >>>>>> >>>>>> datetime.time(0,0,0) is just another one :-) >>>>> >>>>> The salient difference is: >>>>> >>>>> With the former set, they all embody concepts of either ?empty? or ?zero >>>>> magnitude?. They strongly connote ?not there? within their domain of >>>>> values, hence map well to Boolean false. >>>>> >>>>> With ?datetime.time? there is no equivalent: midnight is not special in >>>>> the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point >>>>> on a continuum, and is not a ?not there? value. >>>> >>>> This is true in a date/time continuum, but not true if you regard >>>> time as "time of day". In the latter interpretation, the day starts >>>> at midnight, so the day is "not there" yet at midnight. >>> >>> The day is absolutely here at midnight. Midnight isn?t some void where >>> time ceases to exist. >> >> No, but midnight starts the day, just like 0 starts the positive >> real numbers (itself not being positive) :-) > > And yet 0 is False not because it starts the positive real numbers (not sure > what that has to do with a value in an integer type) but because it signifies > an empty or zero magnitude object. Well, the duration between the start of day and midnight is zero and returning to the interpretation of time of day being a measurement relative to midnight, this makes time(0,0,0) a zero magnitude object. q.e.d. Anyway, it's all up in the air for interpretation :-) There are reasons to have time(0,0,0) be False just as there are reasons for it not to be False. There's no clear answer. >>>>> It should not be Boolean false any more than any other time. >>>> >>>> Just like with most interpretations, you can just as easily find >>>> one which implies the exact opposite. >>>> >>>> It's all smoke and mirrors anyway :-) >>>> >>>> Perhaps Tim just wanted to be able to make float(time(0,0,0)) return >>>> 0.0 seconds some day. That's what mxDateTime does for time values. >>>> >>>> -- >>>> Marc-Andre Lemburg >>>> eGenix.com >>>> >>>> Professional Python Services directly from the Source (#1, Mar 05 2014) >>>>>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>>>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >>>> ________________________________________________________________________ >>>> 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go >>>> >>>> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: >>>> >>>> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >>>> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >>>> Registered at Amtsgericht Duesseldorf: HRB 46611 >>>> http://www.egenix.com/company/contact/ >>>> _______________________________________________ >>>> Python-ideas mailing list >>>> Python-ideas at python.org >>>> https://mail.python.org/mailman/listinfo/python-ideas >>>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >>> >>> ----------------- >>> Donald Stufft >>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA >>> >> >> -- >> Marc-Andre Lemburg >> eGenix.com >> >> Professional Python Services directly from the Source (#1, Mar 05 2014) >>>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ >> ________________________________________________________________________ >> 2014-04-09: PyCon 2014, Montreal, Canada ... 35 days to go >> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: >> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg >> Registered at Amtsgericht Duesseldorf: HRB 46611 >> http://www.egenix.com/company/contact/ > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From alexander.belopolsky at gmail.com Thu Mar 6 00:22:54 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 18:22:54 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> Message-ID: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> On Wed, Mar 5, 2014 at 6:10 PM, Donald Stufft <donald at stufft.io> wrote: > And yet 0 is False not because it starts the positive real numbers (not > sure > what that has to do with a value in an integer type) but because it > signifies > an empty or zero magnitude object. > No. It is False because it equals False: >>> 0 == False True Numbers are not containers. You cannot tell whether a number is empty or not. Python interpretation of numbers in boolean context is different from that of containers. I think the disagreement stems from different view on what time-of-day is. For me, it is the fraction of the day that passed or distance from midnight and in any case fundamentally some kind of number expressed in a Babylonian base-60 notation. I think proponents of bool(time(0)) == True view it as a container of hours, minutes and seconds. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/97f4cb74/attachment.html> From greg.ewing at canterbury.ac.nz Thu Mar 6 00:23:03 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 12:23:03 +1300 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> Message-ID: <5317B1D7.4090305@canterbury.ac.nz> Carl Smith wrote: > I was thinking of a short word which describes an > expression who's evaluation changes depending on the local > circumstances. Like a legal bill, not a dollar bill. Oh. Well, I must say that this was very far from being the first interpretation of the word "bill" that sprang to mind when I saw it. I wasn't even aware of this meaning of the term "legal bill" (although admittedly I'm not used to living in a country where the laws vary wildly from one place to another). -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 00:23:15 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 12:23:15 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <201403051259.40289.shai@platonix.com> <CAHVvXxSB768k7se+8rX9XnZGySO5Dc8SQ+3OFjkb4VrUdaaqGg@mail.gmail.com> Message-ID: <5317B1E3.9020409@canterbury.ac.nz> Oscar Benjamin wrote: > The question is surely whether the issue is worth a backwards > compatibility break not whether the current behaviour is a good idea > (it clearly isn't). The currrent behaviour is sufficiently useless that it seems unlikely there would be much code out there relying on it. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 00:23:28 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 12:23:28 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> Message-ID: <5317B1F0.3070408@canterbury.ac.nz> Paul Moore wrote: > Agreed that the odd behaviour of time in a > boolean context is why this whole class of bugs exists, but it's only > a subclass if the wider problem that people shouldn't truth-test > values without thinking But this bites you even if you *do* think, because there is no reason to suspect that any time value should be regarded as false, especially since the docs make no mention of this. Note that "fix the docs" is *not* the right response to this, IMO -- that would just be enshrining a bad design decision. Better to fix the design while we have the chance. -- Greg From oscar.j.benjamin at gmail.com Thu Mar 6 00:24:46 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 5 Mar 2014 23:24:46 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <5317A6D9.5040704@canterbury.ac.nz> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <5317A6D9.5040704@canterbury.ac.nz> Message-ID: <CAHVvXxTBb1cavd6ipgRDAy7AogSsqyeE1wY-aEQ5saUi17QDBw@mail.gmail.com> On 5 March 2014 22:36, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Steven D'Aprano wrote: >> >> (By the way, I think it is somewhat amusing that Python not only has a >> built-in complex type, but also *syntax* for creating complex numbers, but >> no built-in support for exact rationals.) > > I gather that Guido's experiences with ABC have led him > to believe such a feature would be an attractive nuisance. > Rationals have some nasty performance traps if you use > them without being fully aware of what you're doing. I've read statement's to that effect but they were specifically about having rational as the *default* mode for integer division which I am not suggesting. It's already possible to opt-in to using Fractions so that rationals are used for division. Currently you have to import a module and write F(1, 7) or F('1/7') everywhere in your code. If it were possible to write 1/7F or 1/7r or whatever then I've personally written code where I would have used that so that it would be easier to read, would look more natural with syntax highlighting etc. Oscar From donald at stufft.io Thu Mar 6 00:28:12 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 18:28:12 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317B1F0.3070408@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <5317B1F0.3070408@canterbury.ac.nz> Message-ID: <39FD7C84-C7AB-47EA-9A9C-9216DAE5D286@stufft.io> On Mar 5, 2014, at 6:23 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Paul Moore wrote: >> Agreed that the odd behaviour of time in a >> boolean context is why this whole class of bugs exists, but it's only >> a subclass if the wider problem that people shouldn't truth-test >> values without thinking > > But this bites you even if you *do* think, because there > is no reason to suspect that any time value should be > regarded as false, especially since the docs make no > mention of this. > > Note that "fix the docs" is *not* the right response to > this, IMO -- that would just be enshrining a bad design > decision. Better to fix the design while we have the > chance. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ The docs do mention it: > in Boolean contexts, a time object is considered to be true if and only > if, after converting it to minutes and subtracting utcoffset() (or 0 if that?s > None), the result is non-zero. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/937e362f/attachment.sig> From greg.ewing at canterbury.ac.nz Thu Mar 6 00:33:49 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 12:33:49 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> Message-ID: <5317B45D.6000606@canterbury.ac.nz> Paul Moore wrote: > """in Boolean contexts, a time object is considered to be true if and > only if, after converting it to minutes and subtracting utcoffset() > (or 0 if that's None), the result is non-zero.""" Aaaaargh... I didn't know this had since been documented. Wy opinion still stands -- documenting this was the wrong thing to do. I hope it can still be fixed. -- Greg From rob.cliffe at btinternet.com Thu Mar 6 01:21:36 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 06 Mar 2014 00:21:36 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> Message-ID: <5317BF90.9030901@btinternet.com> On 05/03/2014 17:27, Alexander Belopolsky wrote: > > On Wed, Mar 5, 2014 at 11:38 AM, Shai Berger <shai at platonix.com > <mailto:shai at platonix.com>> wrote: > > So, I rephrase my request: Would anybody object to a silent > warning issued > whenever a time-object is evaluated as Boolean? > > > I would. Users should not be penalized for using a documented behavior. We hope that the Python documentation describes the semantics of the language and packages as completely as is reasonably possible. So you are effectively saying that no backwards-incompatible change should ever be made. If we accepted that, this discussion would end now. > There are legitimate uses for bool(midnight) being False. Midnight > is special in many contexts. For example, it is uncertain whether > midnight belongs to the previous or next day. If your application > wants to group midnight differently from other times - it is perfectly > fine to use "if dt.time()" instead of a more verbose "if dt.time() != > datetime.time(0, 0)". > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > No virus found in this message. > Checked by AVG - www.avg.com <http://www.avg.com> > Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14 > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/32cc4221/attachment-0001.html> From harrismh777 at gmail.com Thu Mar 6 01:40:17 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Wed, 5 Mar 2014 16:40:17 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lf841b$rt6$1@ger.gmane.org> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <1394051558.18502.YahooMailNeo@web181005.mail.ne1.yahoo.com> <lf841b$rt6$1@ger.gmane.org> Message-ID: <a91be33d-6fd9-49c8-8d61-b5d2cdcf2ed4@googlegroups.com> On Wednesday, March 5, 2014 3:13:29 PM UTC-6, Ron Adam wrote: > > It's also a matter of how far in the future you are looking. If he waited > > until later to propose something like this, it most likely wouldn't get in. > > I'm not sure he's expecting a firm answer now, but probably is hoping > for > > a "hey lets look into this more and see if it's really possible" kind of > maybe. For that, it's good to start early. > hi Ron, yes, correct. Something like this would require huge effort, determined strategy, and possibly years of planning. Its not going to happen over-night, but in stages could be brought to fruition given some thought and a lot of dialogue. kind regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/053784ca/attachment.html> From harrismh777 at gmail.com Thu Mar 6 01:30:58 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Wed, 5 Mar 2014 16:30:58 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <28d03184-4934-4840-8f92-082901ae0c30@googlegroups.com> On Wednesday, March 5, 2014 2:10:18 AM UTC-6, Andrew Barnert wrote: The types you dislike are not programming concepts that get in the way of > human mathematics, they are human mathematical concepts. The integers, the > rationals, and the reals all behave differently. And they're not the only > types of numbers?Python handles complex numbers natively, and it's very > easy to extend it with, say, quaternions or even arbitrary matrices that > act like numbers. > hi Andrew, your response reminds me of that scene from 'Star Trek: First Contact,' when Data was getting a little lippy with her eminence The Borg (the one who is many) (he was trying to explain to her why she doesn't exist, since her ship was destroyed just after coming through the chronometric particle vortex--time travel--) and she responded, "You think so three dimensionally... " When asked about the Borg hierarchy she stated simply, "You imply disparity where none exists..." Now, that was one hot cybernetic babe. My point is philosophical, really. I do not dislike numerical types. I simply do not believe they are necessary. In fact, Rexx (in a simple way) and other languages too for that matter, have demonstrated that types are superficial paradigms in symbolic thought that can quite readily be eliminated for most AI applications given enough memory, and enough time. Fortunately, memory is becoming more plentiful (less expensive) and time is shrinking away through processor speed, pipe-lining, and software engineering like decimal.Decimal. Keep in mind that I am NOT proposing the elimination of numbers, I am proposing the abstraction of *python numbers (yes, all of them...) *in such a way that programmers and specialists, and accountants and grade school teachers may use Python without having to be computer scientists (all of the work happens under the covers, written by someone like me, or you) and everything at the top is simple and human readable; for them. Yes, its work. And, its doable. A *number *should be no more convoluted in a modern age of symbol set processing than any other abstract idea (just because of arbitrary limits and abstraction paradigms). Unifying *python numbers *requires a high order paradigm shift for sure; but the concept is absolutely doable. kind regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/a917adbd/attachment.html> From alexander.belopolsky at gmail.com Thu Mar 6 01:52:53 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 19:52:53 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317BF90.9030901@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> Message-ID: <CAP7h-xZhNaU-8YqVPe2EhRZXFTejZqNK7xN5G=xoiUv_5FO_Ng@mail.gmail.com> On Wed, Mar 5, 2014 at 7:21 PM, Rob Cliffe <rob.cliffe at btinternet.com>wrote: > So you are effectively saying that no backwards-incompatible change should > ever be made. > If we accepted that, this discussion would end now. > This was the rule for the datetime module for many years and we were able to put in many nice enhancements in a fully backward compatible way. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/dd2c5ffc/attachment.html> From harrismh777 at gmail.com Thu Mar 6 02:06:38 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Wed, 5 Mar 2014 17:06:38 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305135646.GB28804@ando> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> Message-ID: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> On Wednesday, March 5, 2014 7:56:46 AM UTC-6, Steven D'Aprano wrote: > > On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote: > > > The idea of *python number* means that there are no types, no limits, > no > > constraints, and that all *python numbers *are dynamic. The upper level > > syntax is also very simple; all* python numbers *are simply human. > > What makes this a "python number"? In what way are they "dynamic"? > > At this point in the discussion please try not to think "implementation," rather think conceptually in an open framework where anything is possible. Dynamic in terms of *python numbers *is not that far off from the concept of dynamic name binding used everywhere else in python. In terms of unifying numbers as *python numbers *which needs to be defined at some point, the idea of dynamic binding means first that numeric symbol sets have no type, and are not 'bound' until run-time, if at all. Consider: What is this? ===> PI / 2 Is it two numbers with a divisor,? or a Decimal and an int with a divisor,? or a *python number ?* Or, is it a string of characters that are readily recognizable to humans that means 'the top of the unit circle," or half_of_PI whatever I mean by that, or '1.57079632679489661923132169163975144209858469968755291048747229615390820314' Dynamic means the symbols sets are bound (name-wise, abstraction-wise) only at run-time, and with some 'smarts' as it were (AI) so that the right things happen under the covers (not magic) because the machine has been taught to think, as it were, about numbers and symbols the way normal humans do (in the kitchen, at high school, at the university, in science and business). Another way to think of this Steven is to think of 'pretty print' on the TI 89 Platinum edition graphical programming calculator (my personal favorite). Anyone who plays with it the first time is frustrated because *numbers *are symbolized instead of printed out in strings of numerals. So if I enter {1} [/] {3} and press enter, I get 1/3 <==== Yes, I can press evaluate and get .33333333333333333 but 1/3 is just as good a number as any other number (and what type is it???) PI / 2 is another example. That's a great number and I don't really need to see a numeric list of digits to use it, do I... and what is its type??? who cares. Another very simple (and not quite good enough) example is the pdeclib file I just created over the past couple of days. I have a need within the module to use PI or half_PI for some method or function (but I don't need the user to know that, and I don't want to calculate it over and over, and I want it within accuracy for the context on demand)... so what? I created __gpi__ global PI. It is reset to Decimal(0) if the dscale(n) precision context changes. Then, and function that requires PI of half_PI can check first to see if it is cached as __gpi__ and if so pull it back, or if not calculate it. If the context does not change than I half some whopping value of PI for the duration of the context (but, and this is the big but, if I don't need it --- then it never gets bound). Binding for *python numbers * must be dynamic, typeless, limitless, and no constraints. If I enter a formula of symbols into python (whatever I mean by that) python just handles it dynamically just like the TI 89--- if I want explicit control of the processing, I am free to ask for that too, just like the TI 89. By the by, complex numbers are no different what-so-ever, nor fractions either, nor any other *number.* The system needs to be able to parse symbol sets intelligently (Alonzo Church, Alan Turning, and John McCarthy dreamed of this, but had no way to accomplish it in their day) and then dynamically bind the naming / representation to the abstraction (or implementation) that makes sense for that symbol set in 'that' context, for 'that' problem. This is not simply business as usual... it requires AI, it requires a severe paradigm shift of a very high order. Kind regards, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/8a485596/attachment-0001.html> From python at mrabarnett.plus.com Thu Mar 6 02:27:45 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 06 Mar 2014 01:27:45 +0000 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <5317B1D7.4090305@canterbury.ac.nz> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> <5317B1D7.4090305@canterbury.ac.nz> Message-ID: <5317CF11.20100@mrabarnett.plus.com> On 2014-03-05 23:23, Greg Ewing wrote: > Carl Smith wrote: >> I was thinking of a short word which describes an >> expression who's evaluation changes depending on the local >> circumstances. Like a legal bill, not a dollar bill. > > Oh. Well, I must say that this was very far from being > the first interpretation of the word "bill" that sprang > to mind when I saw it. > > I wasn't even aware of this meaning of the term "legal > bill" (although admittedly I'm not used to living in a > country where the laws vary wildly from one place > to another). > In the UK we talk about "Bills" being put before Parliament. It reminds me of that quotation about the UK and US: ?Two nations divided by a common language?. In the UK you can pay a bill using a cheque; in the US you can pay the check using bills (but a "check" is also a (UK) cheque!). From tim.peters at gmail.com Thu Mar 6 02:34:13 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 19:34:13 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317BF90.9030901@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> Message-ID: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> [Rob Cliffe <rob.cliffe at btinternet.com>] > We hope that the Python documentation describes the semantics of the > language and packages as completely as is reasonably possible. > So you are effectively saying that no backwards-incompatible change should > ever be made. > If we accepted that, this discussion would end now. There should be a very high bias against changing documented behavior that is in fact working as documented. The document is a contract with users. Contracts shouldn't be violated without truly compelling cause. "I am altering the deal. Pray I don't alter it any further." There's a reason Darth Vader wasn't proclaimed Python's BDFL ;-) From tim.peters at gmail.com Thu Mar 6 02:37:37 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 19:37:37 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317B45D.6000606@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> <5317B45D.6000606@canterbury.ac.nz> Message-ID: <CAExdVNmPK8f5vQm7p3RwCXWK0CutaSVc2Bdi+_M2Pv9Jzv749g@mail.gmail.com> [Paul Moore] >> """in Boolean contexts, a time object is considered to be true if and >> only if, after converting it to minutes and subtracting utcoffset() >> (or 0 if that's None), the result is non-zero.""" [Greg Ewing] > Aaaaargh... I didn't know this had since been documented. > > Wy opinion still stands -- documenting this was the > wrong thing to do. I hope it can still be fixed. The quoted text has always been there, from datetime's initial release: http://docs.python.org/2.3/lib/datetime-time.html That was a bit over 9 years ago. From donald at stufft.io Thu Mar 6 02:38:41 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 20:38:41 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> Message-ID: <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> On Mar 5, 2014, at 8:34 PM, Tim Peters <tim.peters at gmail.com> wrote: > [Rob Cliffe <rob.cliffe at btinternet.com>] >> We hope that the Python documentation describes the semantics of the >> language and packages as completely as is reasonably possible. >> So you are effectively saying that no backwards-incompatible change should >> ever be made. >> If we accepted that, this discussion would end now. > > There should be a very high bias against changing documented behavior > that is in fact working as documented. The document is a contract > with users. Contracts shouldn't be violated without truly compelling > cause. > > "I am altering the deal. Pray I don't alter it any further." There's > a reason Darth Vader wasn't proclaimed Python's BDFL ;-) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ When the documented behavior is both nonsensical and the cause of hard to debug bugs that is a pretty compelling use case to me, unless you actively enjoy being user hostile. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/c79bfc98/attachment.sig> From tim.peters at gmail.com Thu Mar 6 02:53:02 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 19:53:02 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> Message-ID: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> [Bruce Leban] > And the documentation is not exactly clear for understanding when a time > object is falsey. In particular, the "converting it to minutes" part seems > irrelevant That's there because `utcoffset()` is defined to return minutes. A rule is needed to specify how the naive time object and `utcoffset()` interact to define the result. > as a time of 1 microsecond from midnight is truthy and timedeltas > are not numbers and are not in minutes timedeltas have nothing to do with this. The docs could be expanded to specify precisely how the naive time object is converted to minutes, but I think that's so obvious it would just be tedious to do so. > Perhaps a reasonable improvement would be to change this to: > > a time object is considered to be true unless it represents exactly midnight > local time, that is, it's false if subtracting the utcoffset() from the time > produces an all-zero result (or it's all-zero to start with if the utcoffset > is None). You still need a rule to define _how_ "subtracting the utcoffset() from the time" is intended to work, since arithmetic on naive time objects is not defined. Add "convert it to minutes" ;-) From tim.peters at gmail.com Thu Mar 6 03:08:56 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 20:08:56 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317A696.8040605@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> Message-ID: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> [M.-A. Lemburg] > ... > Perhaps Tim just wanted to be able to make float(time(0,0,0)) return > 0.0 seconds some day. That's what mxDateTime does for time values. Now you're getting close :-) There was an endless list of features various people said they wanted in a datetime module. At the time, we picked a large subset we could do in time, and released it. There was every expectation that more features may be added later. Among them were various kinds of conversions and various kinds of arithmetic enhancements. For example, modular arithmetic on time objects was a possibility. In that case, time(0, 0, 0) would become "a zero value" just as much as 0 or 0.0 or 0j or 0x00 are zero values. But if it weren't _treated_ as "a zero value" from the start, backward compatibility would prevent it from being treated as a zero value for the first time later. So, what the heck - like all other zero/empty objects, time(0, 0, 0) was fiddled to evaluate to False in a Boolean context, AND (something nobody seems to have noticed yet) plain time() was fiddled to return it: >>> bool(int()) is bool(complex()) is bool(float()) is bool(datetime.time()) True People can say that "midnight" isn't a compelling "sentinel value", and I agree about the "compelling" part, but in all implementation respects it does _act_ as "a sentinel value". If people don't want to use it as one, fine, that's their choice. I've used it as one, and it works fine for that purpose. As things turned out, nobody seemed to care much about modular (are any other kind of) arithmetic on time objects after all. Nevertheless, we went about 9 years before anyone complained about the code added to support the _possibility_ for doing that cleanly. The "seconds from midnight" view was also in play, with a view toward clean conversions to float-ish types. From alexander.belopolsky at gmail.com Thu Mar 6 03:12:17 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 21:12:17 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> Message-ID: <CAP7h-xbCuUKhsuJ4szik3FW3pONK2ew3CLqwqtLbi_dwW=pTfQ@mail.gmail.com> On Wed, Mar 5, 2014 at 8:53 PM, Tim Peters <tim.peters at gmail.com> wrote: > > That's there because `utcoffset()` is defined to return minutes. More accurately, it returns "a timedelta object representing a whole number of minutes": time.utcoffset() If tzinfo is None, returns None, else returns self.tzinfo.utcoffset(None), and raises an exception if the latter doesn?t return None or a timedelta object representing a whole number of minutes with magnitude less than one day. http://docs.python.org/3/library/datetime.html#datetime.time.utcoffset -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e9b28b0c/attachment.html> From steve at pearwood.info Thu Mar 6 03:14:57 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 13:14:57 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> Message-ID: <20140306021457.GF28804@ando> On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote: > I find the argument that it would only be fixed in 3.something and thus not be really > fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python > so by that logic we should simply give up on fixing anything in Python ever? Not at all. But it does mean that the benefit of fixing it needs to be worth the wait. Suppose we said "Sure, we'll fix it, but you have to pay for the fix." How much would you be willing to pay to have it fixed? ("Pay" doesn't have to mean money. It can also mean, how much pain or inconvenience are you willing to experience, or how much time and effort are you willing to out into this.) If I said, "Oh by the way, although you have to pay right now, you won't see any benefit until 2024", now how much are you willing to pay? Design flaws are more costly to fix than outright bugs, not because they take more time and effort to change, but because you're changing behaviour that *somebody out there is relying on*. What you consider a terrible misfeature is *somebody's* critical feature, and you're proposing that we take that away. That's okay, we can do that after a suitable deprecation period. (You wouldn't want us to break your code, so you ought to extend the same courtesy to other people.) But even if there isn't anyone relying on this, and we fix it as soon as possible (which will be 3.5), your library *still* can't rely on it until it drops all support for versions below 3.5. Since you can't rely on that feature being present, you have to write your library code as if it wasn't present. So there is *absolutely no difference* between your library with this (alleged) bug being fixed, or it not being fixed. The situation for application developers is a bit different. An application developer can simply say, I'm going to immediately migrate to 3.5 to take advantage of this new feature/bug fix. That's okay too. The job of the core developers is to balance all these competing interests: the library developers, the application developers, the users who don't care one way or another but do care about some other bug that's not being fixed because people are arguing about this one, the people (actual or hypothetical) who actually like this (mis)feature the way it is, people who don't like it but have written code that relies on it, and, yes, even the core developers themselves, who are perfectly entitled to say that the effort in fixing this is greater than the benefit so they're not going to do it. If you haven't already read Nick's blog posts on these issues, you should: http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html For what it's worth, I think the current behaviour is a misfeature, and in a perfect world it would be fixed. But the cost of this is so miniscule, and the work-around so trivial, that although the fix is cheap, the benefit is even lower. I've only spent a few minutes reading and responding to this thread. Those few minutes are already worth far more than any benefit to me in fixing this. That's *my* personal opinion on this, others may differ, so I'm a +0 on deprecating the behaviour in 3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately in 3.5. -- Steven From tim.peters at gmail.com Thu Mar 6 03:15:59 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 20:15:59 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> Message-ID: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> [Donald Stufft] > When the documented behavior is both nonsensical and the cause of hard > to debug bugs that is a pretty compelling use case to me, unless you actively > enjoy being user hostile. Breaking code that currently works is as hostile as hostile gets. Where is the evidence that no code relies on the current behavior? For that matter, where is the evidence that the current behavior is a significant cause of bugs? I know I've used "if some_time_object:", but it did exactly what I expected it to do. From steve at pearwood.info Thu Mar 6 03:19:48 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 13:19:48 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <85siqwcmvt.fsf@benfinney.id.au> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> Message-ID: <20140306021948.GG28804@ando> On Thu, Mar 06, 2014 at 09:16:06AM +1100, Ben Finney wrote: > With ?datetime.time? there is no equivalent: midnight is not special in > the way ?empty? or ?zero magnitude? are. Midnight is an arbitrary point > on a continuum, and is not a ?not there? value. It should not be Boolean > false any more than any other time. It may only be a convention that an instant before midnight is the end of the day and midnight the beginning of the next, but it is a convention: midnight is the origin (i.e. zero point) of the day. That makes it arguably as falsey as 0, 0.0 and 0j. -- Steven From donald at stufft.io Thu Mar 6 03:23:37 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 21:23:37 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> Message-ID: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: > [Donald Stufft] >> When the documented behavior is both nonsensical and the cause of hard >> to debug bugs that is a pretty compelling use case to me, unless you actively >> enjoy being user hostile. > > Breaking code that currently works is as hostile as hostile gets. > Where is the evidence that no code relies on the current behavior? > For that matter, where is the evidence that the current behavior is a > significant cause of bugs? I know I've used "if some_time_object:", > but it did exactly what I expected it to do. Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s implementation. It?s hard to do any sort of search for this, however in an informal poll where I?ve shown people this code not a single person thought it made sense, and most of them responded with ?wtf??. I?ve also seen first hand through Django (which is why the person who started this thread) get caught by this bug and have to spend hours trying to figure out why it?s behaving that way. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/6b3e8888/attachment.sig> From donald at stufft.io Thu Mar 6 03:33:24 2014 From: donald at stufft.io (Donald Stufft) Date: Wed, 5 Mar 2014 21:33:24 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306021457.GF28804@ando> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> <20140306021457.GF28804@ando> Message-ID: <18676193-CEEA-40E2-B996-ECDCDB01A365@stufft.io> On Mar 5, 2014, at 9:14 PM, Steven D'Aprano <steve at pearwood.info> wrote: > On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote: > >> I find the argument that it would only be fixed in 3.something and thus not be really >> fixed for X years disingenuous at best. That fact is true for *ALL* bug fixes in Python >> so by that logic we should simply give up on fixing anything in Python ever? > > Not at all. But it does mean that the benefit of fixing it needs to be > worth the wait. Suppose we said "Sure, we'll fix it, but you have > to pay for the fix." How much would you be willing to pay to have it > fixed? > > ("Pay" doesn't have to mean money. It can also mean, how much pain or > inconvenience are you willing to experience, or how much time and effort > are you willing to out into this.) > > If I said, "Oh by the way, although you have to pay right now, you won't > see any benefit until 2024", now how much are you willing to pay? Well I?ve offered to not only write the patch (assuming that someone else doesn?t do it before me since my time is limited), but also shepherd it through review and commit it myself. So I?m willing to pay with my free time which is a very limited resource :) In general I agree that everything has to go through a cost/benefit analysis, and in this case the cost and benefit are both fairly low in the grand scheme of things but the benefit does outweigh the cost as far as I?m concerned because the workaround requires you to know that the workaround is even required at all. > > Design flaws are more costly to fix than outright bugs, not because they > take more time and effort to change, but because you're changing > behaviour that *somebody out there is relying on*. What you consider a > terrible misfeature is *somebody's* critical feature, and you're > proposing that we take that away. That's okay, we can do that after a > suitable deprecation period. (You wouldn't want us to break your code, > so you ought to extend the same courtesy to other people.) It?s not obvious to me that design flaws are more costly to fix because people rely on even buggy behavior :) I don?t think i?ve advocated for not sending this through a normal deprecation process at all. > > But even if there isn't anyone relying on this, and we fix it as soon as > possible (which will be 3.5), your library *still* can't rely on it > until it drops all support for versions below 3.5. Since you can't rely > on that feature being present, you have to write your library code as if > it wasn't present. So there is *absolutely no difference* between your > library with this (alleged) bug being fixed, or it not being fixed. > There is a difference in the future when 3.5 (or whatever) is a reasonable minimum for a library. People say the same sort of thing about *anything* that is fixed or added to Python 3.x only that can?t be backported easily. There is a wait time until library authors can depend on anything new or changed in Python and there always has been an always will be. All that means is that a change now won?t be widely useful until some years down the road. But that?s true for *anything*, asyncio, PEP453, yield from, and this change too. > The situation for application developers is a bit different. An > application developer can simply say, I'm going to immediately migrate > to 3.5 to take advantage of this new feature/bug fix. That's okay too. > > The job of the core developers is to balance all these competing > interests: the library developers, the application developers, the users > who don't care one way or another but do care about some other bug > that's not being fixed because people are arguing about this one, the > people (actual or hypothetical) who actually like this (mis)feature the > way it is, people who don't like it but have written code that relies on > it, and, yes, even the core developers themselves, who are perfectly > entitled to say that the effort in fixing this is greater than the > benefit so they're not going to do it. > > If you haven't already read Nick's blog posts on these issues, you > should: > > http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html > http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html I?ve read them. I have ~opinions~ about them but I?m not going to derail this thread with them. > > > For what it's worth, I think the current behaviour is a misfeature, and > in a perfect world it would be fixed. But the cost of this is so > miniscule, and the work-around so trivial, that although the fix is > cheap, the benefit is even lower. I've only spent a few minutes reading > and responding to this thread. Those few minutes are already worth far > more than any benefit to me in fixing this. That's *my* personal opinion > on this, others may differ, so I'm a +0 on deprecating the behaviour in > 3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately > in 3.5. > > As I said, the work around is trivial, *once you know it is needed*. This change is reducing the cognitive burden and chance for mishap for users who aren?t aware of this misfeature. It?s making Python a friendlier language to use. > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/f08922f3/attachment-0001.sig> From tim.peters at gmail.com Thu Mar 6 03:33:53 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 20:33:53 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> Message-ID: <CAExdVNnrye3OMJShuXnVHDgUMyrGmiU26zFCXwQ4DjZ82YggVw@mail.gmail.com> > Forgive me if I'm wrong, but aren't you the author of the date time module? Yes. > If that's the case what you expect it to do isn't particularly relevant as you're > intimately aware of it's implementation. Drat! Checkmated again! I get tired of putting smileys after everything ;-) > It's hard to do any sort of search for this, however in an informal poll where I've shown > people this code Precisely which code did you show them? What were the backgrounds of these people? Etc etc etc. > not a single person thought it made sense, and most of them responded > with "wtf?" Which, via amazing coincidence or foresight, is exactly what I bet you were expecting ;-) >. I've also seen first hand through Django (which is why the person who started > this thread) get caught by this bug and have to spend hours trying to figure out why it's > behaving that way. Too sketchy to say much about. If this is a variation of testing "if object:" as a shorthand for testing "if object is None:", little sympathy from me. In any case, this took more of my time so far than any possible outcome would be worth to me, so I'll just close with -0 on changing it (would rather not bother, but won't complain if it is changed). From harrismh777 at gmail.com Thu Mar 6 03:21:37 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Wed, 5 Mar 2014 18:21:37 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> Message-ID: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> > > > Then, and function that requires PI > of half_PI can check first to see if it is cached as __gpi__ and if so > pull it back, or if not calculate > it. If the context does not change than I half some whopping value of PI > for the duration of the > context (but, and this is the big but, if I don't need it --- then it > never gets bound). > > um, sorry, I wixed my mords... please forgive, "Then, and only then, a function that requires PI or half_PI will first check to see if __gpi__ has been cached and if so pull it back, or if not calculate it (once). If the context does not change then the function can take half of some whopping value of PI, for the duration of the context, and use that, but if not needed for the context will never bind it ! marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/004e934b/attachment.html> From steve at pearwood.info Thu Mar 6 04:00:02 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 14:00:02 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> Message-ID: <20140306030002.GH28804@ando> On Wed, Mar 05, 2014 at 09:23:37PM -0500, Donald Stufft wrote: > It?s hard to do any sort of search for this, however in an informal poll where I?ve shown > people this code not a single person thought it made sense, and most of them responded > with ?wtf??. Well I don't know who these people are, what their background is, or exactly how you phrased the question. But in my experience, most programmers have extremely strongly held opinions about the sensibility of certain features that have little or nothing with any rational analysis of the pros and cons and far more likely to be "that's different from the first language I learned, therefore it's rubbish". Having read Tim's explanation for why time() is falsey, I no longer think it's a misfeature, and am now -0.5 on changing it at all. > I?ve also seen first hand through Django (which is why the person who > started this thread) get caught by this bug and have to spend hours > trying to figure out why it?s behaving that way. Again, I don't know the background here, and we all know that sometimes tracking down bugs can be difficult, but if somebody is spending "hours" trying to debug why "if time_value" sometimes does the wrong thing, *without* bothering to read the documentation for time objects to find out how they're supposed to behave, then I don't have a lot of sympathy. Also, if you write "if spam" as a short-cut for "if spam is not None", then again any bugs you experience are self-inflicted. -- Steven From guido at python.org Thu Mar 6 04:47:24 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Mar 2014 19:47:24 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> Message-ID: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> I wonder if we need yet another list, python-speculative-ideas? Or python-waxing-philosophically? I've got a feeling were straying quite far from the topic of the design of even Python 4, let alone what could possibly happen in Python 3. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5c850ebf/attachment.html> From sturla.molden at gmail.com Thu Mar 6 04:54:24 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 06 Mar 2014 04:54:24 +0100 Subject: [Python-ideas] Add "goto fail" to Python? Message-ID: <lf8rh3$suv$1@ger.gmane.org> "goto fail" is a well-known error handling mechanism in open source software, widely reputed for its robusteness: http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c I believe Python needs to add support for this superior paradigm. It would involve a new keyword "fail" and some means of goto'ing to it. I suggest "raise to fail": if (some_error): raise to fail fail: <error handling code> Unless there are many objections, this fantastic idea might be submitted in a (short) PEP somewhere around the beginning of next month. There is some obvious overlap with the rejected "goto PEP" (PEP 3163) and the Python 2.3 goto module. However, the superiority of goto fail as error generation and error handling paradigm has since then been thoroughly proven. http://legacy.python.org/dev/peps/pep-3136/ http://entrian.com/goto/download.html Regards, Sturla Molden From bruce at leapyear.org Thu Mar 6 05:00:35 2014 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 5 Mar 2014 20:00:35 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> Message-ID: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> On Wed, Mar 5, 2014 at 5:53 PM, Tim Peters <tim.peters at gmail.com> wrote: > That's there because `utcoffset()` is defined to return minutes. > It returns either minutes *or a timedelta*: datetime.utcoffset()?<http://docs.python.org/2/library/datetime.html#datetime.datetime.utcoffset> If tzinfo <http://docs.python.org/2/library/datetime.html#datetime.tzinfo> is None, returns None, else returns self.tzinfo.utcoffset(self), and raises an exception if the latter doesn't return None, or a timedelta<http://docs.python.org/2/library/datetime.html#datetime.timedelta> object representing a whole number of minutes with magnitude less than one day. tzinfo.utcoffset(*self*, *dt*) Return offset of local time from UTC, in minutes east of UTC. If local time is west of UTC, this should be negative. Note that this is intended to be the total offset from UTC; for example, if atzinfo<http://docs.python.org/2/library/datetime.html#datetime.tzinfo> object represents both time zone and DST adjustments, utcoffset()<http://docs.python.org/2/library/datetime.html#datetime.tzinfo.utcoffset> should return their sum. If the UTC offset isn't known, return None. Else the value returned must be a timedelta<http://docs.python.org/2/library/datetime.html#datetime.timedelta>object specifying a whole number of minutes in the range -1439 to 1439 inclusive (1440 = 24*60; the magnitude of the offset must be less than one day). A > rule is needed to specify how the naive time object and `utcoffset()` > interact to define the result. > Not sure why you attach 'naive' to time. Time objects carry units with them, unlike integers. I would say 'naive time value' in reference to an integer. Combining a time and an integer can be done by converting the time to minutes (which is a bit confusing since in most contexts in this class, minutes values are integers) or it can be done by interpreting the integer utcoffset result as minutes (as it is documented). > > Perhaps a reasonable improvement would be to change this to: > > > > a time object is considered to be true unless it represents exactly > midnight > > local time, > The above part is the key part of my suggested improvement. Reading the current documentation, the fact that it means midnight local time is falsey is not obvious. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5bbe1659/attachment-0001.html> From abarnert at yahoo.com Thu Mar 6 05:07:13 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 20:07:13 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> Message-ID: <DF3B61D8-81D3-448F-BD85-7CA21748DDB6@yahoo.com> On Mar 5, 2014, at 17:34, Tim Peters <tim.peters at gmail.com> wrote: > "I am altering the deal. Pray I don't alter it any further." There's > a reason Darth Vader wasn't proclaimed Python's BDFL ;-) I always assumed it was because he never contributed even a single patch. From tim.peters at gmail.com Thu Mar 6 05:13:59 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 22:13:59 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> Message-ID: <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> [Tim] >> That's there because `utcoffset()` is defined to return minutes. [Bruce Leban] > It returns either minutes or a timedelta: I wrote the code and I wrote the docs. I know what it does ;-) Going into tedious detail (as the docs do) is simply irrelevant to the point here: utcoffset() returns minutes. Whether that's _spelled_ as an integer or as a timedelta, minutes is minutes is minutes. > ... >> A rule is needed to specify how the naive time object and `utcoffset()` >> interact to define the result. > Not sure why you attach 'naive' to time. Time objects carry units with them, > unlike integers. I would say 'naive time value' in reference to an integer. Start with the second paragraph of the datetime docs: There are two kinds of date and time objects: "naive" and "aware". Etc. I attached "naive" to "time" to make clear that I mean the time object stripped of its tzinfo attribute. An aware time object is a naive time object with a non-None tzinfo attribute. > .... > The above part is the key part of my suggested improvement. Reading the > current documentation, the fact that it means midnight local time is falsey > is not obvious. Before this thread, I had never heard anyone express it as "midnight local time" before. Now that I have heard it, eh - I don't find it a helpful way to view it. From alexander.belopolsky at gmail.com Thu Mar 6 05:15:31 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 5 Mar 2014 23:15:31 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> Message-ID: <CAP7h-xZB=UYhO5fF3ujVQKFfx7w1WrpiAYtLZB-o2eMwqLPqjw@mail.gmail.com> On Wed, Mar 5, 2014 at 11:00 PM, Bruce Leban <bruce at leapyear.org> wrote: > Not sure why you attach 'naive' to time. 'naive' is a defined term in the context of the datetime module. It means tzinfo is None. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/e728ae3e/attachment.html> From abarnert at yahoo.com Thu Mar 6 05:16:52 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 5 Mar 2014 20:16:52 -0800 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <5317CF11.20100@mrabarnett.plus.com> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> <5317B1D7.4090305@canterbury.ac.nz> <5317CF11.20100@mrabarnett.plus.com> Message-ID: <5119B096-30C7-4B36-BB59-BE0FEC9B599D@yahoo.com> On Mar 5, 2014, at 17:27, MRAB <python at mrabarnett.plus.com> wrote: > In the UK we talk about "Bills" being put before Parliament. In the US also, except it's Congress, and most of our political discussion isn't about bills before Congress or anything else of political consequence. > It reminds me of that quotation about the UK and US: ?Two nations > divided by a common language?. In the UK you can pay a bill using a > cheque; in the US you can pay the check using bills (but a "check" is > also a (UK) cheque!). How many restaurants still accept cheques (or checks)? The only thing they're really used for anymore is paying money to the government... So this bit of confusion will die away. Pants, on the other hand, aren't going away any time soon. From greg.ewing at canterbury.ac.nz Thu Mar 6 05:41:11 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 17:41:11 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140305192419.GD28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> Message-ID: <5317FC67.7060600@canterbury.ac.nz> Steven D'Aprano wrote: > There's also the argument from consistency: midnight is the zero point > of the day, and zero is falsey. Regarding midnight as a "zero point" is an arbitrary convention. The integer zero is not arbitrary, because it's the additive identity. There is no corresponding additive identity for times, because you can't add two times. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 05:46:46 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 17:46:46 +1300 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> Message-ID: <5317FDB6.5000806@canterbury.ac.nz> CFK wrote: > then it shouldn't be too hard > for a project to add its own UUID to the annotation dictionary. I can't see what benefit there is in bringing UIIDs into the picture. If you're suggesting that people write the UUIDs directly into their code as literals, that's *not* going to fly. It would be appallingly ugly and error-prone. The only way to make it usable would be to import the UUIDs from somewhere under human-readable names. But then there's no need to use UUIDs, any set of unique sentinel objects will do. -- Greg From mertz at gnosis.cx Thu Mar 6 05:59:48 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 5 Mar 2014 20:59:48 -0800 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <lf8rh3$suv$1@ger.gmane.org> References: <lf8rh3$suv$1@ger.gmane.org> Message-ID: <CAEbHw4YQmejBovpcbLqRTw8C2t73RbEJtpYkmut4WCf08fWUsQ@mail.gmail.com> This is *exactly* what Python does right now! class ToFailError(Exception): pass try: do_some_stuff() if (some_error): raise ToFailError do_other_stuff() try: thing_that_might_fail() except SomeOtherError: handle_it() except ToFailError: here_we_are() On Wed, Mar 5, 2014 at 7:54 PM, Sturla Molden <sturla.molden at gmail.com>wrote: > "goto fail" is a well-known error handling mechanism in open source > software, widely reputed for its robusteness: > > http://opensource.apple.com/source/Security/Security- > 55471/libsecurity_ssl/lib/sslKeyExchange.c > > https://www.gitorious.org/gnutls/gnutls/source/ > 6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c > > I believe Python needs to add support for this superior paradigm. > > It would involve a new keyword "fail" and some means of goto'ing to it. I > suggest "raise to fail": > > if (some_error): > raise to fail > > fail: > <error handling code> > > Unless there are many objections, this fantastic idea might be submitted > in a (short) PEP somewhere around the beginning of next month. > > There is some obvious overlap with the rejected "goto PEP" (PEP 3163) and > the Python 2.3 goto module. However, the superiority of goto fail as error > generation and error handling paradigm has since then been thoroughly > proven. > > http://legacy.python.org/dev/peps/pep-3136/ > http://entrian.com/goto/download.html > > > Regards, > Sturla Molden > > > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/6f030e41/attachment-0001.html> From bruce at leapyear.org Thu Mar 6 06:16:49 2014 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 5 Mar 2014 21:16:49 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> Message-ID: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> On Wed, Mar 5, 2014 at 8:13 PM, Tim Peters <tim.peters at gmail.com> wrote: > [Tim] > >> That's there because `utcoffset()` is defined to return minutes. > > [Bruce Leban] > > It returns either minutes or a timedelta: > > I wrote the code and I wrote the docs. I know what it does ;-) Going > into tedious detail (as the docs do) is simply irrelevant to the point > here: utcoffset() returns minutes. Whether that's _spelled_ as an > integer or as a timedelta, minutes is minutes is minutes. > Just because you wrote the docs doesn't mean you know what they mean to other readers. The point of documentation is to explain it to someone who doesn't know what it does after all. > > > Start with the second paragraph of the datetime docs: > > There are two kinds of date and time objects: "naive" and "aware". > Thanks. Before this thread, I had never heard anyone express it as "midnight > local time" before. Now that I have heard it, eh - I don't find it a > helpful way to view it. Well, it's not surprising that you don't find it's helpful. In fact, it's wrong, which to me indicates how confusing this. Which of the following are falsey? 00:00:00+01:00 01:00:00+01:00 01:00:00-01:00 07:00:00+07:00 23:00:00-01:00 I find it particularly surprising that the bug that the OP described finding would occur in the Russia and China, but not in the US. Obviously, not an accident. :-) --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/098d445e/attachment.html> From greg.ewing at canterbury.ac.nz Thu Mar 6 06:18:47 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:18:47 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317A4BB.1090303@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <DCA1D330-432B-430F-9323-538076D1F6BD@stufft.io> <5317A4BB.1090303@egenix.com> Message-ID: <53180537.1050708@canterbury.ac.nz> M.-A. Lemburg wrote: > The reasoning becomes clear if you regard a time value as > number of seconds since midnight (the time of day is a relative > measure). But there's no clear reason to regard that particular way of representing a time of day as superior to any other. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 06:22:07 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:22:07 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317A696.8040605@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> Message-ID: <531805FF.8070409@canterbury.ac.nz> M.-A. Lemburg wrote: > This is true in a date/time continuum, but not true if you regard > time as "time of day". In the latter interpretation, the day starts > at midnight, That's not always the most convenient way of looking at it. For example, for booking a stay in a hotel, it may be better to think of "nights" as stretching from midday to midday. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 06:30:21 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:30:21 +1300 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> <53179C51.4020001@canterbury.ac.nz> <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> Message-ID: <531807ED.3020003@canterbury.ac.nz> Masklinn wrote: > On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >>Also, Haskell has the advantage of knowing that the value >>won't change > > I do not see why that would be relevant to thunks. The result of the > thunk may be mutable, so what? That's not what I mean. The value of the thunk can depend on other things that change. In general you can't get away with evaluating it once and caching the result. -- Greg From sturla.molden at gmail.com Thu Mar 6 06:30:01 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 6 Mar 2014 05:30:01 +0000 (UTC) Subject: [Python-ideas] Add "goto fail" to Python? References: <lf8rh3$suv$1@ger.gmane.org> <CAEbHw4YQmejBovpcbLqRTw8C2t73RbEJtpYkmut4WCf08fWUsQ@mail.gmail.com> Message-ID: <444107387415775972.062814sturla.molden-gmail.com@news.gmane.org> David Mertz <mertz at gnosis.cx> wrote: > This is *exactly* what Python does right now! I know, it would by syntaxic sugar for try-except. I am worried that Python's significant indentation might interfere with some of the strengths of "goto fail", for example placement of code that could be vital for national security. We might therefore need an exception to the indentation syntax so that if (error): raise to fail raise to fail <suite> is interpreted as if (error): raise to fail raise to fail if 1: <suite> Sturla From greg.ewing at canterbury.ac.nz Thu Mar 6 06:34:49 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:34:49 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317B1BE.4090804@egenix.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> <5317B1BE.4090804@egenix.com> Message-ID: <531808F9.1020605@canterbury.ac.nz> M.-A. Lemburg wrote: > Well, the duration between the start of day and midnight is zero > and returning to the interpretation of time of day being a measurement > relative to midnight, The point of having time objects as an abstraction is so that you *don't* have to think of them as an offset from anything. You simply think of them as times. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 6 06:38:40 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:38:40 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> Message-ID: <531809E0.6000905@canterbury.ac.nz> Alexander Belopolsky wrote: > For me, it is the fraction of the day that passed or distance from > midnight and in any case fundamentally some kind of number expressed in > a Babylonian base-60 notation. Yes, and that number is conventionally written as "12:00", which is *obviously* equal to zero... er, what? -- Greg From guido at python.org Thu Mar 6 06:42:22 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Mar 2014 21:42:22 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <531809E0.6000905@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> <531809E0.6000905@canterbury.ac.nz> Message-ID: <CAP7+vJKoRJaSZF_bE5sSjiujQWdniffrTNnfRwGSY861LKPbdQ@mail.gmail.com> The 12:00 notation is US-centric. Or, perhaps, common in Anglophone countries. The rest of the world does not use the AM/PM conventions and calls midnight 00:00. On Wed, Mar 5, 2014 at 9:38 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote: > Alexander Belopolsky wrote: > >> For me, it is the fraction of the day that passed or distance from >> midnight and in any case fundamentally some kind of number expressed in a >> Babylonian base-60 notation. >> > > Yes, and that number is conventionally written as > "12:00", which is *obviously* equal to zero... er, > what? > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/5fb9138b/attachment.html> From harrismh777 at gmail.com Thu Mar 6 05:05:37 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Wed, 5 Mar 2014 20:05:37 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> Message-ID: <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote: > > I wonder if we need yet another list, python-speculative-ideas? Or > python-waxing-philosophically? > hi Guido, no probably not. I just got to thinking again about *number *again as I was thinking about decimal floating point as default. Its the mathematician in me; go for the general case, and that would be a type-less unified *number *system. For the near case design issues the following realist ideas might be considered in order: 1) decimal literal like 1.23d 2) default decimal floating point with a binary type 1.23b 3) type-less unified *number *yes, probably way out there into the future kind regards, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/bc00238c/attachment.html> From greg.ewing at canterbury.ac.nz Thu Mar 6 06:50:10 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 18:50:10 +1300 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <5317CF11.20100@mrabarnett.plus.com> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> <5317B1D7.4090305@canterbury.ac.nz> <5317CF11.20100@mrabarnett.plus.com> Message-ID: <53180C92.7070904@canterbury.ac.nz> MRAB wrote: > In the UK we talk about "Bills" being put before Parliament. In NZ too, but once it's passed, it's the law -- it's not up to individual interpretation! -- Greg From tim.peters at gmail.com Thu Mar 6 06:53:19 2014 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 5 Mar 2014 23:53:19 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> Message-ID: <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> [Bruce Leban <bruce at leapyear.org>] > ... > Just because you wrote the docs doesn't mean you know what they mean to > other readers. The point of documentation is to explain it to someone who > doesn't know what it does after all. Of course. What's your point here? >> ... >> Before this thread, I had never heard anyone express it as "midnight >> local time" before. Now that I have heard it, eh - I don't find it a >> helpful way to view it. > Well, it's not surprising that you don't find it's helpful. In fact, it's > wrong, which to me indicates how confusing this. Bruce, you're the one who made up the "midnight local time" business, which has just confused you. Try again to read what the docs actually say, without assuming it must mean something else? > Which of the following are falsey? It would help a lot if you explained which ones surprised _you_, and how you read the docs in such a way that what they actually say caused your confusion. > 00:00:00+01:00 Convert the naive time to minutes: 0. Get the UTC offset in minutes (these offsets are always returned directly as minutes in the code, but there's an "extra" calculation step needed here because you're starting from a string representation that also uses units of hours): 1*60 + 0 = 60. Subtract: 0-60 = -60 Is that zero? No, so this one is true. > 01:00:00+01:00 60 - 60 = 0. False. > 01:00:00-01:00 60 - -60 = 120. True. > 07:00:00+07:00 420 - 420 = 0. False. > 23:00:00-01:00 23*60 - -60 = 24*60. True. > I find it particularly surprising that the bug that the OP described finding > would occur in the Russia and China, but not in the US. Obviously, not an > accident. :-) Indeed, it is the key to liberating both Ukraine and Tibet ;-) From guido at python.org Thu Mar 6 06:56:02 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Mar 2014 21:56:02 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> Message-ID: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> Do you actually have a degree in math, or do you just remember your high school algebra? The numbers in math usually are quite strictly typed: whole theories only apply to integers or even positive integers, other things to all reals but not to complex numbers. (And what about quaternions? Or various infinities? :-) On Wed, Mar 5, 2014 at 8:05 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > > > On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote: >> >> I wonder if we need yet another list, python-speculative-ideas? Or >> python-waxing-philosophically? >> > > hi Guido, no probably not. I just got to thinking again about *number *again > as > I was thinking about decimal floating point as default. Its the > mathematician in me; go > for the general case, and that would be a type-less unified *number * > system. > > For the near case design issues the following realist ideas might be > considered in order: > > 1) decimal literal like 1.23d > > 2) default decimal floating point with a binary type 1.23b > > 3) type-less unified *number *yes, probably way out there into the future > > > kind regards, > marcus > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/8cc3d198/attachment-0001.html> From alexander.belopolsky at gmail.com Thu Mar 6 07:47:12 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 01:47:12 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> Message-ID: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> On Thu, Mar 6, 2014 at 12:16 AM, Bruce Leban <bruce at leapyear.org> wrote: > Well, it's not surprising that you don't find it's helpful. In fact, it's > wrong, which to me indicates how confusing this. Which of the following are > falsey? > > 00:00:00+01:00 > 01:00:00+01:00 > 01:00:00-01:00 > 07:00:00+07:00 > 23:00:00-01:00 > > > I find it particularly surprising that the bug that the OP described > finding would occur in the Russia and China, but not in the US. Obviously, > not an accident. :-) > "Aware" time objects are problematic even before you consider their bool behavior. Fortunately, you don't encounter them too often because dt.time() returns naive time even if dt is an aware datetime instance. Again, the use case that I find compelling is to test dt.time() for falsehood to detect datetime objects that fall on midnight. I don't understand why people are denying that midnights are special points. All real numbers are equal, but integers are more equal than others. When you plot say hourly timeseries it is not uncommon to mark midnight ticks on the time axis with dt.date() and other ticks with dt.hour. Whenever you need to do something special when you start a new date in a simulation of a physical process it is convenient to have a simple test for midnight. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/deeb546e/attachment.html> From greg.ewing at canterbury.ac.nz Thu Mar 6 07:51:06 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 19:51:06 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> Message-ID: <53181ADA.2030401@canterbury.ac.nz> Tim Peters wrote: > For example, modular arithmetic on time > objects was a possibility. In that case, time(0, 0, 0) would become > "a zero value" That only follows if you interpret "modular arithmetic on time objects" as meaning that you can directly add one time object to another. But that would be confusing times with timedeltas. The concept of a zero *timedelta* makes sense, but not a "zero time". > People can say that "midnight" isn't a compelling "sentinel value", > and I agree about the "compelling" part, but in all implementation > respects it does _act_ as "a sentinel value". If someone is using midnight to represent an unspecified time, I'd say they have a bug. An appointment scheduled at midnight is not the same as an appointment that's not scheduled at all. -- Greg From bruce at leapyear.org Thu Mar 6 08:02:34 2014 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 5 Mar 2014 23:02:34 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> Message-ID: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > it is convenient to have a simple test for midnight. > Except this isn't it. That's only works for naive times. For aware times: 00:00:00+00:00 = midnight England = midnight UTC => False 00:00:00+01:00 = midnight France = 2300 UTC => True 01:00:00+01:00 = 1 am France = midnight UTC => False 19:00:00-05:00 = 1 am Boston = midnight UTC => True Code that relies that bool(time) is False for midnight is broken IMHO. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140305/9f6da53f/attachment.html> From alexander.belopolsky at gmail.com Thu Mar 6 08:20:30 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 02:20:30 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> Message-ID: <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> On Thu, Mar 6, 2014 at 2:02 AM, Bruce Leban <bruce at leapyear.org> wrote: > > On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky < > alexander.belopolsky at gmail.com> wrote: > >> it is convenient to have a simple test for midnight. >> > > Except this isn't it. That's only works for naive times. For aware times: > > 00:00:00+00:00 = midnight England = midnight UTC => False > 00:00:00+01:00 = midnight France = 2300 UTC => True > 01:00:00+01:00 = 1 am France = midnight UTC => False > 19:00:00-05:00 = 1 am Boston = midnight UTC => True > > Code that relies that bool(time) is False for midnight is broken IMHO. > > You keep fighting your own straw-man. My use case is not using an arbitrary time object that someone decided to imbue with a timezone info. What I want is given a *datetime* instance dt to check whether it falls on midnight in whatever timezone dt.tzinfo represent or naively if dt.tzinfo is None. I claim that "if dt.time()" is a perfectly good shorthand for "if dt.time() != datetime.time(0)". I am not trying to argue that "if not dt.timetz()" test is equally useful. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5c6e8f81/attachment.html> From stephen at xemacs.org Thu Mar 6 08:29:20 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 16:29:20 +0900 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> <53179C51.4020001@canterbury.ac.nz> <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> Message-ID: <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp> Masklinn writes: > On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > > Masklinn wrote: > >> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > > That comes at the expense of making *everything* a thunk > > until its value is needed [in Haskell]. > > Of course not, Haskell allows explicitly forcing a thunk and the > compiler/runtime pair can use strictness analysis and not create thunks > in the first place. That doesn't seem fair to me. "Explicitly forcing" by definition means the programmer has decided the value is needed, and "strictness analysis" is an optimization, not part of the language definition. Am I missing something? > Consider that a thunk is a deferral of an expression's evaluation, > why would said expression's evaluation happen more than once? The > only thing which changes is *when* the actual evaluation happens. Are thunks guaranteed not to leak out of the scope of the containing expression, so assignment is impossible? If they can leak, the thunk would be an object, and in Python "assigning" it to a "variable" actually just creates a reference. Evaluation could memoize the thunk's value ("evaluation only happens once") or reevaluate the thunk each time its value is requested. Either seems potentially surprising to me. Memoization makes sense if you think of the thunk as the closure of a computation that conceptually takes place at definition time. I'm not sure if in the use cases for thunks it really makes sense to "close" the thunk at evaluation time, but it seems like a plausible interpretation to me. From donald at stufft.io Thu Mar 6 08:30:39 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 02:30:39 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> Message-ID: <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> On Mar 6, 2014, at 2:20 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > > > > On Thu, Mar 6, 2014 at 2:02 AM, Bruce Leban <bruce at leapyear.org> wrote: > > On Wed, Mar 5, 2014 at 10:47 PM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > it is convenient to have a simple test for midnight. > > Except this isn't it. That's only works for naive times. For aware times: > > 00:00:00+00:00 = midnight England = midnight UTC => False > 00:00:00+01:00 = midnight France = 2300 UTC => True > 01:00:00+01:00 = 1 am France = midnight UTC => False > 19:00:00-05:00 = 1 am Boston = midnight UTC => True > > Code that relies that bool(time) is False for midnight is broken IMHO. > > > You keep fighting your own straw-man. My use case is not using an arbitrary time > object that someone decided to imbue with a timezone info. What I want is given > a *datetime* instance dt to check whether it falls on midnight in whatever timezone > dt.tzinfo represent or naively if dt.tzinfo is None. So check for it falling on midnight. It?s not reasonable to expect that midnight will evaluate to false, especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!). That makes the behavior even more surprising and more footgun-ey. > > I claim that "if dt.time()" is a perfectly good shorthand for "if dt.time() != datetime.time(0)?. And I claim that you?re reaching to justify stupid behavior. > > I am not trying to argue that "if not dt.timetz()" test is equally useful And yet you can?t control what people do with it. Consistency is a virtue and how objects act with a tzinfo and how they act without a tzinfo should be as close as possible. > . > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cb0038f5/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cb0038f5/attachment.sig> From shai at platonix.com Thu Mar 6 08:25:26 2014 From: shai at platonix.com (Shai Berger) Date: Thu, 06 Mar 2014 09:25:26 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306030002.GH28804@ando> References: <201403051216.12392.shai@platonix.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <20140306030002.GH28804@ando> Message-ID: <1978788.hqjmn1vSZ0@deblack> On Thursday 06 March 2014 14:00:02 Steven D'Aprano wrote: > tracking down bugs can be difficult, but if somebody is spending "hours" > trying to debug why "if time_value" sometimes does the wrong thing, > *without* bothering to read the documentation for time objects to find > out how they're supposed to behave, then I don't have a lot of sympathy. > This argument keeps repeating. It is a gross misunderstanding of the problem. The hours are not spent trying to debug why an if went the wrong way. The hours are spent trying to understand why some feature of a program doesn't work as expected -- *sometimes*. It can easily take hours to try and reproduce the wrong behavior, in vain, because it only shows up on unique data; and the wrong behavior will often not show itself as "an if went the wrong way", but, for example, "sometimes a number here is wrong" -- which happens when the calculation of the number goes over a collection of objects and does complex things including decisions based on time. Of course, once you get to the "if" it's relatively easy (surprising behavior is still surprising, but less mystifying). It's getting to it that's the issue. Thanks, Shai. From greg.ewing at canterbury.ac.nz Thu Mar 6 08:33:43 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2014 20:33:43 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> Message-ID: <531824D7.8020302@canterbury.ac.nz> Bruce Leban wrote: > Except this isn't it. That's only works for naive times. For aware times: > > 00:00:00+00:00 = midnight England = midnight UTC => False > 00:00:00+01:00 = midnight France = 2300 UTC => True > 01:00:00+01:00 = 1 am France = midnight UTC => False > 19:00:00-05:00 = 1 am Boston = midnight UTC => True That's... I don't know what it is. Is there a rationale for this? I'm totally failing to see it. -- Greg From alexander.belopolsky at gmail.com Thu Mar 6 08:40:24 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 02:40:24 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> Message-ID: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote: > It?s not reasonable to expect that midnight will evaluate > to false, .. > Only in the world where it is not reasonable to expect programmers to read library documentation. In my world it is reasonable to expect that the behavior that was documented in 10 major versions and for 10 years can be relied on. > especially when it doesn?t if you happen to have a tzinfo on the time > (sometimes!). > As long as tzinfo specifies a fixed offset, there is no problem with the current definition. If you are unfortunate enough to live in a place with semi-annual DST adjustment, aware time objects are problematic for reasons that have nothing to do with the discussion at hand. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b7d44ebd/attachment.html> From donald at stufft.io Thu Mar 6 08:43:18 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 02:43:18 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> Message-ID: <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io> On Mar 6, 2014, at 2:40 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > > On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote: > It?s not reasonable to expect that midnight will evaluate > to false, .. > > Only in the world where it is not reasonable to expect programmers to read library documentation. In my world it is reasonable to expect that the behavior that was documented in 10 major versions and for 10 years can be relied on. > > > especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!). > > As long as tzinfo specifies a fixed offset, there is no problem with the current definition. If you are unfortunate enough to live in a place with semi-annual DST adjustment, aware time objects are problematic for reasons that have nothing to do with the discussion at hand. > Nope! It essentially in an aware time would be ?false if whatever time is at the same time as UTC midnight, but only if your UTC offset is positive?. If your UTC offset is negative then you end up with 1440 instead of 0. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b0044cae/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/b0044cae/attachment-0001.sig> From markus at unterwaditzer.net Thu Mar 6 08:38:41 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Thu, 06 Mar 2014 08:38:41 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306021457.GF28804@ando> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> <20140306021457.GF28804@ando> Message-ID: <556cc654-167b-4d06-8c30-77c414e54069@email.android.com> On 6 March 2014 03:14:57 CET, Steven D'Aprano <steve at pearwood.info> wrote: >On Wed, Mar 05, 2014 at 04:31:36PM -0500, Donald Stufft wrote: > >> I find the argument that it would only be fixed in 3.something and >thus not be really >> fixed for X years disingenuous at best. That fact is true for *ALL* >bug fixes in Python >> so by that logic we should simply give up on fixing anything in >Python ever? > >Not at all. But it does mean that the benefit of fixing it needs to be >worth the wait. Suppose we said "Sure, we'll fix it, but you have >to pay for the fix." How much would you be willing to pay to have it >fixed? > >("Pay" doesn't have to mean money. It can also mean, how much pain or >inconvenience are you willing to experience, or how much time and >effort >are you willing to out into this.) > >If I said, "Oh by the way, although you have to pay right now, you >won't >see any benefit until 2024", now how much are you willing to pay? > >Design flaws are more costly to fix than outright bugs, not because >they >take more time and effort to change, but because you're changing >behaviour that *somebody out there is relying on*. Just because "somebody" is relying on a wart in the language it doesn't mean that this person is knowing what they are doing, is representative of the rest of Python users or that the wart shouldn't be removed. In my opinion it even means that the person is a bad programmer in a way, because even though he is relying documented behavior, that behavior is relatively unknown and using it intentionally certainly doesn't help readability. Actually i think that this programmer put extra effort into making their own code less readable. Partially, this point has already been made many times in this thread, but apparently this doesn't suffice to make the opposing side realize or respond to it. What you consider a >terrible misfeature is *somebody's* critical feature, and you're >proposing that we take that away. That's okay, we can do that after a >suitable deprecation period. (You wouldn't want us to break your code, >so you ought to extend the same courtesy to other people.) > >But even if there isn't anyone relying on this, and we fix it as soon >as >possible (which will be 3.5), your library *still* can't rely on it >until it drops all support for versions below 3.5. Arguably many libraries already do rely on this currently nonexistent feature. >Since you can't rely > >on that feature being present, you have to write your library code as >if >it wasn't present. So there is *absolutely no difference* between your >library with this (alleged) bug being fixed, or it not being fixed. > >The situation for application developers is a bit different. An >application developer can simply say, I'm going to immediately migrate >to 3.5 to take advantage of this new feature/bug fix. That's okay too. > >The job of the core developers is to balance all these competing >interests: the library developers, the application developers, the >users >who don't care one way or another but do care about some other bug >that's not being fixed because people are arguing about this one, the >people (actual or hypothetical) who actually like this (mis)feature the > >way it is, people who don't like it but have written code that relies >on >it, and, yes, even the core developers themselves, who are perfectly >entitled to say that the effort in fixing this is greater than the >benefit so they're not going to do it. > >If you haven't already read Nick's blog posts on these issues, you >should: > >http://www.curiousefficiency.org/posts/2011/02/status-quo-wins-stalemate.html >http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html > > >For what it's worth, I think the current behaviour is a misfeature, and > >in a perfect world it would be fixed. But the cost of this is so >miniscule, and the work-around so trivial, that although the fix is >cheap, the benefit is even lower. I've only spent a few minutes reading > >and responding to this thread. Those few minutes are already worth far >more than any benefit to me in fixing this. That's *my* personal >opinion >on this, others may differ, so I'm a +0 on deprecating the behaviour in > >3.5 and changing it in 3.6 or 3.7, and -0.5 on changing it immediately >in 3.5. From donald at stufft.io Thu Mar 6 08:49:11 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 02:49:11 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> <CB631BD4-59E4-4871-84E2-D5EA3144A040@stufft.io> Message-ID: <CCD4088B-78D5-4C4F-968F-79288C43D4AA@stufft.io> On Mar 6, 2014, at 2:43 AM, Donald Stufft <donald at stufft.io> wrote: > > On Mar 6, 2014, at 2:40 AM, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > >> >> On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote: >> It?s not reasonable to expect that midnight will evaluate >> to false, .. >> >> Only in the world where it is not reasonable to expect programmers to read library documentation. In my world it is reasonable to expect that the behavior that was documented in 10 major versions and for 10 years can be relied on. >> >> >> especially when it doesn?t if you happen to have a tzinfo on the time (sometimes!). >> >> As long as tzinfo specifies a fixed offset, there is no problem with the current definition. If you are unfortunate enough to live in a place with semi-annual DST adjustment, aware time objects are problematic for reasons that have nothing to do with the discussion at hand. >> > > Nope! It essentially in an aware time would be ?false if whatever time is at the same time as UTC midnight, but only if your UTC offset is positive?. If your UTC offset is negative then you end up with 1440 instead of 0. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ Actually now that I think about it more, in aware times it?s impossible for it to time objects to ever be false if the tzinfo is negative, because that would require the time to negative. Take -5 for instance: x - (-5 * 60) = 0 x - (-300) = 0 x + 300 = 0 x = -300 So in order for a time with UTC offset of -5 to ever be False, it would have to be -05:00:00 there. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f17eabc5/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f17eabc5/attachment.sig> From markus at unterwaditzer.net Thu Mar 6 08:53:46 2014 From: markus at unterwaditzer.net (Markus Unterwaditzer) Date: Thu, 06 Mar 2014 08:53:46 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> <CAGu0AnvOjfrJnDkPc8qp3LLmeXDUEGDKa5Jq+YurerQ0AgvsOg@mail.gmail.com> <CAP7h-xa8a=+Z7F31sh1nbmbfJa2WErdbFD3aT38Pr_zZjPRefw@mail.gmail.com> <11438CE8-253F-463B-B9B4-6C2EE9958A89@stufft.io> <CAP7h-xbzhqf1fKX9f9jzp7y5YQBZ_0+_Y3q4X9bjvBnFMWf3BQ@mail.gmail.com> Message-ID: <5392dc1c-26e8-4963-a47e-d5d624fea853@email.android.com> On 6 March 2014 08:40:24 CET, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: >On Thu, Mar 6, 2014 at 2:30 AM, Donald Stufft <donald at stufft.io> wrote: > >> It?s not reasonable to expect that midnight will evaluate >> to false, .. >> > >Only in the world where it is not reasonable to expect programmers to >read >library documentation. I disagree. Consider this code: class MyType: def __bool__(self): return bool(random.random()) Even though the behavior in boolean context is documented, it doesn't have to make any sense or be reasonable. >In my world it is reasonable to expect that the >behavior that was documented in 10 major versions and for 10 years can >be >relied on. > It *is* reasonable to expect to be able to rely on such features, yet IMO it is not reasonable to actually rely on that feature, for reasons of readability already mentioned by me in a reply to Steven D'Aprano. Considering that (from my subjective view) many people new to Python expect the kind of behavior proposed by the OP, i think it might be time for a change. Maybe an actual survey is the only way to find out. > > >> especially when it doesn?t if you happen to have a tzinfo on the time >> (sometimes!). >> > >As long as tzinfo specifies a fixed offset, there is no problem with >the >current definition. If you are unfortunate enough to live in a place >with >semi-annual DST adjustment, aware time objects are problematic for >reasons >that have nothing to do with the discussion at hand. > > >------------------------------------------------------------------------ > >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ From harrismh777 at gmail.com Thu Mar 6 09:14:11 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 00:14:11 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> Message-ID: <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote: > > Do you actually have a degree in math, or do you just remember your high > school algebra? > hi Guido, ouch. Its a story, glad you asked. My first trip to college 1974-1977 (UMKC) was to study EE; minor in mathematics. I completed my math course work (Calc 1-5, Diff Eq, Linear Algebra, Theory of Stats and Prob, &c) ... all (A). Not that it matters, because in 1977 IBM made me an offer in their CE department at Kansas City... so I joined IBM for the next 25 years and did not complete my EE degree... but, I did complete my mathematics training. Of course, prior to UMKC I attended a high school that offered calculus, math analysis, trig... and intro to linear algebra. So by the time I joined IBM I got the math twice. So, yeah, I know what I'm doing. I am an amateur mathematician today, computer scientist, and computer hobbyist. While at IBM I was a staff software engineer, Tampa, Chicago, Atlanta, and at the lab at Rochester MN (where I left IBM in 2002; yet I still dwell there). Education is a life long commitment and I continue to study math, comp sci, music and philosophy. I just completed my course work for the MDiv degree at Bethel Seminary. Back in the day, I added the scientific and transcendental math functions to the Rexx library for the internal VM370 systems, because Rexx (like python) also had no decimal floating point math package. So, yeah, its one of the things I know how to do, and its one of those things that most people never think about; but I've noticed that they appreciate having it once the work is done. My pdeclib package on PyPI is in infancy stage, probably has bugs (yet nobody has complained yet) and pdeclib will have to mature there for some time. But that has really nothing to do with whether we continue to use IEEE 754 1985 floats & doubles nor whether we discuss default decimal floating point arithmetic nor whether we discuss a unified *python number *sytem (sometime in the distant future) that would make | allow common average ordinary people to leverage mathematics in computer science without having to understand the underlying mechanics of implementation including but not limited to types. We might agree to stick with the discussion, if you're are willing, and stay away from *ad hominem* attacks, nor credential bashing. A person either knows what they are doing, or they don't. And if they are willing to contribute, why knock it, or them? > The numbers in math usually are quite strictly typed: whole theories only > apply to integers or even positive integers, other things to all reals but > not to complex numbers. > Everyone keeps making the points noted above! Yes, I know... concur~ Guido, all data types in most computer high level languages are all very strictly statically bound also. So what? You know this better than anyone, because you have taken so much abuse from trolls about it over the years. We all know that the best way to handle name binding is dynamically. And that was a paradigm shift, was it not? Go take a look at Rexx. It has NO types. None. Not at the surface, anyway. Everything to the user of the language is a string. This is even true for most of "object" Rexx too. *Numbers *are just strings of characters that are parsed and interpreted by Rexx as valid *Rexx Numbers*. It works. There is no real point in arguing that its a bad idea, because it served the Rexx community for many years... even now, Rexx is not dead. Yes, under the covers (not magic) a complex number is going to be handled differently than a real number. Yeah, what is your point? They are handled differently in my TI 89 too... so what, I change the context on the 89 and now I'm doing complex number math (not often, I might add). If I set the context for reals, then now I'm doing real math... it really is not that difficult. Think about this for just a minute. I have used complex math exactly three times in my life. I used it in high school to study the concept in math analysis. I used it in my EE classes... electrical engineers get a lot out of complex numbers. And I used it when I was interested in the Mendlebrot set many years ago when it was cool to plot the famous fractal on early cga screens. When was the last time you used complex numbers? When was the last time a bank, or a hospital, or Montgomery Wards used complex numbers? If someone needs complex numbers, Python could change the context "dynamically" and move through the problem set. Why should the user need to "manually" change the context if (AI) could change it for them on-the-fly? Just try to get a feel for the question, and stop with trying to beat me up on my credentials. > (And what about quaternions? Or various infinities? :-) > What about quaternions? If you extend the complex number system you change the context. That's not hard... You would not expect the context for "reals" processing to be the same for the context of processing three dimensional space with pairs of complex numbers, would you? Python does complex number processing now. But that is a very specialized category of use case that requires special context and (under the covers) typing relevant to complex number pairs. The context can change "dynamically" to suit the problem set, that's all I'm saying. I am not saying in all of my dreaming here that typing is not important (down under). Python is called an object based language, yes? Not object oriented, right? But we all know that down under the covers (not magic) python uses Classes and instances of classes--objects. We don't pretend that what makes object based languages work is magic? do we? Unifying the number system on a computer does not have to be grounded in paradigms that serviced the industry (and academy) for the past 40 years; mostly because memory was expensive and processing was slow. I suggest that if memory had been cheap, back in the day, and processing had been very fast (as it is today) IEEE 754 1985 floats and doubles would never have been used. We would have used Decimal floating points right from the get-go. Initially, that really is all I'm asking for on the outset--- lets move to default decimal floating point arithmetic for real numbers processing. In the future I am thinking about systems of languages that interact with human speech; understanding *spoken number* just like they will understand *symbol number.* There is really no reason (other than paradigm) that this would not be possible either. Otherwise, Guido, we might all of us just continue to use C and code up our statically bound types by hand using nothing but int, long, float, and double. Its time to innovate. Kind regards, BDfL, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/4c1b9950/attachment-0001.html> From stephen at xemacs.org Thu Mar 6 09:34:06 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 17:34:06 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> Message-ID: <87wqg7pvy9.fsf@uwakimon.sk.tsukuba.ac.jp> Yann Kaiser writes: > if event.date: > schedule_event() > > I cannot fathom one example where it could read as "does the party > start at midnight?". Can you? No, because it obviously reads as "... WTF??!?"[1] Of course, that's because I've imbibed of the "Python is not C or C++" Kool-Aid, but I just don't see why we should encourage C idioms in Python when there are plenty of other cases where use of a bare expression rather "... is None" can bite. I find Skip Montanaro's implied issue Please reconsider the Boolean evaluation of None far more plausible.<wink/> Footnotes: [1] if event.date: print("Happy New Year/Century/Millennium!") From mal at egenix.com Thu Mar 6 09:35:59 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 09:35:59 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> Message-ID: <5318336F.7030009@egenix.com> On 06.03.2014 03:23, Donald Stufft wrote: > > On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: > >> [Donald Stufft] >>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that >>> is a pretty compelling use case to me, unless you actively enjoy being user hostile. >> >> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that >> no code relies on the current behavior? For that matter, where is the evidence that the >> current behavior is a significant cause of bugs? I know I've used "if some_time_object:", >> but it did exactly what I expected it to do. > > Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case > what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s > implementation. Just in case you're looking for a relevant use case: Having bool(time(0,0,0)) allows you to quickly check whether you are dealing with a datetime value with (non-trivial) time part or not. You run into situations where you have to test for this in date/time conversions, arithmetics, etc. e.g. if you need to convert a datetime value to a date and want to prevent truncation of information, or if you want to format a datetime value in a user friendly way by omitting the zero time part, or if you're doing datetime arithmetic that has to follow special requirements w/r to days (think interest or insurance math) and you want to quickly check whether you can use the fast path simple calculation, or you need to do the full blown complicated part. Now, those use cases may still not appear relevant to you, but then the argument that "if x:" should not trigger for time(0,0,0) because x might actually be None, is also not relevant to me, since it masks a bug in the example code and those should never go undetected :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, >>> mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From donald at stufft.io Thu Mar 6 09:46:17 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 03:46:17 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318336F.7030009@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> Message-ID: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote: > On 06.03.2014 03:23, Donald Stufft wrote: >> >> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: >> >>> [Donald Stufft] >>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that >>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile. >>> >>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that >>> no code relies on the current behavior? For that matter, where is the evidence that the >>> current behavior is a significant cause of bugs? I know I've used "if some_time_object:", >>> but it did exactly what I expected it to do. >> >> Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case >> what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s >> implementation. > > Just in case you're looking for a relevant use case: > > Having bool(time(0,0,0)) allows you to quickly check whether you are > dealing with a datetime value with (non-trivial) time part or not. I don?t see how midnight is any more or less trivial than 12:01. > > You run into situations where you have to test for this in > date/time conversions, arithmetics, etc. e.g. if you need to convert > a datetime value to a date and want to prevent truncation of > information, or if you want to format a datetime value in a user > friendly way by omitting the zero time part, or if you're doing > datetime arithmetic that has to follow special requirements w/r to > days (think interest or insurance math) and you want to > quickly check whether you can use the fast path simple > calculation, or you need to do the full blown complicated part. I think these would be better served by actually checking for what you mean. For the record I also think that checking for a date time to not be None is also better served by actually checking for what you mean. The difference is in intent and likelihood of confusion. For all of those you?re going to be explicitly testing and working with midnight objects, so you?re going to test them, you?re going to exercise that code. In the ``is None`` case, Midnight is just another value so that bug will lay dormant and undetected until someone just happens to hit an unlucky time. And once you get that it?s likely to be difficult to reproduce. Further more this only ever works reliably on naive times and doesn?t realistically work ?sanely? (if you consider this behavior sane) on aware times at all. So to be specific, I think using ``if time_or_something_that_may_be_none:`` is always wrong, but I think the like hood for user confusion and hard to reproduce bugs are far greater when a more or less arbitrary time (especially in the sense of aware times) evaluates as false when a user omits the ``is None`` than when someone expects ``if time:`` to be false at midnight. > > Now, those use cases may still not appear relevant to you, but then > the argument that "if x:" should not trigger for time(0,0,0) > because x might actually be None, is also not relevant to me, > since it masks a bug in the example code and those should never > go undetected :-) > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 06 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, >>>> mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/75cb9ab0/attachment.sig> From donald at stufft.io Thu Mar 6 09:50:46 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 03:50:46 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> Message-ID: <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io> On Mar 6, 2014, at 3:46 AM, Donald Stufft <donald at stufft.io> wrote: > So to be specific, I think using ``if time_or_something_that_may_be_none:`` > is always wrong, but I think the like hood for user confusion and hard to > reproduce bugs are far greater when a more or less arbitrary time (especially > in the sense of aware times) evaluates as false when a user omits the > ``is None`` than when someone expects ``if time:`` to be false at midnight. I realized this was ambiguous, I always consider directly checking the boolean value of a time object as wrong, as I don?t believe time has any True/False implications in it. This includes both using ``if thing:`` to detect None and using ``if thing:`` to detect midnight. I just think the ``if thing:`` to detect None case is more common and also more difficult to detect when you have a bug because of it and more surprising behavior than the other case. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/4b6b5934/attachment-0001.sig> From flying-sheep at web.de Thu Mar 6 10:22:08 2014 From: flying-sheep at web.de (Philipp A.) Date: Thu, 6 Mar 2014 10:22:08 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> <22932C54-DCDF-4CF3-B91B-DB69B4FDA13A@stufft.io> Message-ID: <CAN8d9gngoVTnEgiE+X2aJKkXz+oX+9cqT7sqghAYF2ngxMmB1Q@mail.gmail.com> 2014-03-06 6:34 GMT+01:00 Greg Ewing <greg.ewing at canterbury.ac.nz>: > The point of having time objects as an abstraction is > so that you *don't* have to think of them as an offset > from anything. You simply think of them as times. but times *are* offsets from midnight (24h clock) or midnight and noon (12h clock) 1 o?clock = 1 hour after midnight 12 o?clock pm = 12 hours after noon they are essentially radial coordinates ? and radial coordinates do have a 0 point. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/cdf0ebcc/attachment.html> From mal at egenix.com Thu Mar 6 10:32:06 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 10:32:06 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> Message-ID: <53184096.3050409@egenix.com> On 06.03.2014 09:46, Donald Stufft wrote: > > On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote: > >> On 06.03.2014 03:23, Donald Stufft wrote: >>> >>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: >>> >>>> [Donald Stufft] >>>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that >>>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile. >>>> >>>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that >>>> no code relies on the current behavior? For that matter, where is the evidence that the >>>> current behavior is a significant cause of bugs? I know I've used "if some_time_object:", >>>> but it did exactly what I expected it to do. >>> >>> Forgive me if I?m wrong, but aren?t you the author of the date time module? If that?s the case >>> what you expect it to do isn?t particularly relevant as you?re intimately aware of it?s >>> implementation. >> >> Just in case you're looking for a relevant use case: >> >> Having bool(time(0,0,0)) allows you to quickly check whether you are >> dealing with a datetime value with (non-trivial) time part or not. > > I don?t see how midnight is any more or less trivial than 12:01. > >> >> You run into situations where you have to test for this in >> date/time conversions, arithmetics, etc. e.g. if you need to convert >> a datetime value to a date and want to prevent truncation of >> information, or if you want to format a datetime value in a user >> friendly way by omitting the zero time part, or if you're doing >> datetime arithmetic that has to follow special requirements w/r to >> days (think interest or insurance math) and you want to >> quickly check whether you can use the fast path simple >> calculation, or you need to do the full blown complicated part. > > I think these would be better served by actually checking for what you mean. > For the record I also think that checking for a date time to not be None > is also better served by actually checking for what you mean. The > difference is in intent and likelihood of confusion. FWIW, I expect bool(time(0,0,0)) == False, just like I expect bool(0.0) == False. I would find it confusing to have bool(zero_element) == True. > For all of those you?re going to be explicitly testing and working with > midnight objects, so you?re going to test them, you?re going to exercise > that code. In the ``is None`` case, Midnight is just another value so that > bug will lay dormant and undetected until someone just happens to > hit an unlucky time. And once you get that it?s likely to be difficult to > reproduce. datetime values with zero time are *very* common in practice (you run into them whenever you convert a date value into a datetime value), so the "is None" case is easy to reproduce and will hit you more often than you like. BTW: Not using "is None" will bite you in many other ways as well. None it typically used as value for "not initialized" or "no value provided". This meaning is completely different from "empty selection" or "empty string", so in order to address all those cases as well, you'd have to change: bool(()) == True bool('') == True etc. The "if x is None:" test is also faster than "if x:", so actually win something by coding correctly ;-) Please provide a stronger argument for needing to change bool(time(0,0,0)) than one which is built on buggy code :-) (I'm running out of smileys...) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at gmail.com Thu Mar 6 10:34:32 2014 From: fuzzyman at gmail.com (Michael Foord) Date: Thu, 6 Mar 2014 09:34:32 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53184096.3050409@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> <53184096.3050409@egenix.com> Message-ID: <CAKCKLWzWyhsxhxT2tvoF4KrUAF5okYp1aTqUZ96C3X5T90kGuQ@mail.gmail.com> On 6 March 2014 09:32, M.-A. Lemburg <mal at egenix.com> wrote: > On 06.03.2014 09:46, Donald Stufft wrote: > > > > On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote: > > > >> On 06.03.2014 03:23, Donald Stufft wrote: > >>> > >>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: > >>> > >>>> [Donald Stufft] > >>>>> When the documented behavior is both nonsensical and the cause of > hard to debug bugs that > >>>>> is a pretty compelling use case to me, unless you actively enjoy > being user hostile. > >>>> > >>>> Breaking code that currently works is as hostile as hostile gets. > Where is the evidence that > >>>> no code relies on the current behavior? For that matter, where is the > evidence that the > >>>> current behavior is a significant cause of bugs? I know I've used > "if some_time_object:", > >>>> but it did exactly what I expected it to do. > >>> > >>> Forgive me if I'm wrong, but aren't you the author of the date time > module? If that's the case > >>> what you expect it to do isn't particularly relevant as you're > intimately aware of it's > >>> implementation. > >> > >> Just in case you're looking for a relevant use case: > >> > >> Having bool(time(0,0,0)) allows you to quickly check whether you are > >> dealing with a datetime value with (non-trivial) time part or not. > > > > I don't see how midnight is any more or less trivial than 12:01. > > > >> > >> You run into situations where you have to test for this in > >> date/time conversions, arithmetics, etc. e.g. if you need to convert > >> a datetime value to a date and want to prevent truncation of > >> information, or if you want to format a datetime value in a user > >> friendly way by omitting the zero time part, or if you're doing > >> datetime arithmetic that has to follow special requirements w/r to > >> days (think interest or insurance math) and you want to > >> quickly check whether you can use the fast path simple > >> calculation, or you need to do the full blown complicated part. > > > > I think these would be better served by actually checking for what you > mean. > > For the record I also think that checking for a date time to not be None > > is also better served by actually checking for what you mean. The > > difference is in intent and likelihood of confusion. > > FWIW, I expect bool(time(0,0,0)) == False, just like I expect > bool(0.0) == False. I would find it confusing to have > bool(zero_element) == True. > > > For all of those you're going to be explicitly testing and working with > > midnight objects, so you're going to test them, you're going to exercise > > that code. In the ``is None`` case, Midnight is just another value so > that > > bug will lay dormant and undetected until someone just happens to > > hit an unlucky time. And once you get that it's likely to be difficult to > > reproduce. > > datetime values with zero time are *very* common in practice > (you run into them whenever you convert a date value into a datetime > value), so the "is None" case is easy to reproduce and will hit > you more often than you like. > > BTW: Not using "is None" will bite you in many other ways as > well. None it typically used as value for "not initialized" or > "no value provided". This meaning is completely different from > "empty selection" or "empty string", so in order to address all > those cases as well, you'd have to change: > > bool(()) == True > bool('') == True > etc. > > The "if x is None:" test is also faster than "if x:", so actually > win something by coding correctly ;-) > > Please provide a stronger argument for needing to change > bool(time(0,0,0)) than one which is built on buggy code :-) > (I'm running out of smileys...) > > I'm very surprised that "midnight evaluates to False is insane" is not in and of itself a strong enough argument to change it. :-/ Michael > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 06 2014) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f31ac7ce/attachment-0001.html> From solipsis at pitrou.net Thu Mar 6 10:34:55 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 10:34:55 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> Message-ID: <lf9ffh$8ml$1@ger.gmane.org> Le 06/03/2014 03:15, Tim Peters a ?crit : > [Donald Stufft] >> When the documented behavior is both nonsensical and the cause of hard >> to debug bugs that is a pretty compelling use case to me, unless you actively >> enjoy being user hostile. > > Breaking code that currently works is as hostile as hostile gets. > Where is the evidence that no code relies on the current behavior? > For that matter, where is the evidence that the current behavior is a > significant cause of bugs? I know I've used "if some_time_object:", > but it did exactly what I expected it to do. The current behaviour being highly unintuitive is a likely source of bugs, IMO. Regards Antoine. From solipsis at pitrou.net Thu Mar 6 10:52:12 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 10:52:12 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318336F.7030009@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> Message-ID: <lf9gft$jnh$1@ger.gmane.org> Le 06/03/2014 09:35, M.-A. Lemburg a ?crit : > Just in case you're looking for a relevant use case: > > Having bool(time(0,0,0)) allows you to quickly check whether you are > dealing with a datetime value with (non-trivial) time part or not. This sounds like the kind of shortcut that only an expert would know to take, not something that's actually a good design decision. It's as though pathlib.Path('') evaluated to false for some obscure use case that I cared about. It is not a coincidence, IMO, that the three persons arguing that the current behaviour is reasonable are implementors of datetime modules: you, Tim and Alexander. All other people find the behaviour baffling. Regards Antoine. From ncoghlan at gmail.com Thu Mar 6 11:11:21 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 6 Mar 2014 20:11:21 +1000 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <lf8rh3$suv$1@ger.gmane.org> References: <lf8rh3$suv$1@ger.gmane.org> Message-ID: <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> On 6 Mar 2014 13:55, "Sturla Molden" <sturla.molden at gmail.com> wrote: > > "goto fail" is a well-known error handling mechanism in open source software, widely reputed for its robusteness: > > http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c > > https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c > > I believe Python needs to add support for this superior paradigm. Unfortunately, we can't throw stones about those, since we currently have hostname matching off by default and expect developers to turn it on. That needs to change, but there are a few thorny practical issues to sort out in order to get there (Christian Heimes has been working through several of them). Cheers, Nick. > > It would involve a new keyword "fail" and some means of goto'ing to it. I suggest "raise to fail": > > if (some_error): > raise to fail > > fail: > <error handling code> > > Unless there are many objections, this fantastic idea might be submitted in a (short) PEP somewhere around the beginning of next month. > > There is some obvious overlap with the rejected "goto PEP" (PEP 3163) and the Python 2.3 goto module. However, the superiority of goto fail as error generation and error handling paradigm has since then been thoroughly proven. > > http://legacy.python.org/dev/peps/pep-3136/ > http://entrian.com/goto/download.html > > > Regards, > Sturla Molden > > > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/a6fbc1f8/attachment.html> From solipsis at pitrou.net Thu Mar 6 11:13:08 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:13:08 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAP7h-xbETLGZTPgRjz4hxwBPCycrOsYuQ0o+_wT9-WYfQhe0aA@mail.gmail.com> Message-ID: <lf9hn6$56a$1@ger.gmane.org> Le 06/03/2014 07:47, Alexander Belopolsky a ?crit : > I don't understand why people are denying that midnights are special > points. Because they are not? Really, this is getting ridiculous. In any intuitive and common notion of time, midnight is not a special point at all. There is a reason most people on this thread disagree with you. (and, no, it's not because they are being dense or ignorant) Regards Antoine. From solipsis at pitrou.net Thu Mar 6 11:19:46 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:19:46 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <loom.20140305T220007-633@post.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> Message-ID: <lf9i3j$56a$2@ger.gmane.org> Le 05/03/2014 22:10, Alex Gaynor a ?crit : > First, I'd like to dispense with the notion that ``if foo`` is somehow a bad > practice. This is an EXTREMELY common practice as a shorthand for checking for > None, a great many people consider it more idiomatic, this really isn't a place > to bikeshed that particlar idiom. Agreed. If we didn't want people to rely on that idiom then __bool__ would have to raise TypeError on most types except a select ones. Regards Antoine. From solipsis at pitrou.net Thu Mar 6 11:27:22 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:27:22 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306021457.GF28804@ando> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <B7913D2A-A272-4B2F-A180-2FA5BC64BCB8@stufft.io> <20140306021457.GF28804@ando> Message-ID: <lf9ihs$fos$1@ger.gmane.org> Le 06/03/2014 03:14, Steven D'Aprano a ?crit : > > For what it's worth, I think the current behaviour is a misfeature, and > in a perfect world it would be fixed. But the cost of this is so > miniscule, and the work-around so trivial, that although the fix is > cheap, the benefit is even lower. Again, the problem is not the cost of the workaround, but the cost of sporadic unexpected failures when you don't *know* your code is vulnerable to this issue. Regards Antoine. From mal at egenix.com Thu Mar 6 11:33:08 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 11:33:08 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9gft$jnh$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> Message-ID: <53184EE4.80105@egenix.com> On 06.03.2014 10:52, Antoine Pitrou wrote: > Le 06/03/2014 09:35, M.-A. Lemburg a ?crit : >> Just in case you're looking for a relevant use case: >> >> Having bool(time(0,0,0)) allows you to quickly check whether you are >> dealing with a datetime value with (non-trivial) time part or not. > > This sounds like the kind of shortcut that only an expert would know to take, not something that's > actually a good design decision. > It's as though pathlib.Path('') evaluated to false for some obscure use case that I cared about. It doesn't ? :-) > It is not a coincidence, IMO, that the three persons arguing that the current behaviour is > reasonable are implementors of datetime modules: you, Tim and Alexander. All other people find the > behaviour baffling. I'm not so sure. Of course, people who have been doing date/time stuff will jump into discussions that deal with them, but the same argument triggering this thread could have applied to other Python types and you would then have a different group of people argue for plausibility of the choice. I find it strange that the only argument against having time(0,0,0) evaluate to False in a boolean context is a wrongly used None test. This argument can be applied to lots of other types as well, where the buggy None test masks errors. Perhaps someone could present at least one compelling argument that is not based on wrong usage of None in if statements. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Thu Mar 6 11:39:20 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:39:20 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> Message-ID: <lf9j8a$ole$1@ger.gmane.org> Le 06/03/2014 00:22, Alexander Belopolsky a ?crit : > > No. It is False because it equals False: > > >>> 0 == False > True I think this is a misunderstanding on your part. Python considered 0 as false-y before the bool type was introduced in the language. Actually, 0 was *the* spelling of "false" before "False" was introduced. > I think proponents of bool(time(0)) == True view it as a container of > hours, minutes and seconds. No, they view it as an abstraction around the common concept of time. Regards Antoine. From solipsis at pitrou.net Thu Mar 6 11:43:12 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:43:12 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306021948.GG28804@ando> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <20140306021948.GG28804@ando> Message-ID: <lf9jfh$ole$2@ger.gmane.org> Le 06/03/2014 03:19, Steven D'Aprano a ?crit : > > It may only be a convention that an instant before midnight is the end > of the day and midnight the beginning of the next, but it is a > convention: midnight is the origin (i.e. zero point) of the day. That > makes it arguably as falsey as 0, 0.0 and 0j. A zero in algebra has a special meaning with respect to an operator (e.g. addition). Midnight doesn't have such a special meaning. Regards Antoine. From solipsis at pitrou.net Thu Mar 6 11:46:13 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 11:46:13 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53184EE4.80105@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> Message-ID: <lf9jl7$ole$3@ger.gmane.org> Le 06/03/2014 11:33, M.-A. Lemburg a ?crit : > > I find it strange that the only argument against having time(0,0,0) > evaluate to False in a boolean context is a wrongly used > None test. It's only "wrongly used" because of that particular wart. In other words, the "wrongly used" None test *exposes* the wart, it is not its *cause*. There are other contexts with implicit boolean conversion, such as any() and all(). Regards Antoine. From steve at pearwood.info Thu Mar 6 11:52:53 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 21:52:53 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5317FC67.7060600@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> Message-ID: <20140306105253.GI28804@ando> On Thu, Mar 06, 2014 at 05:41:11PM +1300, Greg Ewing wrote: > Steven D'Aprano wrote: > >There's also the argument from consistency: midnight is the zero point > >of the day, and zero is falsey. > > Regarding midnight as a "zero point" is an arbitrary > convention. True, as I mentioned in another post. Nevertheless, we have many arbitrary conventions enshrined in Python. Using base 10 for the representation of numbers, instead of, say, balanced ternary notation, is an arbitrary convention. Numbering sequences from 0 is another. If you would prefer another convention for the zero point of the day (say, midday, dusk, dawn, or 2:15pm) then feel free to subclass datetime.time :-) Or even no zero point at all. Tossing out a wild idea, why not leave datetime.time alone and add a new class which is exactly the same but is always truthy? That doesn't break backward compatibility, and so it could happen immediately (too late for 3.4, but certainly for 3.5). All we need is an appropriate name, and a decision as to which ought to inherit from which. The bike-shedding should be over by 3.5 alpha 1 *wink* Despite my quip about the bike-shedding, I am actually serious. -- Steven From masklinn at masklinn.net Thu Mar 6 11:53:59 2014 From: masklinn at masklinn.net (Masklinn) Date: Thu, 6 Mar 2014 11:53:59 +0100 Subject: [Python-ideas] One more time... lambda function <--- from *** signature def. In-Reply-To: <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <leqgah$b0o$1@ger.gmane.org> <20140301034611.GD28804@ando> <CAEgL-feoHwv-148QWbi+x8HwHUuSpj2ELtEqtbdhWzhHczZxEQ@mail.gmail.com> <84C450A1-D2C8-4103-9933-F98747CDEFA0@yahoo.com> <20140304110921.GP28804@ando> <5316544D.7050302@canterbury.ac.nz> <05894993-3C4D-42E7-9328-C3365DF92CBE@masklinn.net> <53179C51.4020001@canterbury.ac.nz> <29275C71-F3DC-4D4A-A251-B56AED14E6C1@masklinn.net> <87y50npyy7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4E54B6BB-2AA6-4D57-96BC-3C7D4C807030@masklinn.net> On 2014-03-06, at 08:29 , Stephen J. Turnbull <stephen at xemacs.org> wrote: > Masklinn writes: >> On 2014-03-05, at 22:51 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >>> Masklinn wrote: >>>> On 2014-03-04, at 23:31 , Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > >>> That comes at the expense of making *everything* a thunk >>> until its value is needed [in Haskell]. >> >> Of course not, Haskell allows explicitly forcing a thunk and the >> compiler/runtime pair can use strictness analysis and not create thunks >> in the first place. > > That doesn't seem fair to me. "Explicitly forcing" by definition > means the programmer has decided the value is needed No, it means the programmer wants the value to be strictly evaluated (this is generally a space optimisation). It has no impact on program behavior (assuming infinite resources, one issue of accumulating thunks is the risk of OOM if they remain live but unevaluated). > and "strictness > analysis" is an optimization, not part of the language definition. Am > I missing something? No? It just demonstrates that not everything is a thunk in haskell, there are both thunks and strict values living around in the runtime, and that's not visible in the type system. >> Consider that a thunk is a deferral of an expression's evaluation, >> why would said expression's evaluation happen more than once? The >> only thing which changes is *when* the actual evaluation happens. > > Are thunks guaranteed not to leak out of the scope of the > containing expression, so assignment is impossible? I'm not sure I understand the question. > If they can leak, the thunk would be an object I think that's just one possible implementation, a thunk could also be some sort of reference indirection, a tagged pointer, or a deeply integrated proxy-ish object. > and in Python > "assigning" it to a "variable" actually just creates a reference. > Evaluation could memoize the thunk's value ("evaluation only happens > once") or reevaluate the thunk each time its value is requested. > Either seems potentially surprising to me. Absolutely. > Memoization makes sense if you think of the thunk as the closure of a > computation that conceptually takes place at definition time. I'm not > sure if in the use cases for thunks it really makes sense to "close" > the thunk at evaluation time, but it seems like a plausible > interpretation to me. I find it the most *useful* definition: for the other one we already have functions and I'm not sure thunks would have much use as shorter arguments-less functions. It also has the property that "client" code (code receiving thunks without necessarily being aware of it) remains valid and unsurprising in that a = foo b = foo assert a is b remains true regardless of `foo` being a thunk (the only difference being that if `foo` is a thunk it may remain unforced throughout). From p.f.moore at gmail.com Thu Mar 6 11:57:22 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 6 Mar 2014 10:57:22 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53184EE4.80105@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> Message-ID: <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com> On 6 March 2014 10:33, M.-A. Lemburg <mal at egenix.com> wrote: >> It is not a coincidence, IMO, that the three persons arguing that the current behaviour is >> reasonable are implementors of datetime modules: you, Tim and Alexander. All other people find the >> behaviour baffling. > > I'm not so sure. Of course, people who have been doing date/time > stuff will jump into discussions that deal with them, but the same > argument triggering this thread could have applied to other Python > types and you would then have a different group of people argue > for plausibility of the choice. I had promised to stay out of this discussion as I no longer thing it adds much value (note that the bug has been reopened long since) and I was getting tired of having people decide that I'd said something I didn't mean. But just to add a perspective from someone who *doesn't* develop a datetime module, I encounter "special" midnight values very often. Databases (at least Oracle, and I believe others) typically store date values ("Thursday 6th March 2014") as a timestamp value with midnight as a time part. So checking the time part of a datetime value for midnight as a "special, means omitted" value is not uncommon in that domain. Paul From stephen at xemacs.org Thu Mar 6 11:57:27 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 19:57:27 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403052118.38295.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403052033.44162.shai@platonix.com> <lf7rf0$e7m$1@ger.gmane.org> <201403052118.38295.shai@platonix.com> Message-ID: <87vbvrppbc.fsf@uwakimon.sk.tsukuba.ac.jp> Shai Berger writes: > Unless you're aware of the issue, you are not going to unit-test > specifically for midnight. If you are aware, you don't need to > detect it, you write "is not None". The issue, as Skip has argued at length, is that by using the implicit cast to Boolean you've changed the nature of the test from "membership in valid values" to "a true-ish value".[1] This can be detected generically with grep, something like: grep -E '^ *if [^<>=]*:' | grep -v '^ *if.* is .*:' with a (possibly large) number of false positives. (I suspect it's not worth worrying about no false negatives; if the 'is' operator is present, the test is boolean except if it's part of an arithmetic expression such as "3 + (2 is not None)".) You might want to include modulo (%) in the list of operators. Or it might be more accurate to check for bare variables only and assume any expression should OK. The false positives can be drastically reduced by checking for a Boolean or container initializer in the case of "if var:". None of pep8, pylint, or pychecker does this now AFAICT, but they could, I think. Footnotes: [1] Implementing a membership test as a test for a sentinel is a bit fishy in this context of arguing about the "nature of the test", but given the much less obvious nature of the conversion to bool in complex types like dates and times, I'll go with Skip any time. From mal at egenix.com Thu Mar 6 12:04:56 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 12:04:56 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9jl7$ole$3@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> Message-ID: <53185658.5080507@egenix.com> On 06.03.2014 11:46, Antoine Pitrou wrote: > Le 06/03/2014 11:33, M.-A. Lemburg a ?crit : >> >> I find it strange that the only argument against having time(0,0,0) >> evaluate to False in a boolean context is a wrongly used >> None test. > > It's only "wrongly used" because of that particular wart. > In other words, the "wrongly used" None test *exposes* the wart, it is not its *cause*. Wait. Let's be clear on this: Writing if x: print ('x is None') or if x == None: print ('x is None') is wrong code. None is a singleton, so you have to use the "is" operator, i.e. if x is None: print ('x is None') is the only correct way of testing for None. Adding a test to pylint for this would probably be more helpful and address the real problem in a more productive way, than the current discussion to making the above problematic code work in the single use case of x being a datetime.time object. > There are other contexts with implicit boolean conversion, such as any() and all(). If you expect those to work correctly for None testing, then you're writing wrong code as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Thu Mar 6 12:09:12 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 12:09:12 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185658.5080507@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> Message-ID: <lf9l09$9nv$1@ger.gmane.org> Le 06/03/2014 12:04, M.-A. Lemburg a ?crit : > > Writing > > if x: print ('x is None') > > or > > if x == None: print ('x is None') > > is wrong code. No it isn't. > None is a singleton, so you have to use the "is" operator, i.e. > > if x is None: print ('x is None') > > is the only correct way of testing for None. No it isn't. x == None is a perfectly well-defined way of doing it, even if it isn't the stylistically preferred one. Regards Antoine. From stefan at bytereef.org Thu Mar 6 12:13:36 2014 From: stefan at bytereef.org (Stefan Krah) Date: Thu, 6 Mar 2014 11:13:36 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Please_reconsider_the_Boolean_evaluation?= =?utf-8?q?_of=09midnight?= References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> Message-ID: <loom.20140306T120303-396@post.gmane.org> Antoine Pitrou <solipsis at ...> writes: > Le 06/03/2014 09:35, M.-A. Lemburg a ?crit : > > Just in case you're looking for a relevant use case: > > > > Having bool(time(0,0,0)) allows you to quickly check whether you are > > dealing with a datetime value with (non-trivial) time part or not. > > This sounds like the kind of shortcut that only an expert would know to > take, not something that's actually a good design decision. > It's as though pathlib.Path('') evaluated to false for some obscure use > case that I cared about. On the other hand I would not dare to assume it's True without checking. > It is not a coincidence, IMO, that the three persons arguing that the > current behaviour is reasonable are implementors of datetime modules: > you, Tim and Alexander. All other people find the behaviour baffling. Not all. :) I think viewing midnight as (unix_time % 86400) is quite reasonable. Stefan Krah From mal at egenix.com Thu Mar 6 12:15:47 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 12:15:47 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185658.5080507@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> Message-ID: <531858E3.3030805@egenix.com> On 06.03.2014 12:04, M.-A. Lemburg wrote: > On 06.03.2014 11:46, Antoine Pitrou wrote: >> Le 06/03/2014 11:33, M.-A. Lemburg a ?crit : >>> >>> I find it strange that the only argument against having time(0,0,0) >>> evaluate to False in a boolean context is a wrongly used >>> None test. >> >> It's only "wrongly used" because of that particular wart. >> In other words, the "wrongly used" None test *exposes* the wart, it is not its *cause*. > > Wait. Let's be clear on this: > > Writing > > if x: print ('x is None') The above is obviously wrong :-). Let's try again: if not x: print ('x is None') > or > > if x == None: print ('x is None') > > is wrong code. > > None is a singleton, so you have to use the "is" operator, i.e. > > if x is None: print ('x is None') > > is the only correct way of testing for None. > > Adding a test to pylint for this would probably be more helpful > and address the real problem in a more productive way, than the > current discussion to making the above problematic code work in > the single use case of x being a datetime.time object. > >> There are other contexts with implicit boolean conversion, such as any() and all(). > > If you expect those to work correctly for None testing, then > you're writing wrong code as well. > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Thu Mar 6 12:22:47 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 12:22:47 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9l09$9nv$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <lf9l09$9nv$1@ger.gmane.org> Message-ID: <53185A87.5010109@egenix.com> On 06.03.2014 12:09, Antoine Pitrou wrote: > Le 06/03/2014 12:04, M.-A. Lemburg a ?crit : >> >> Writing >> >> if x: print ('x is None') >> >> or >> >> if x == None: print ('x is None') >> >> is wrong code. > > No it isn't. > >> None is a singleton, so you have to use the "is" operator, i.e. >> >> if x is None: print ('x is None') >> >> is the only correct way of testing for None. > > No it isn't. x == None is a perfectly well-defined way of doing it, even if it isn't the > stylistically preferred one. Depends on what kind of type x is and how that type implements the comparison slots. It is perfectly well possible to define a type that returns True for (x == None), even though x is not None :-) >>> class C: ... def __eq__(self, other): ... return (other is None) ... >>> o = C() >>> o == None True -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rob.cliffe at btinternet.com Thu Mar 6 12:36:40 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 06 Mar 2014 11:36:40 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com> Message-ID: <53185DC8.3000707@btinternet.com> On 06/03/2014 10:57, Paul Moore wrote: > > I had promised to stay out of this discussion as I no longer thing it > adds much value (note that the bug has been reopened long since) and I > was getting tired of having people decide that I'd said something I > didn't mean. > > But just to add a perspective from someone who *doesn't* develop a > datetime module, I encounter "special" midnight values very often. > Databases (at least Oracle, and I believe others) typically store date > values ("Thursday 6th March 2014") as a timestamp value with midnight > as a time part. So checking the time part of a datetime value for > midnight as a "special, means omitted" value is not uncommon in that > domain. > > Paul You're joking, right? How do you distinguish "omitted" from "provided, equals midnight" ? Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14 > > From rob.cliffe at btinternet.com Thu Mar 6 12:40:21 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 06 Mar 2014 11:40:21 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306105253.GI28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando> Message-ID: <53185EA5.3060802@btinternet.com> On 06/03/2014 10:52, Steven D'Aprano wrote: > > True, as I mentioned in another post. > > Nevertheless, we have many arbitrary conventions enshrined in Python. > Using base 10 for the representation of numbers, instead of, say, > balanced ternary notation, is an arbitrary convention. Numbering > sequences from 0 is another. > > If you would prefer another convention for the zero point of the day > (say, midday, dusk, dawn, or 2:15pm) then feel free to subclass > datetime.time :-) > > Or even no zero point at all. Tossing out a wild idea, why not leave > datetime.time alone and add a new class which is exactly the same but is > always truthy? That doesn't break backward compatibility, and so it > could happen immediately (too late for 3.4, but certainly for 3.5). > > All we need is an appropriate name, and a decision as to which ought to > inherit from which. The bike-shedding should be over by 3.5 alpha 1 > *wink* > > Despite my quip about the bike-shedding, I am actually serious. You're joking, right? Suppose you were considering taking up a new language and you found something in the docs which said: "Here's a class that does so-and-so. And here's another slightly different one that you can use instead, because we didn't get the first one quite right." Would that be a turn-on? Rob Cliffe > > From stephen at xemacs.org Thu Mar 6 12:51:29 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 20:51:29 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <201403051924.05252.shai@platonix.com> References: <201403051216.12392.shai@platonix.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <201403051924.05252.shai@platonix.com> Message-ID: <87txbbpmta.fsf@uwakimon.sk.tsukuba.ac.jp> Shai Berger writes: > At issue here is the fact that the idiom doesn't fit where one > would expect that it should. This is the real problem. People are programming in un-Pythonic style by analogy to either other languages or to other cases in Python where the __len__ method provides a natural cast to Boolean, and deciding since they like the idiom, the behavior of time.time is "unexpected." But your claim is *not* a "fact". Quite the reverse, I was mildly surprised even the last time around that anybody would argue that it's a good idea to use a sentinel to indicate "uninitialized" and then fail to test explicitly for the sentinel. For example, I wouldn't be surprised to see something like this in an MTA or other program that should not let data out of its sight without a timestamp: my_time = None if not my_time and x: my_time = x.supposed_to_be_time_but_in_fact_sometimes_0 if not my_time: my_time = time.now() What are the odds that you'll detect the bug in initializing 'x' in a timely fashion? OTOH, if you test for "is None" you'll probably get a quick TypeError the first time the bug manifests. (No guarantees, of course, but why trade a potentially easy bug for a probably hard one?) If you asked me to implement a Time class, would I have any false instances? Quite possibly, yes. Of course I might implement it as a namedtuple (h, m, s), in which case it would always be true. OTOH, I might implement it as seconds-since-midnight, and probably not bother to give it a __bool__ method to ensure bool(midnight) == True. So I don't really care *why* Uncle Timmy chose to make midnight be false, I only offer kudos for documenting it. :-) From p.f.moore at gmail.com Thu Mar 6 12:54:34 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 6 Mar 2014 11:54:34 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185DC8.3000707@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <CACac1F82rrVAm_R1aA0=tge0OYY5bDMkNQcN9ygrh-92f+ba=w@mail.gmail.com> <53185DC8.3000707@btinternet.com> Message-ID: <CACac1F-Ed0A-5PNp6b+VVs9CzeMDYYGq_H8=mHW_N2P4LkEQcw@mail.gmail.com> On 6 March 2014 11:36, Rob Cliffe <rob.cliffe at btinternet.com> wrote: > You're joking, right? No. You just sent 2 "you're joking" messages. You're joking, right? Any time anyone presents a use case or proposal that contradicts what you want it must be a joke? I really am sick of being misinterprested and misrepresented on this thread. > How do you distinguish "omitted" from "provided, equals midnight" ? I will not explain all the details of how database applications process dates here. All I offered was a perspective that some application domains, in certain cases, see "midnight" as having a special meaning distinct from the specific point in time. I did so because other people's views were being characterised as irrelevant because they developed datetime modules (as if that made them somehow *less* qualified to have an opinion) and I sympathised with them over the hostile reception they were getting. I didn't comment on whether that meant the Python behaviour was good or bad, or whether the practice of truth-testing time values was good or not. Feel free to investigate the details if you care, or ignore my post if you don't. But I'm not trolling, nor do I appreciate the implication that I am. Paul. From rob.cliffe at btinternet.com Thu Mar 6 12:56:31 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 06 Mar 2014 11:56:31 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> Message-ID: <5318626F.5080203@btinternet.com> On 06/03/2014 02:15, Tim Peters wrote: > [Donald Stufft] >> When the documented behavior is both nonsensical and the cause of hard >> to debug bugs that is a pretty compelling use case to me, unless you actively >> enjoy being user hostile. > Breaking code that currently works is as hostile as hostile gets. > Where is the evidence that no code relies on the current behavior? > For that matter, where is the evidence that the current behavior is a > significant cause of bugs? I know I've used "if some_time_object:", > but it did exactly what I expected it to do. Well, let's start a survey. So far we have: Instances of existing code that would be broken by the change: 0 Instances of code where the unexpected behaviour caused a bug: 1 (the OP's code). I invite everyone to add instances that they know about.:-) Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > ----- > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6655 - Release Date: 03/05/14 > > From ncoghlan at gmail.com Thu Mar 6 13:08:54 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 6 Mar 2014 22:08:54 +1000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185658.5080507@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> Message-ID: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> On 6 March 2014 21:04, M.-A. Lemburg <mal at egenix.com> wrote: > Wait. Let's be clear on this: > > Writing > > if x: print ('x is None') > > or > > if x == None: print ('x is None') The case in question is essentially this one: if x: assert x is not None # Always valid! .... else: assert x is None # Valid for most user defined types There is a learned intuition that people naturally acquire when learning Python: - numbers may be false (it means zero) - containers may be false (it means empty) - everything else is always true (as that's the default for user defined classes) Where datetime().time() is confusing is the fact the expected behaviour changes based on *which* category you put the number in. The vast majority of Python's users will place structured date and time objects in the "arbitrary object" category, and expect them to always be true. When using None as a sentinel for such types, writing "if x:" when you really mean "if x is not None:" is a harmless style error, with no practical ill-effects. When dealing with time objects, such users are unlikely to be crafting test cases to ensure that "midnight UTC" is handled correctly, so their "harmless style error" is in fact a subtle data driven bug waiting to bite them. It is *really* hard for a static analyser to pick this up, because at point of use "if x:" gives no information about the type of "x", and hence any such alert would have an unacceptably high false positive rate. Now, let's consider the time with the *best* possible claim to being false: timestamp zero. How does that behave? Python 3.3: >>> bool(dt.datetime.fromtimestamp(0)) True >>> bool(dt.datetime.fromtimestamp(0).date()) True >>> bool(dt.datetime.fromtimestamp(0).time()) True Huh, if times are supposed to be valid as truth values, that looks rather weird. What's going on? >>> dt.datetime.fromtimestamp(0).time() datetime.time(10, 0) Oh, I'm in Brisbane - *of course* the truthiness of timestamps should depend on my timezone! Clearly, what I really meant was timestamp -36000, or perhaps 50400: >>> bool(dt.datetime.fromtimestamp(-36000).time()) False >>> bool(dt.datetime.fromtimestamp(50400).time()) False So, unless I happen to live in UTC, it's highly unlikely that I'm going to infer from Python's *behaviour* that datetime.time() (unlike datetime.date() and datetime.datetime()) belong in the "number" category, rather than the "arbitrary object" category. Perhaps it behaves like a number in other ways: >>> utcmidnight = dt.datetime.fromtimestamp(50400).time() >>> utcmidnight + 1 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'datetime.time' and 'int' >>> utcmidnight * 1 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for *: 'datetime.time' and 'int' >>> int(utcmidnight) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: int() argument must be a string or a number, not 'datetime.time' Hmm, nope. And that last one *explicitly* tells me it's not a number! There's a great saying in the usability world: "You can't document your way out of a usability problem". What it means is that if all the affordances of your application (or programming language!) push users towards a particular logical conclusion (in this case, "datetime.time values are not numbers"), having a caveat in your documentation isn't going to help, because people aren't even going to think to ask the question. It doesn't matter if you originally had a good reason for the behaviour, you've ended up in a place where your behaviour is confusing and inconsistent, because there is one piece of behaviour that is out of line with an otherwise consistent mental model. But perhaps I've been told "midnight is false in boolean context". But which midnight? There are three that apply to me: >>> naivemidnight datetime.time(0, 0) >>> utcmidnight datetime.time(0, 0, tzinfo=datetime.timezone.utc) >>> localmidnight datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000))) Are they all False? No, no they're not (unless your local timezone is UTC): >>> bool(utcmidnight) False >>> bool(naivemidnight) False >>> bool(localmidnight) True There's a phrase for APIs like this one: "expert friendly". Experts like a particular behaviour because it lets them do advanced things (like leave the door open for modular arithmetic on timestamp.time() values). However, that's not normally something we see as a virtue when designing APIs for Python - instead, we generally aim for layered complexity, where simple things are simple, and we provide power tools for advanced users that want them. Now, suppose this boolean behaviour went away. How would I detect if a value was midnight or not? Well, first, I would need to clarify my question. Do I mean midnight UTC? Or do I mean midnight local time? It makes a difference for aware objects, after all. Or perhaps I'm not interested in aware objects at all, and only care about naive midnight. Using appropriately named values like those I defined above, those questions are all very easy to express explicitly in ways that don't rely on readers understanding that "bool(x)" on a datetime.time object means the same thing as "x is naive midnight or UTC midnight": if x not in (naivemidnight, utcmidnight): # This is the current bool(x) .... if x != naivemidnight: # This has no current shorthand .... if x != utcmidnight: # This has no current shorthand .... if x != localmidnight: # This has no current shorthand .... And just to demonstrate that equivalence is accurate: >>> local_10am = localmidnight.replace(hour=10) >>> local_10am datetime.time(10, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000))) >>> bool(local_10am) False >>> local_10am != utcmidnight False >>> local_10am not in (naivemidnight, utcmidnight) False While it was originally the desire to reduce the impact of a very common bug that prompted me to reopen the issue, it is the improved consistency in the behaviour presented to users of Python 3.6+ that makes me belief this is actually worth fixing. We embarked on the whole Python 3 exercise in the name of making the language easier to learn by removing legacy features and fixing some problematic defaults and design warts - this is a tiny tweak by comparison, and will still go through the full normal deprecation cycle (warning in 3.5, behavioural change in 3.6) Regards, Nick. From stephen at xemacs.org Thu Mar 6 13:18:26 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 21:18:26 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051553.19311.shai@platonix.com> <CACac1F8ke11313pBBaWv0rfUwAstWO5Mt0e9knVBhqcYV8Vmcg@mail.gmail.com> <201403051719.09276.shai@platonix.com> <CACac1F-kxK1=J=ai6xPGN0=VK0oVQ4rUg4P+Qj3RaSyLiXAuSg@mail.gmail.com> <CABpHFHS0gB2-aRDXP0jUsqPsokB6vQ7_0OBkX78PMCp42_bHYw@mail.gmail.com> Message-ID: <87siqvplkd.fsf@uwakimon.sk.tsukuba.ac.jp> Ryan Hiebert writes: > he'd just like to see Python be better. We all agree that we'd like Python to be better. But many of us feel that the proposed change does not make Python better. It merely makes it more comfortable for what we (many of us, anyway) consider to be badly written code, and encourages people to write more code in poor style as we see it. > it seems that we are disagreeing on whether this strange behavior > should ever change. Well, yes, for Python 2 (obviously) and Python 3. I don't think anybody would object to changing it in Python 4. (Honestly, I would, on the principle above -- it really is only useful if you want to write code in bad style -- but it doesn't take a French soldier on the parapet to let you know in which general direction the wind blows.) > I think that, at some point, it should. If we ever agree on that, > then we can start thinking about when. I really don't think you'll achieve consensus before we start discussing Python 4 seriously. That's when the hunting season on backward incompatible changes (that don't correct showstoppers) opens. From steve at pearwood.info Thu Mar 6 13:35:20 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 6 Mar 2014 23:35:20 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185EA5.3060802@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando> <53185EA5.3060802@btinternet.com> Message-ID: <20140306123520.GL28804@ando> On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote: > > On 06/03/2014 10:52, Steven D'Aprano wrote: > >Or even no zero point at all. Tossing out a wild idea, why not leave > >datetime.time alone and add a new class which is exactly the same but is > >always truthy? That doesn't break backward compatibility, and so it > >could happen immediately (too late for 3.4, but certainly for 3.5). > > > >All we need is an appropriate name, and a decision as to which ought to > >inherit from which. The bike-shedding should be over by 3.5 alpha 1 > >*wink* > > > >Despite my quip about the bike-shedding, I am actually serious. > You're joking, right? What part of "I am actually serious" suggests to you that I am joking? No, I am not joking. When you have a misdesigned API that cannot be fixed without breaking backwards compatibility, or at least cannot be fixed *quickly*, it may be sensible to leave the old API in place and place a new one side-by-side. If the old one is actively harmful, it can even be deprecated for eventual removal, otherwise it can remain indefinitely. Does the term "legacy API" sound familiar? This is *extremely common* in the software industry, at least in that part that isn't run by cowboy coders who have no concern about breaking their users' code. It's quite common in Python too: consider the legacy API of os.system, the deprecated API of the popen2 module, and the modern subprocess API. > Suppose you were considering taking up a new language and you found > something in the docs which said: > "Here's a class that does so-and-so. And here's another slightly > different one that you can use instead, because we didn't get the first > one quite right." > Would that be a turn-on? Yes. This would tell me that the developers of this language took backward compatibility seriously, and that they weren't afraid to admit to mistakes. -- Steven From stefan at bytereef.org Thu Mar 6 13:36:46 2014 From: stefan at bytereef.org (Stefan Krah) Date: Thu, 6 Mar 2014 12:36:46 +0000 (UTC) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> Message-ID: <loom.20140306T131407-724@post.gmane.org> Mark H. Harris <harrismh777 at ...> writes: > At this point in the discussion please try not to think "implementation," > rather think conceptually in an open framework where anything is possible. As I understand, you are suggesting some kind of number system with symbolic representations that can be converted to decimal if needed. The topic is so vast that the only chance of it happening is for someone to provide an implementation on PyPI and see if people like it. There are packages for symbolic mathematics like sympy, perhaps that is a start. Even if people like it, it would be nearly impossible to integrate it into the core language without breaking third-party packages on a very large scale. So it is unlikely that such a thing will make it into the (hypothetical) 4.0. Regarding decimal literals: Possible in 3.x, but it would require some investigation if people really want arbitrary precision arithmetic by default rather than e.g. IEEE Decimal64. Regarding decimal floats by default: Perhaps in 4.0, but IMO only with *full* backing by the large SciPy and NumPy communities. My guess is that it's not going to happen, not in the least because the literature for floating point algorithms (with proofs!) has an overwhelming focus on binary floating point. From donald at stufft.io Thu Mar 6 13:46:26 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 07:46:26 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306123520.GL28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando> <53185EA5.3060802@btinternet.com> <20140306123520.GL28804@ando> Message-ID: <3C649111-A44E-46D8-BCCB-76A5BFACFA41@stufft.io> On Mar 6, 2014, at 7:35 AM, Steven D'Aprano <steve at pearwood.info> wrote: > On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote: >> >> On 06/03/2014 10:52, Steven D'Aprano wrote: > >>> Or even no zero point at all. Tossing out a wild idea, why not leave >>> datetime.time alone and add a new class which is exactly the same but is >>> always truthy? That doesn't break backward compatibility, and so it >>> could happen immediately (too late for 3.4, but certainly for 3.5). >>> >>> All we need is an appropriate name, and a decision as to which ought to >>> inherit from which. The bike-shedding should be over by 3.5 alpha 1 >>> *wink* >>> >>> Despite my quip about the bike-shedding, I am actually serious. > >> You're joking, right? > > What part of "I am actually serious" suggests to you that I am joking? > No, I am not joking. > > When you have a misdesigned API that cannot be fixed without breaking > backwards compatibility, or at least cannot be fixed *quickly*, it may > be sensible to leave the old API in place and place a new one > side-by-side. If the old one is actively harmful, it can even be > deprecated for eventual removal, otherwise it can remain indefinitely. > Does the term "legacy API" sound familiar? > > This is *extremely common* in the software industry, at least in that > part that isn't run by cowboy coders who have no concern about breaking > their users' code. It's quite common in Python too: consider the legacy > API of os.system, the deprecated API of the popen2 module, and the > modern subprocess API. > > >> Suppose you were considering taking up a new language and you found >> something in the docs which said: >> "Here's a class that does so-and-so. And here's another slightly >> different one that you can use instead, because we didn't get the first >> one quite right." >> Would that be a turn-on? > > Yes. This would tell me that the developers of this language took > backward compatibility seriously, and that they weren't afraid to admit > to mistakes. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ I think the decision to deprecate and change in place or create a new API depends greatly on how much code is going to break as a result of the deprecation, how hard it will be to restore the previous behavior, and how different the new API is. Language designers have to both consider backwards compatibility and the cognitive burden of having two (or more!) ways of doing the same thing. If you don't strike this balance and you never break backwards compatibility you very quickly end up with a massive number of new APIs. Remember that every bug fix is a potential backwards compatibility break for someone (https://xkcd.com/1172/ is relevant). In the case of datetime.time()?s boolean evaluation, I don?t believe a new API is warranted. I don?t believe (but I do not have proof) that there is going to be a lot of code out that that is depending on datetime.time evaluating to False in the cases it does (midnight in naive times, midnight utc when the utc offset is 0 or positive in aware times). I believe it is quite simple to replace the checks with a more explicit check when the user has been warned through the deprecation process. Finally I believe that the difference between the two APIs would be so slight as to increase the cognitive overhead for very little benefit. If we were talking about a major shift in how datetime.time() functions then maybe that would be warranted, however this is a small usability fix in an obscure corner of the API. It's not hardly, in my opinion, worth an entire new API revision. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/340e4de0/attachment.sig> From stephen at xemacs.org Thu Mar 6 13:46:55 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 21:46:55 +0900 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305135646.GB28804@ando> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> Message-ID: <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > The problem is that fractions can be unbounded in memory, Seminumerical Algorithms has a few problems on "floating slash", ie a representation of numbers where the total memory allocated to a fraction is fixed, but the amounts allocated to numerator and denominator are variable. I don't know if it has ever been tried in practice, though. And of course any reasonable fixed size would have very limited ability to express very large or very small magnitudes. From oscar.j.benjamin at gmail.com Thu Mar 6 13:51:01 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 6 Mar 2014 12:51:01 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <loom.20140306T131407-724@post.gmane.org> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <loom.20140306T131407-724@post.gmane.org> Message-ID: <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com> On 6 March 2014 12:36, Stefan Krah <stefan at bytereef.org> wrote: > > Regarding decimal literals: > > Possible in 3.x, but it would require some investigation if > people really want arbitrary precision arithmetic by default > rather than e.g. IEEE Decimal64. Interesting. I wasn't aware of Decimal64. I don't understand your question/point though. The Decimal module doesn't provide "arbitrary" precision arithmetic by default. It provides 28 digit precision arithmetic by default. It does however allow the creation of Decimal instances with effectively arbitrary precision from strings, integers etc. So I would assume that the idea was that nothing would change about decimal arithmetic (by default or when importing the module). The difference would just be that you could write 1.12345e-23d which would be equivalent to Decimal('1.12345e-23'). In this way it would be possible with a literal to express any decimal value exactly (even if it exceeds the arithmetic precision). It would be possible to do e.g. "if x <= 0.00001d:" and know that the test was exact. > Regarding decimal floats by default: > > Perhaps in 4.0, but IMO only with *full* backing by the large SciPy > and NumPy communities. My guess is that it's not going to happen, not > in the least because the literature for floating point algorithms > (with proofs!) has an overwhelming focus on binary floating point. Yes, it's very hard to gauge the effect of something like this. Understanding where in a numeric code base 64-bit binary floating point is assumed would be harder than disentangling unicode, from bytes, from 8-bit encodings I expect. Oscar From stephen at xemacs.org Thu Mar 6 14:02:21 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 22:02:21 +0900 Subject: [Python-ideas] Another way to avoid clumsy lambdas, while adding new functionality In-Reply-To: <53180C92.7070904@canterbury.ac.nz> References: <CAP-uhDcne+s8Z=N8RXqAg-Zyi+kT==+Ane4sZ=rTqDn-aCA42w@mail.gmail.com> <5317B1D7.4090305@canterbury.ac.nz> <5317CF11.20100@mrabarnett.plus.com> <53180C92.7070904@canterbury.ac.nz> Message-ID: <87pplzpjj6.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > MRAB wrote: > > > In the UK we talk about "Bills" being put before Parliament. > > In NZ too, but once it's passed, it's the law -- > it's not up to individual interpretation! NZ is a common law country, is it not? From mal at egenix.com Thu Mar 6 14:21:52 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 06 Mar 2014 14:21:52 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> Message-ID: <53187670.6070302@egenix.com> On 06.03.2014 13:08, Nick Coghlan wrote: > But perhaps I've been told "midnight is false in boolean context". But > which midnight? There are three that apply to me: > >>>> naivemidnight > datetime.time(0, 0) >>>> utcmidnight > datetime.time(0, 0, tzinfo=datetime.timezone.utc) >>>> localmidnight > datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000))) > > Are they all False? No, no they're not (unless your local timezone is UTC): > >>>> bool(utcmidnight) > False >>>> bool(naivemidnight) > False >>>> bool(localmidnight) > True Now this is a what I consider a valid argument for making a change. Thanks, Nick. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 06 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rob.cliffe at btinternet.com Thu Mar 6 14:47:05 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Thu, 06 Mar 2014 13:47:05 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306123520.GL28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando> <53185EA5.3060802@btinternet.com> <20140306123520.GL28804@ando> Message-ID: <53187C59.4080903@btinternet.com> On 06/03/2014 12:35, Steven D'Aprano wrote: > On Thu, Mar 06, 2014 at 11:40:21AM +0000, Rob Cliffe wrote: >> On 06/03/2014 10:52, Steven D'Aprano wrote: >>> Or even no zero point at all. Tossing out a wild idea, why not leave >>> datetime.time alone and add a new class which is exactly the same but is >>> always truthy? That doesn't break backward compatibility, and so it >>> could happen immediately (too late for 3.4, but certainly for 3.5). >>> >>> All we need is an appropriate name, and a decision as to which ought to >>> inherit from which. The bike-shedding should be over by 3.5 alpha 1 >>> *wink* >>> >>> Despite my quip about the bike-shedding, I am actually serious. >> You're joking, right? > What part of "I am actually serious" suggests to you that I am joking? > No, I am not joking. > > When you have a misdesigned API that cannot be fixed without breaking > backwards compatibility, or at least cannot be fixed *quickly*, it may > be sensible to leave the old API in place and place a new one > side-by-side. If the old one is actively harmful, it can even be > deprecated for eventual removal, otherwise it can remain indefinitely. > Does the term "legacy API" sound familiar? > > This is *extremely common* in the software industry, at least in that > part that isn't run by cowboy coders who have no concern about breaking > their users' code. It's quite common in Python too: consider the legacy > API of os.system, the deprecated API of the popen2 module, and the > modern subprocess API. > > >> Suppose you were considering taking up a new language and you found >> something in the docs which said: >> "Here's a class that does so-and-so. And here's another slightly >> different one that you can use instead, because we didn't get the first >> one quite right." >> Would that be a turn-on? > Yes. This would tell me that the developers of this language took > backward compatibility seriously, and that they weren't afraid to admit > to mistakes. > > Fair point. (It just feels untidy to me!) At least having two classes would highlight the difference in behaviour. Which makes me think, maybe the right thing to do now is to draw more attention to the behaviour in the docs. Section 8.1.5 time Objects currently states "in Boolean contexts, a time <http://docs.python.org/2/library/datetime.html#datetime.time> object is considered to be true if and only if, after converting it to minutes and subtracting utcoffset() (or 0 if that's None), the result is non-zero." This could be expanded, using the sort of examples Nick Coghlan produced, and a warning added that using bool on time objects is discouraged. And section 3.1 Truth Value Testing which claims to explicitly list all the falsey objects includes "instances of user-defined classes, if the class defines a __nonzero__() or __len__() method, when that method returns the integer zero or bool value |False|" I think this could be reworded. "user-defined classes" does not suggest to me that it includes classes in the stdlib. Would it be going too far to include in this section a short warning that using bool (explicitly or implicitly) on time objects is discouraged, or at least may not behave as expected? Rob Cliffe -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/569a6541/attachment.html> From donald at stufft.io Thu Mar 6 14:49:29 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 08:49:29 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53187670.6070302@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> Message-ID: <1B2F38D1-F5D0-4CD7-AC2B-CD51C2E3DA6A@stufft.io> On Mar 6, 2014, at 8:21 AM, M.-A. Lemburg <mal at egenix.com> wrote: > On 06.03.2014 13:08, Nick Coghlan wrote: >> But perhaps I've been told "midnight is false in boolean context". But >> which midnight? There are three that apply to me: >> >>>>> naivemidnight >> datetime.time(0, 0) >>>>> utcmidnight >> datetime.time(0, 0, tzinfo=datetime.timezone.utc) >>>>> localmidnight >> datetime.time(0, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 36000))) >> >> Are they all False? No, no they're not (unless your local timezone is UTC): >> >>>>> bool(utcmidnight) >> False >>>>> bool(naivemidnight) >> False >>>>> bool(localmidnight) >> True > > Now this is a what I consider a valid argument for making a change. > > Thanks, Nick. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 06 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ Yea Nick put into words what I?ve been attempting to say better than I ever could have :) ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ad2ed353/attachment-0001.sig> From rosuav at gmail.com Thu Mar 6 15:04:01 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 01:04:01 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53185EA5.3060802@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <5317FC67.7060600@canterbury.ac.nz> <20140306105253.GI28804@ando> <53185EA5.3060802@btinternet.com> Message-ID: <CAPTjJmrk=UyRdSOrBiQN-aqPEL4WxfFs9Cdon+imOmNxMz9gvg@mail.gmail.com> On Thu, Mar 6, 2014 at 10:40 PM, Rob Cliffe <rob.cliffe at btinternet.com> wrote: > Suppose you were considering taking up a new language and you found > something in the docs which said: > "Here's a class that does so-and-so. And here's another slightly different > one that you can use instead, because we didn't get the first one quite > right." > Would that be a turn-on? If the first one is marked deprecated, then I'd just ignore that one. The only (minor) downside is remembering which one's the deprecated one, so I avoid wasting time going to its docs and then going to the other. Compare: http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Sql/postgres.html http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Sql/pgsql.html In new code, which of those would you use? Is it sufficiently clear? I think it is. With Python's time types, though, there may be links with datetime. Which type does datetime.time() return? Will there be two parallel methods to resolve that? It's slightly more complicated. Still, it's not a killer. I've worked with far messier APIs - look at PHP and MySQL and the "real" functions. Which ones do you need _real_ in and which not? ChrisA From stephen at xemacs.org Thu Mar 6 15:33:10 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 06 Mar 2014 23:33:10 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53187670.6070302@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> Message-ID: <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> M.-A. Lemburg writes: > > Are they all False? No, no they're not (unless your local timezone is UTC): > > > >>>> bool(utcmidnight) > > False > >>>> bool(naivemidnight) > > False > >>>> bool(localmidnight) > > True > > Now this is a what I consider a valid argument for making a change. > > Thanks, Nick. I don't understand this turnabout. After all, time with time zones is the temporal equivalent of ISO 2022. It's just madness encapsulated in a formal data type definition.[1] Nobody should expect to get sane results from manipulating that stuff, only hope to get lucky. The only argument I've seen that corresponds to sane programming practice is Nick's statement that in Python, None, zeros, and empty containers are naturally false-ys, and all other objects are expected to be truth-ys. I find *that* compelling, even though it is backward incompatible with the documented behavior of time.time, and would not affect code using time.time objects that was written in good style. But I still don't like the way actually making the change would validate using "if var:" where "if var is not None:" is appropriate. I don't care if the change is made, but I hope it will be made absolutely clear that it's purely to work around *old* bugs that may never get fixed, and that "if var:" which is actually testing for identity with a false-y sentinel is still strongly discouraged. Footnotes: [1] Time counted in seconds from an epoch is hard enough, just Google for "monotonic clock". :-( From solipsis at pitrou.net Thu Mar 6 15:39:38 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 06 Mar 2014 15:39:38 +0100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <lfa1as$8bm$1@ger.gmane.org> Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit : > I don't care if the change is made, but I hope it will be made > absolutely clear that it's purely to work around *old* bugs that may > never get fixed, and that "if var:" which is actually testing for > identity with a false-y sentinel is still strongly discouraged. Why "strongly discouraged"? This is more of a style issue than anything else? Regards Antoine. From alexander.belopolsky at gmail.com Thu Mar 6 16:16:44 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 10:16:44 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9j8a$ole$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CA6768C6-0BD3-4C87-950F-EF4A747E9907@stufft.io> <5317AE64.90007@egenix.com> <45321B3B-B896-4A73-91AA-7F0188FCFD2C@stufft.io> <CAP7h-xbt_sGg0UeY=dnp1PvN1JNW+4JnUBZ6mvMDvicp-vx=Cg@mail.gmail.com> <lf9j8a$ole$1@ger.gmane.org> Message-ID: <CAP7h-xYRr9h5FeioXc1_+3cRV_Yt5cvQZC_454ZdOvfF+TAetQ@mail.gmail.com> On Thu, Mar 6, 2014 at 5:39 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > I think proponents of bool(time(0)) == True view it as a container of > > hours, minutes and seconds. > > No, they view it as an abstraction around the common concept of time. > OK, it looks like we are getting closer to the root of the disagreement. We are talking about the datetime.time type here. It is not "an abstraction around the common concept of time." That role belongs to the datetime.datetime type. The datetime.time type is an implementation of the concept of the *time of day*. In other words, for a datetime instance dt, dt.time() is the fractional part of dt and dt.date() is the integral part. A test for dt.time() is just as natural for datetime as a test for t % (24*60*60) for posix time. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/fcdd679d/attachment.html> From stefan at bytereef.org Thu Mar 6 16:49:04 2014 From: stefan at bytereef.org (Stefan Krah) Date: Thu, 6 Mar 2014 15:49:04 +0000 (UTC) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <loom.20140306T131407-724@post.gmane.org> <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com> Message-ID: <loom.20140306T163813-515@post.gmane.org> Oscar Benjamin <oscar.j.benjamin at ...> writes: > On 6 March 2014 12:36, Stefan Krah <stefan at ...> wrote: > > Regarding decimal literals: > > > > Possible in 3.x, but it would require some investigation if > > people really want arbitrary precision arithmetic by default > > rather than e.g. IEEE Decimal64. > > Interesting. I wasn't aware of Decimal64. > > I don't understand your question/point though. The Decimal module > doesn't provide "arbitrary" precision arithmetic by default. It > provides 28 digit precision arithmetic by default. It does however > allow the creation of Decimal instances with effectively arbitrary > precision from strings, integers etc. Well, yes. You can also emulate Decimal64 with the appropriate context parameters. I meant "arbitrary precision facilities by setting context parameters", but probably many people want to keep those. Due to the compact representation (bid64 or bid128), Decimal64 or Decimal128 would be interesting for doing arithmetic on huge matrices. Stefan Krah From alexander.belopolsky at gmail.com Thu Mar 6 17:13:57 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 11:13:57 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53187670.6070302@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> Message-ID: <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com> On Thu, Mar 6, 2014 at 8:21 AM, M.-A. Lemburg <mal at egenix.com> wrote: > > Are they all False? No, no they're not (unless your local timezone is > UTC): > > > >>>> bool(utcmidnight) > > False > >>>> bool(naivemidnight) > > False > >>>> bool(localmidnight) > > True > > Now this is a what I consider a valid argument for making a change. FWIW, I would be +0 for making a change so that t.tzinfo is not None implies bool(t.timetz()) is True. My intuition goes like this. Given >>> from datetime import * >>> t1 = datetime(2014, 1, 1) >>> t2 = datetime(2014, 1, 1, 12) >>> t3 = datetime(2014, 1, 1, tzinfo=timezone.utc) Instance t1 has no time information, so t1.time() is false-y. Instance t2 has time information, so t2.time() is true-y. Instance t3 is such that t3.time() == t1.time(), but t3.timetz() has an additional information >>> t3.timetz() datetime.time(0, 0, tzinfo=datetime.timezone.utc) >>> t3.time() datetime.time(0, 0) So t3.timetz() is not the same object as returned by the constructor with no arguments - time(). This is reason enough to make it true-y. This also does not affect my use-case because t3.time() is still false-y in this logic: >>> [bool(t) for t in [t1.time(), t2.time(), t3.time(), time()]] [False, True, False, False] Which is consistent with >>> t3.time() == t1.time() == time() True -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/9f1b8972/attachment-0001.html> From alexander.belopolsky at gmail.com Thu Mar 6 17:36:11 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 11:36:11 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> Message-ID: <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> On Thu, Mar 6, 2014 at 12:53 AM, Tim Peters <tim.peters at gmail.com> wrote: > [Bruce Leban <bruce at leapyear.org>] > > ... > > Just because you wrote the docs doesn't mean you know what they mean to > > other readers. The point of documentation is to explain it to someone who > > doesn't know what it does after all. > > Of course. What's your point here? > I am with Bruce here. I tried to digest your notes on timezone algebra [1] for years and I still don't understand it. [1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/2e8975aa/attachment.html> From oscar.j.benjamin at gmail.com Thu Mar 6 17:41:34 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 6 Mar 2014 16:41:34 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <loom.20140306T163813-515@post.gmane.org> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <loom.20140306T131407-724@post.gmane.org> <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com> <loom.20140306T163813-515@post.gmane.org> Message-ID: <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com> On 6 March 2014 15:49, Stefan Krah <stefan at bytereef.org> wrote: > Oscar Benjamin <oscar.j.benjamin at ...> writes: >> On 6 March 2014 12:36, Stefan Krah <stefan at ...> wrote: >> > Regarding decimal literals: >> > >> > Possible in 3.x, but it would require some investigation if >> > people really want arbitrary precision arithmetic by default >> > rather than e.g. IEEE Decimal64. >> >> Interesting. I wasn't aware of Decimal64. >> >> I don't understand your question/point though. The Decimal module >> doesn't provide "arbitrary" precision arithmetic by default. It >> provides 28 digit precision arithmetic by default. It does however >> allow the creation of Decimal instances with effectively arbitrary >> precision from strings, integers etc. > > Well, yes. You can also emulate Decimal64 with the appropriate > context parameters. > > I meant "arbitrary precision facilities by setting context parameters", > but probably many people want to keep those. > > Due to the compact representation (bid64 or bid128), Decimal64 or > Decimal128 would be interesting for doing arithmetic on huge matrices. I'm still not totally clear here. Do you mean that decimals created with a decimal literal e.g. 1.123d would be some other kind of decimal object that always used a special arithmetic context so that they behaved like a fixed width Decimal64 type? And then someone who wants to do decimal arithmetic with other contexts would still need to import decimal and do the context stuff themselves? Oscar From tim.peters at gmail.com Thu Mar 6 17:50:53 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 10:50:53 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> Message-ID: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> [Bruce Leban <bruce at leapyear.org>] >>> ... >>> Just because you wrote the docs doesn't mean you know what they mean to >>> other readers. The point of documentation is to explain it to someone >>> who doesn't know what it does after all. [Tim] >> Of course. What's your point here? [Alexander Belopolsky <alexander.belopolsky at gmail.com>] > I am with Bruce here. I tried to digest your notes on timezone algebra [1] > for years and I still don't understand it. > > [1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922 But those aren't user docs - they're notes for developers who are deeply steeped in internal details. And they're hard! Mucking with "all possible" time zones raises mountains of excruciatingly subtle details and corner cases. The masses of tedious "algebra" were needed to justify how the internals get away with doing "so little" - and to explain why the code still wouldn't work right in all cases if the "standard offset" of a single time zone changed (or changes) over time. Do you claim you still don't understand when bool(time) returns True and False? Those are user docs. I don't claim that reading them will make anyone feel good ;-), but I do believe the computation is described clearly enough to enable a reasonable user to predict the result in all cases. From donald at stufft.io Thu Mar 6 17:53:31 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 11:53:31 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> Message-ID: <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io> On Mar 6, 2014, at 11:50 AM, Tim Peters <tim.peters at gmail.com> wrote: > [Bruce Leban <bruce at leapyear.org>] >>>> ... >>>> Just because you wrote the docs doesn't mean you know what they mean to >>>> other readers. The point of documentation is to explain it to someone >>>> who doesn't know what it does after all. > > [Tim] >>> Of course. What's your point here? > > [Alexander Belopolsky <alexander.belopolsky at gmail.com>] >> I am with Bruce here. I tried to digest your notes on timezone algebra [1] >> for years and I still don't understand it. >> >> [1] http://hg.python.org/cpython/file/b07400659dba/Lib/datetime.py#l1922 > > But those aren't user docs - they're notes for developers who are > deeply steeped in internal details. And they're hard! Mucking with > "all possible" time zones raises mountains of excruciatingly subtle > details and corner cases. The masses of tedious "algebra" were needed > to justify how the internals get away with doing "so little" - and to > explain why the code still wouldn't work right in all cases if the > "standard offset" of a single time zone changed (or changes) over > time. > > Do you claim you still don't understand when bool(time) returns True > and False? Those are user docs. I don't claim that reading them will > make anyone feel good ;-), but I do believe the computation is > described clearly enough to enable a reasonable user to predict the > result in all cases. I don?t think a reasonable user would predict that midnight UTC in a timezone with a UTC offset of +5 would be false, while midnight UTC in a timezone with a UTC offset of -5 would not be false. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ce9216f7/attachment.sig> From tim.peters at gmail.com Thu Mar 6 18:01:56 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 11:01:56 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io> Message-ID: <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com> [TIm] >> ... >> Do you claim you still don't understand when bool(time) returns True >> and False? Those are user docs. I don't claim that reading them will >> make anyone feel good ;-), but I do believe the computation is >> described clearly enough to enable a reasonable user to predict the >> result in all cases. [Donald Stufft <donald at stufft.io>] > I don't think a reasonable user would predict that midnight UTC in a timezone > with a UTC offset of +5 would be false, while midnight UTC in a timezone with > a UTC offset of -5 would not be false. As I said, I didn't claim a reasonable user would feel good about it, and all you're saying here is that they wouldn't feel good about it. I'm not arguing that they should feel good about it. I said that the docs explained the computation clearly enough that a reasonable user - who read the docs and earnestly tried to apply them - _could_ predict the results. And you just showed that you, for example, can predict them. Since you're the very definition of "reasonable", I close my case for that ;-) From alexander.belopolsky at gmail.com Thu Mar 6 18:02:36 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 12:02:36 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> Message-ID: <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com> On Thu, Mar 6, 2014 at 11:50 AM, Tim Peters <tim.peters at gmail.com> wrote: > Do you claim you still don't understand when bool(time) returns True > and False? Those are user docs. I don't claim that reading them will > make anyone feel good ;-), but I do believe the computation is > described clearly enough to enable a reasonable user to predict the > result in all cases. > I have to admit that before this discussion, I did not know about the following consequence of the documented behavior: >>> from datetime import * >>> t = datetime(2014, 1, 1, tzinfo=timezone.utc) >>> tz1 = timezone(timedelta(hours=5)) >>> tz2 = timezone(timedelta(hours=-5)) >>> bool(t.timetz()) False >>> bool(t.astimezone(tz1).timetz()) False >>> bool(t.astimezone(tz2).timetz()) True -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/78af5167/attachment.html> From tim.peters at gmail.com Thu Mar 6 18:05:39 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 11:05:39 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com> Message-ID: <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com> [Alexander Belopolsky <alexander.belopolsky at gmail.com>] > I have to admit that before this discussion, I did not know about the > following consequence of the documented behavior: > >>>> from datetime import * >>>> t = datetime(2014, 1, 1, tzinfo=timezone.utc) >>>> tz1 = timezone(timedelta(hours=5)) >>>> tz2 = timezone(timedelta(hours=-5)) >>>> bool(t.timetz()) > False >>>> bool(t.astimezone(tz1).timetz()) > False >>>> bool(t.astimezone(tz2).timetz()) > True I'm not sure I did either! An "aware" time object is a bizarre beast, and I've never used one outside of writing datetime test cases. I'm not even sure why aware time objects exist - we'd have to dig into Guido's subconscious for that one ;-) From donald at stufft.io Thu Mar 6 18:10:23 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 12:10:23 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> <BEA010DF-AA9A-45E0-AEE1-F91521D759EC@stufft.io> <CAExdVNmQ97o9i0fVPtxi55-5-6GMKMTqMXc3kC2sNJYnb2v7Eg@mail.gmail.com> Message-ID: <B4BDA6D6-6CA3-40A2-A608-BA7EF2F3ADD3@stufft.io> On Mar 6, 2014, at 12:01 PM, Tim Peters <tim.peters at gmail.com> wrote: > [TIm] >>> ... >>> Do you claim you still don't understand when bool(time) returns True >>> and False? Those are user docs. I don't claim that reading them will >>> make anyone feel good ;-), but I do believe the computation is >>> described clearly enough to enable a reasonable user to predict the >>> result in all cases. > > [Donald Stufft <donald at stufft.io>] >> I don't think a reasonable user would predict that midnight UTC in a timezone >> with a UTC offset of +5 would be false, while midnight UTC in a timezone with >> a UTC offset of -5 would not be false. > > As I said, I didn't claim a reasonable user would feel good about it, > and all you're saying here is that they wouldn't feel good about it. > I'm not arguing that they should feel good about it. > > I said that the docs explained the computation clearly enough that a > reasonable user - who read the docs and earnestly tried to apply them > - _could_ predict the results. And you just showed that you, for > example, can predict them. Since you're the very definition of > "reasonable", I close my case for that ;-) Actually I wasn?t able to predict it :) Someone else posted a number of times offsets and their boolean values. I was really confused when it turned out that 01:00:00+01:00 (1AM France/Midnight UTC) was False but 19:00:00-05:00 (7PM Boston/Midnight UTC) wasn?t. I only realized that the cause was the specific formula that was used when I reversed it and tried to figure out what time in Boston would be False and realized it was -05:00. I don?t think it?s reasonable by any definition of the word that in order to predict the results a user would have to turn it into an algebraic formula and solve for X to determine that the only way for Boston to have a False time is if time was negative. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/ab100030/attachment.sig> From alexander.belopolsky at gmail.com Thu Mar 6 18:11:33 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 6 Mar 2014 12:11:33 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051924.05252.shai@platonix.com> <CANc-5Ux9aUb3jU9K2U5CzfzNYhyhE4ve736vFGrgksz+bwANbg@mail.gmail.com> <201403052044.55720.shai@platonix.com> <20140305192419.GD28804@ando> <CAGu0AnuWsxKyqWobYiPpAdNLXeu_HSpVcHDqNPF=b+d5wi+KQw@mail.gmail.com> <CAExdVNm_cy1_RNgEzXS-eGg5b_Zu5MgopJu9yJCOoc0JzJ5wZg@mail.gmail.com> <CAGu0Anum6mPgLPbYyAuF2natbn47nDm2GgBucSqJ1jSMoHg5eg@mail.gmail.com> <CAExdVN=WEKYkjU=TkiVE+81xcEhNhOewv8VBe9n08XR1FSKNtg@mail.gmail.com> <CAGu0Anv-z=YpoLbT6b4wEXo-VNd48+PzJ53Zg_MuE6G5HsssTQ@mail.gmail.com> <CAExdVN=bX=hKvKbox2cSvfm+Si2JyHE4rNN0ekZ+xc4ScV+a0Q@mail.gmail.com> <CAP7h-xYVz-nb4RyFNqzWs5Z8O7z_NLBGunaOvjkw6FMv6o6dQw@mail.gmail.com> <CAExdVNkUrBQU1MBZU_3KeLbnT1fMyUsFbL+5c59F7KdK0FSoJg@mail.gmail.com> <CAP7h-xZMHrxtyMLmeh8YVfUGOahTsuFRkZt2vWshMg5MRoja4A@mail.gmail.com> <CAExdVNkOJZF39RwgjOFjpXduCuQ92VoB1FkKeG0jP9RX8Vajow@mail.gmail.com> Message-ID: <CAP7h-xbd7zH_jUGhTccuaptav6BLS9_WGf5p1TfD2TYgzupGJA@mail.gmail.com> On Thu, Mar 6, 2014 at 12:05 PM, Tim Peters <tim.peters at gmail.com> wrote: > I'm not sure I did either! An "aware" time object is a bizarre beast, > and I've never used one outside of writing datetime test cases. I'm > not even sure why aware time objects exist - we'd have to dig into > Guido's subconscious for that one ;-) > They are occasionally useful if you want to split datetimes into date and time parts, process those separately and recombine in the end without loosing tzinfo. I don't think anything involving time.utcoffset() makes much sense and it is unfortunate that time.utcoffset() can be invoked implicitly when aware time is used in boolean context. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/7cd673a0/attachment.html> From tim.peters at gmail.com Thu Mar 6 18:13:36 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 11:13:36 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9gft$jnh$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> Message-ID: <CAExdVN=Nhx8V179nwFzw9dFkieSWkf0A9sJgj50or+tBkg_kuw@mail.gmail.com> [Antoine Pitrou <solipsis at pitrou.net>] > ... > It is not a coincidence, IMO, that the three persons arguing that the > current behaviour is reasonable are implementors of datetime modules: you, > Tim and Alexander. All other people find the behaviour baffling. The behavior of "aware" time objects is indeed bizarre, but then aware time objects are bizarre beasts from the start (a time zone attached to a mere time-of-day, divorced from date info, doesn't make any real sense to me). The reasons I already explained (possible extensions to naive time's repertoire, in the areas of type conversions and arithmetic) account for why bool(time()) returns False. _As things turned out_ no such extensions were made, so we're left with a foundation sticking out of the ground with nothing built on top of it. Yawn ;-) From songofacandy at gmail.com Thu Mar 6 18:23:31 2014 From: songofacandy at gmail.com (INADA Naoki) Date: Fri, 7 Mar 2014 02:23:31 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53184096.3050409@egenix.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <AF79E619-68BF-4CFA-B1B4-36358E411BEF@stufft.io> <53184096.3050409@egenix.com> Message-ID: <CAEfz+TwU6JStj=s2ScN2fq-=ejXLPQnxvBgx3M7akFg2p9rsZA@mail.gmail.com> I feel zero value of non abelian group should not mean False in bool context. () + () == () "" + "" == "" 0 + 0 == 0 timedelta() + timedelta() == timedelta() time() + time() => TypeError On Thu, Mar 6, 2014 at 6:32 PM, M.-A. Lemburg <mal at egenix.com> wrote: > On 06.03.2014 09:46, Donald Stufft wrote: >> >> On Mar 6, 2014, at 3:35 AM, M.-A. Lemburg <mal at egenix.com> wrote: >> >>> On 06.03.2014 03:23, Donald Stufft wrote: >>>> >>>> On Mar 5, 2014, at 9:15 PM, Tim Peters <tim.peters at gmail.com> wrote: >>>> >>>>> [Donald Stufft] >>>>>> When the documented behavior is both nonsensical and the cause of hard to debug bugs that >>>>>> is a pretty compelling use case to me, unless you actively enjoy being user hostile. >>>>> >>>>> Breaking code that currently works is as hostile as hostile gets. Where is the evidence that >>>>> no code relies on the current behavior? For that matter, where is the evidence that the >>>>> current behavior is a significant cause of bugs? I know I've used "if some_time_object:", >>>>> but it did exactly what I expected it to do. >>>> >>>> Forgive me if I'm wrong, but aren't you the author of the date time module? If that's the case >>>> what you expect it to do isn't particularly relevant as you're intimately aware of it's >>>> implementation. >>> >>> Just in case you're looking for a relevant use case: >>> >>> Having bool(time(0,0,0)) allows you to quickly check whether you are >>> dealing with a datetime value with (non-trivial) time part or not. >> >> I don't see how midnight is any more or less trivial than 12:01. >> >>> >>> You run into situations where you have to test for this in >>> date/time conversions, arithmetics, etc. e.g. if you need to convert >>> a datetime value to a date and want to prevent truncation of >>> information, or if you want to format a datetime value in a user >>> friendly way by omitting the zero time part, or if you're doing >>> datetime arithmetic that has to follow special requirements w/r to >>> days (think interest or insurance math) and you want to >>> quickly check whether you can use the fast path simple >>> calculation, or you need to do the full blown complicated part. >> >> I think these would be better served by actually checking for what you mean. >> For the record I also think that checking for a date time to not be None >> is also better served by actually checking for what you mean. The >> difference is in intent and likelihood of confusion. > > FWIW, I expect bool(time(0,0,0)) == False, just like I expect > bool(0.0) == False. I would find it confusing to have > bool(zero_element) == True. > >> For all of those you're going to be explicitly testing and working with >> midnight objects, so you're going to test them, you're going to exercise >> that code. In the ``is None`` case, Midnight is just another value so that >> bug will lay dormant and undetected until someone just happens to >> hit an unlucky time. And once you get that it's likely to be difficult to >> reproduce. > > datetime values with zero time are *very* common in practice > (you run into them whenever you convert a date value into a datetime > value), so the "is None" case is easy to reproduce and will hit > you more often than you like. > > BTW: Not using "is None" will bite you in many other ways as > well. None it typically used as value for "not initialized" or > "no value provided". This meaning is completely different from > "empty selection" or "empty string", so in order to address all > those cases as well, you'd have to change: > > bool(()) == True > bool('') == True > etc. > > The "if x is None:" test is also faster than "if x:", so actually > win something by coding correctly ;-) > > Please provide a stronger argument for needing to change > bool(time(0,0,0)) than one which is built on buggy code :-) > (I'm running out of smileys...) > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 06 2014) >>>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-04-09: PyCon 2014, Montreal, Canada ... 34 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- INADA Naoki <songofacandy at gmail.com> From tim.peters at gmail.com Thu Mar 6 18:29:37 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 11:29:37 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53181ADA.2030401@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> Message-ID: <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> [Tim Peters] >> For example, modular arithmetic on time >> objects was a possibility. In that case, time(0, 0, 0) would become >> "a zero value" [Greg Ewing <greg.ewing at canterbury.ac.nz>] > That only follows if you interpret "modular arithmetic on > time objects" as meaning that you can directly add one time > object to another. Of course. > But that would be confusing times with timedeltas. The concept of > a zero *timedelta* makes sense, but not a "zero time". The only difference between time and timedelta is in implementation details. In effect, time values are (or _can_ be viewed as) a subset of timedelta values, restricted to non-negative durations strictly less than 24 hours. From that point of view, it's obvious how to do modular (mod 24 hours) arithmetic on time values. And some people said they wanted that. It wasn't obviously useful enough to implement at first, but neither did we want to preclude it in the future. I don't personally want it. >> People can say that "midnight" isn't a compelling "sentinel value", >> and I agree about the "compelling" part, but in all implementation >> respects it does _act_ as "a sentinel value". > If someone is using midnight to represent an unspecified > time, I'd say they have a bug. An appointment scheduled at > midnight is not the same as an appointment that's not > scheduled at all. Again obviously so. In much the same way, there's nothing forbidden about -1 as an integer, but it's ubiquitously used as an error return in CPython's implementation. That doesn't mean CPython is buggy, it implies that CPython has other ways to distinguish whether a -1 return does or does not mean "error!". For example, in some applications (not all), "exactly midnight" simply can't happen as a legitimate starting or ending time. From stephen at xemacs.org Thu Mar 6 18:40:42 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Mar 2014 02:40:42 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lfa1as$8bm$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> <lfa1as$8bm$1@ger.gmane.org> Message-ID: <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit : > > never get fixed, and that "if var:" which is actually testing for > > identity with a false-y sentinel is still strongly discouraged. > > Why "strongly discouraged"? This is more of a style issue than anything > else? No, it's not just a style issue. As Skip points out, it's a *semantic* issue as well. You're really testing for not for the boolean value of values of one type, but for a difference of type ("initialized whatever" vs "uninitialized variable"). Pragmatically, this would be half a dozen of one, six of the other, except that there are two more possibilities: a false-y of a different type, and a truth-y of a different type. I don't think the spurious truth-y matters very much ... you'd expect a TypeError to be raised pretty quickly, and if not, it's a bad bug (by the time you notice the bad value, it's really hard to work back to where and why it's been injected). The spurious false-y, on the other hand, seems quite likely to lead to spurious values *of the expected type*. To restate my earlier example: my_time = None if not my_time and x: my_time = x.expected_to_be_parsed_time_but_parse_bug_returns_0 if not my_time: my_time = time.now() You may never catch *that* parser bug, because the value of 'my_time' always looks right (ie, like a datetime.time value). But in my_time = None if my_time is None and x: my_time = x.expected_to_be_parsed_time_but_parse_bug_returns_0 if my_time is None: my_time = time.now() you're on the same footing as with the truth-y -- just a bit lucky and you get a quick TypeError. This is an objective benefit (although you can say "the probability that there will be a bug to catch is too low to care about" if you like, it's obviously bigger than zero). I also value the psychological benefit of being precise here. I'm not even sure it's possible to fool 'is' into thinking some other object is None, but people do a great job of convincing themselves of things like "datetime.time objects can't be false" all the time. Use the "is None" idiom and you know exactly what you're doing. This latter is close to "just style", of course. Steve From guido at python.org Thu Mar 6 18:48:53 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 6 Mar 2014 09:48:53 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> Message-ID: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> On Thu, Mar 6, 2014 at 12:14 AM, Mark H. Harris <harrismh777 at gmail.com>wrote: > On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote: >> >> Do you actually have a degree in math, or do you just remember your high >> school algebra? >> > > hi Guido, ouch. Its a story, glad you asked. My first trip to > college 1974-1977 [...] > I must have hit a nerve -- I wasn't requesting a transcript! :-) A simple "I dropped out of math in college to pursue a career in programming" would have been sufficient (my own story is pretty similar :-). > We might agree to stick with the discussion, if you're are willing, and > stay away from *ad hominem* > attacks, nor credential bashing. A person either knows what they are > doing, or they don't. And if > they are willing to contribute, why knock it, or them? > I was asking because I am having a hard time getting your point, and it feels like you keep repeating it without clarifying it. > The numbers in math usually are quite strictly typed: whole theories only >> apply to integers or even positive integers, other things to all reals but >> not to complex numbers. >> > > Everyone keeps making the points noted above! Yes, I know... concur~ > Guido, all data types in most computer high level languages > are all very strictly statically bound also. > I'm not sure that that description applies to Python; it sure doesn't match how I *think* about numbers in Python. > So what? You know this better > than anyone, because you have taken so much abuse from trolls about it > over the > years. > Actually, I haven't -- I can smell a troll a mile away and just mute the conversation. > We all know that the best way to handle name binding is dynamically. > That's your opinion. There are pros and cons and both static and dynamic binding have their place. > And that was a paradigm shift, was it not? > I've never claimed such a hyperbole. > Go take a look at Rexx. It has NO types. None. > Not at the surface, anyway. Everything to the user of the language is a > string. This is > even true for most of "object" Rexx too. *Numbers *are just strings of > characters that > are parsed and interpreted by Rexx as valid *Rexx Numbers*. > Why do you italicize this? > It works. There is no > real point in arguing that its a bad idea, because it served the Rexx > community for > many years... even now, Rexx is not dead. Yes, under the covers (not > magic) a complex > number is going to be handled differently than a real number. Yeah, what > is your point? > They are handled differently in my TI 89 too... so what, I change the > context on the 89 > and now I'm doing complex number math (not often, I might add). > By "context", here, do you mean some specific state in the computer, or are you referring to a different way of thinking about it (i.e. in your head)? > If I set the context for > reals, then now I'm doing real math... it really is not that difficult. > So I suppose depending on how you set the context, the square root of -1 either returns a complex number, or raises an error? And comparisons are right out once your context is set for complex numbers? Or are they still allowed when the imaginary part is exactly zero? (What if it is nearly zero?) > Think about this for just > a minute. I have used complex math exactly three times in my life. I used > it in high school > to study the concept in math analysis. I used it in my EE classes... > electrical engineers > get a lot out of complex numbers. And I used it when I was interested in > the Mendlebrot > set many years ago when it was cool to plot the famous fractal on early > cga screens. > When was the last time you used complex numbers? When was the last time a > bank, or > a hospital, or Montgomery Wards used complex numbers? If someone needs > complex > numbers, Python could change the context "dynamically" and move through > the problem set. > Why should the user need to "manually" change the context if (AI) could > change it for them > on-the-fly? Just try to get a feel for the question, and stop with trying > to beat me up on > my credentials. > I still don't understand the question. Can you be more precise instead of using more rhetoric? Is your point just that you don't want to have to deal with complex numbers? That's fine, and it's already (mostly) how Python works -- you must use a complex literal or the cmath module before any complex numbers appear (the exception is that x**y returns a complex number when x is negative and y is not a whole number -- but arguably that's the best answer). > (And what about quaternions? Or various infinities? :-) >> > > What about quaternions? If you extend the complex number system you > change the context. That's not hard... > So, by context you really seem to be referring to some specific state of the program. Perhaps you are even talking about extending the existing notion of numeric context used by Python's Decimal type. But it's not very clear because "you change the context" is also a common term for changing the topic of a discussion. You would not expect the context for "reals" processing to be the same for > the context of > processing three dimensional space with pairs of complex numbers, would > you? > Actually I'm not sure I *need* a context for this. In Python things like this are usually just solved through a combination of operator overloading and dynamic binding, so that e.g. x*y can be a number when x and y are numbers, but it can mean a vector product of some sort if x and/or y are vectors. > Python > does complex number processing now. But that is a very specialized > category of use > case that requires special context and (under the covers) typing relevant > to complex number > pairs. The context can change "dynamically" to suit the problem set, > that's all I'm saying. > I'm sorry, but I am *still* unsure about what you mean by "context". Python does not need a context object to switch between complex and real number processing -- it's all done through operator overloading. > I am not saying in all of my dreaming here that typing is not important > (down under). Python is > called an object based language, yes? Not object oriented, right? > Actually, modern Python is considered object-oriented. I called it object-based in a *very* early stage, when there was no class statement in the language. But IIRC it was added before the first public release (or soon after), although another round of improvements in this area happened in the early 2000s (eradicating most of the differences between classes defined by C and Python code). > But we all know that down > under the covers (not magic) python uses Classes and instances of > classes--objects. We > don't pretend that what makes object based languages work is magic? do we? > "We"? > Unifying the > number system on a computer does not have to be grounded in paradigms that > serviced the > industry (and academy) for the past 40 years; mostly because memory was > expensive and > processing was slow. I suggest that if memory had been cheap, back in the > day, and processing > had been very fast (as it is today) IEEE 754 1985 floats and doubles would > never have been > used. We would have used Decimal floating points right from the get-go. > Initially, that really is > all I'm asking for on the outset--- lets move to default decimal floating > point arithmetic for real > numbers processing. > Now you've got a specific proposal and we can discuss pros and cons. This is a topic that comes up occasionally and it is indeed a pretty interesting trade-off. I suspect the large (and fast-growing) SciPy community would prefer to keep binary floating point, because that's what their libraries written in Fortran or C++ want, and extra binary/decimal conversions at the boundaries would just slow things down. But I'm really speculating here -- maybe they don't need high-performance conversion between binary and decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe it varies per application. > In the future I am thinking about systems of languages that interact with > human speech; understanding > *spoken number* just like they will understand *symbol number.* There is > really no reason (other > than paradigm) that this would not be possible either. > This is the kind of talk that I wish we could keep out of the python-ideas list. (Except for the symbolic algebra, for which various third-party Python libraries already exist.) > Otherwise, Guido, we might all of us just continue to use C and code up > our statically bound types by > hand using nothing but int, long, float, and double. > > Its time to innovate. > Rhetoric again. > Kind regards, BDfL, > > marcus > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5f15a08a/attachment-0001.html> From stefan at bytereef.org Thu Mar 6 19:13:28 2014 From: stefan at bytereef.org (Stefan Krah) Date: Thu, 6 Mar 2014 19:13:28 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <loom.20140306T131407-724@post.gmane.org> <CAHVvXxQSDc7FfiPzjEGq9qrcdvnqF+Wv_uVtBpyRLg+VikvXnQ@mail.gmail.com> <loom.20140306T163813-515@post.gmane.org> <CAHVvXxT9J8ZgdMYua7sQhFS8=Qu5HDw2pPoJs6Ov=AczFvm_fg@mail.gmail.com> Message-ID: <20140306181328.GC31090@sleipnir.bytereef.org> Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > > Well, yes. You can also emulate Decimal64 with the appropriate > > context parameters. > > > > I meant "arbitrary precision facilities by setting context parameters", > > but probably many people want to keep those. > > > > Due to the compact representation (bid64 or bid128), Decimal64 or > > Decimal128 would be interesting for doing arithmetic on huge matrices. > > I'm still not totally clear here. > > Do you mean that decimals created with a decimal literal e.g. 1.123d > would be some other kind of decimal object that always used a special > arithmetic context so that they behaved like a fixed width Decimal64 > type? And then someone who wants to do decimal arithmetic with other > contexts would still need to import decimal and do the context stuff > themselves? That would be one option. I'm not sure if it is a *good* option. Another option is to use the regular Decimal objects with the IEEE context and perhaps add functions to convert to bid64 etc. Yet another issue is whether a Decimal core type should use the function names from IEEE 754-2008. The names are different, though the behavior is largely the same. My main point here is that even for the Decimal literal (which seems innocent enough) there is a lot to discuss, since we can specify the semantics only once. Stefan Krah From abarnert at yahoo.com Thu Mar 6 19:47:41 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 10:47:41 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <loom.20140306T120303-396@post.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <loom.20140306T120303-396@post.gmane.org> Message-ID: <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com> On Mar 6, 2014, at 3:13, Stefan Krah <stefan at bytereef.org> wrote: > Antoine Pitrou <solipsis at ...> writes: >> It is not a coincidence, IMO, that the three persons arguing that the >> current behaviour is reasonable are implementors of datetime modules: >> you, Tim and Alexander. All other people find the behaviour baffling. > > Not all. :) I think viewing midnight as (unix_time % 86400) is > quite reasonable. I agree with Stefan here. Treating midnight as zero is not at all baffling. There's a reason modular arithmetic is taught to kids as "clock arithmetic", after all. And it's not unheard of for other languages/libraries/apps to treat midnight as a 0, or even to use it as a null time. Someone else already mentioned that many RDBMS's don't have a separate date type and store dates as datetimes with midnight time. IEEE datetimes also do this--leaving off any trailing component is equivalent to having 0's there. And this isn't some experts-only thing. Novices regularly ask why we need the date class when a datetime with 0 time does the same thing--often because that's what their first language did, but sometimes just as an a priori expectation. On the other hand, the alternative isn't baffling either. Unlike, say, Oracle, Python has separate types for date, time, and datetime, so we don't need to use 0 for "n/a", and it has tz-aware values for all three types, making midnight much less special, and it doesn't have modular arithmetic on times (and timedeltas), etc. So honestly, if handed the Python time class, I wouldn't know whether to expect midnight to be treated as 0 (and therefore falsey) or not, and would have to read the docs rather than guessing. When a choice really is arbitrary like this, "it does what it's documented to do" is the only notion of correct that makes sense. (As a disclaimer, I have implemented datetime libraries for C and C++, but not for Python.) From abarnert at yahoo.com Thu Mar 6 20:01:16 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 11:01:16 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> Message-ID: <5175E062-E30B-4442-828E-4BDA8A01AD9E@yahoo.com> On Mar 6, 2014, at 4:08, Nick Coghlan <ncoghlan at gmail.com> wrote: > if x not in (naivemidnight, utcmidnight): > # This is the current bool(x) > .... > if x != naivemidnight: > # This has no current shorthand > .... > if x != utcmidnight: > # This has no current shorthand > .... > if x != localmidnight: > # This has no current shorthand > .... This is a great argument. The problem here isn't that we use midnight as 0, but that naive and aware times are the same type and we use UTC midnight as 0 for aware types. The other languages/libraries/apps that treat midnight as 0 are generally either naive-only, or much more explicit about awareness. Since that isn't going to change in Python, maybe it would be better to provide these values (or functions that return them) as names in the module, and explicitly encourage testing against them in place of bool testing, and define bool testing exactly like this, whether or not it's deprecated? From tjreedy at udel.edu Thu Mar 6 20:32:43 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 06 Mar 2014 14:32:43 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf9gft$jnh$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> Message-ID: <lfaigr$8t4$1@ger.gmane.org> On 3/6/2014 4:52 AM, Antoine Pitrou wrote: > It is not a coincidence, IMO, that the three persons arguing that the > current behaviour is reasonable are implementors of datetime modules: > you, Tim and Alexander. All other people find the behaviour baffling. Wrong. I don't. What I find baffling and annoying is the endlessly repeated false claim that there is nothing special about midnight, as if a thousands repetitions will change the fact. Ditto for the claim that there is nothing problematical about using 'a' as a substitute for 'a is not None'. Surely you know that the subset of people who pay attention to an endless drumbeat thread like this is severely biased. -- Terry Jan Reedy From barry at python.org Thu Mar 6 21:21:36 2014 From: barry at python.org (Barry Warsaw) Date: Thu, 6 Mar 2014 15:21:36 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> <lfa1as$8bm$1@ger.gmane.org> <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140306152136.537d79e6@anarchist.wooz.org> On Mar 07, 2014, at 02:40 AM, Stephen J. Turnbull wrote: >I also value the psychological benefit of being precise here. I'm not >even sure it's possible to fool 'is' into thinking some other object >is None, but people do a great job of convincing themselves of things >like "datetime.time objects can't be false" all the time. Use the >"is None" idiom and you know exactly what you're doing. This latter >is close to "just style", of course. Not just "you" (i.e. the author of the code), but everyone who comes after you and reads it. Being more explicit with identity tests against None assists future minions in understanding the intent of the code. When using the Socratic Method Of Code Reviews, I always ask about whether the author really expects the 'var' in "if var:" to be any truth-y value. The answer is usually "no", in which case I encourage tightening up the conditional. There are valid cases where a test is really trying to catch any truth-y or false-y value. I usually encourage a comment to clarify the author's intent in those cases. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/e5450795/attachment-0001.sig> From barry at python.org Thu Mar 6 21:29:36 2014 From: barry at python.org (Barry Warsaw) Date: Thu, 6 Mar 2014 15:29:36 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> Message-ID: <20140306152936.3c990c46@anarchist.wooz.org> On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote: >Instances of existing code that would be broken by the change: 0 You can't know this because you can't audit all the Python code in the world. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/36a8aba3/attachment.sig> From g.brandl at gmx.net Thu Mar 6 22:36:23 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 06 Mar 2014 22:36:23 +0100 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> Message-ID: <lfapo9$t5i$1@ger.gmane.org> Am 06.03.2014 11:11, schrieb Nick Coghlan: > > On 6 Mar 2014 13:55, "Sturla Molden" > <sturla.molden at gmail.com > <mailto:sturla.molden at gmail.com>> wrote: >> >> "goto fail" is a well-known error handling mechanism in open source software, > widely reputed for its robusteness: >> >> > http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c >> >> > https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c >> >> I believe Python needs to add support for this superior paradigm. > > Unfortunately, we can't throw stones about those, since we currently have > hostname matching off by default and expect developers to turn it on. And on another level, we use the same error handling method in C code (which is not bad per se of course). The literal code "goto fail;" actually occurs about 200 times in the CPython sources :) Aaand... we also don't have a policy to always use braces... Georg From kaiser.yann at gmail.com Thu Mar 6 22:41:56 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Thu, 6 Mar 2014 22:41:56 +0100 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <lfapo9$t5i$1@ger.gmane.org> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> Message-ID: <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> What does this add that we can't do with control flow tools such as try..finally or context managers? On 6 March 2014 22:36, Georg Brandl <g.brandl at gmx.net> wrote: > Am 06.03.2014 11:11, schrieb Nick Coghlan: >> >> On 6 Mar 2014 13:55, "Sturla Molden" >> <sturla.molden at gmail.com >> <mailto:sturla.molden at gmail.com>> wrote: >>> >>> "goto fail" is a well-known error handling mechanism in open source software, >> widely reputed for its robusteness: >>> >>> >> http://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c >>> >>> >> https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c41d358a3b:lib/x509/verify.c >>> >>> I believe Python needs to add support for this superior paradigm. >> >> Unfortunately, we can't throw stones about those, since we currently have >> hostname matching off by default and expect developers to turn it on. > > And on another level, we use the same error handling method in C code > (which is not bad per se of course). The literal code "goto fail;" actually > occurs about 200 times in the CPython sources :) > > Aaand... we also don't have a policy to always use braces... > > Georg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From ethan at stoneleaf.us Thu Mar 6 22:35:36 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Mar 2014 13:35:36 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> Message-ID: <5318EA28.3040305@stoneleaf.us> On 03/05/2014 07:42 AM, Paul Moore wrote: > On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote: >> Let me show you an actual zero: >> >> if event.num_attendants: >> prepare_cake() >> else: >> cancel_event() >> >> When no one is coming to your party, is is clearly a different >> condition than if any number of people are coming to your event. When >> you read "if num_attendants", you can clearly tell this is going to do >> something depending on if people are coming or not. > > I'm sorry, are you really trying to say that the above code is better than > > if event.num_attendants != 0: > prepare_cake() > else: > cancel_event() > > ? Yes. This is Python. blah blah blah != 0 is unnecessary boiler-plate [1]. -- ~Ethan~ [1] In most cases. I have used `== 0` or `!= 0` when it makes the code clearer, but that's pretty rare. -- ~Ethan~ From harrismh777 at gmail.com Thu Mar 6 23:23:44 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 14:23:44 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> Message-ID: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote: {snip} hi Guido, thanks for your responses, your candor, and your kindness; I think dialogue will work, and I promise to keep the rhetoric to a bare minimum, if at all. :) There really are only two questions I am going to try to answer here, and some of the answer goes back to Stefan's (and others) questions as well, so I hope its helpful. > So, by context you really seem to be referring to some specific state of > the program. Perhaps you are even talking about extending the existing > notion of numeric context used by Python's Decimal type. But it's not very > clear because "you change the context" is also a common term for changing > the topic of a discussion. > This is what I mean by "context," for this entire discussion, mostly having to do with Decimal and the "context manager" and these two lines: getcontext() with localcontext(ctx=None) as context_manager Allow me to back up a bit... the pdeclib module was not difficult for me because of the mathematics (no brag) I can code transcendental functions in my sleep. The hard part of that project last week was coming to grips with the context manger, and the parameterization of context attributes in the Decimal class. Back in the day when I did this work for Rexx at IBM I used NUMERIC DIGITS and NUMERIC FUZZ to emulate (in a crude way) what Decimal is doing correctly with its context, with the context manager. I credit Oscar Benjamin for helping me get up to speed with this concept (and I am so glad he did too, because its key for this entire discussion. Before we go on, allow me to explain my italics of *Number *and *PythonNumber.* These are each an abstract base class. They are a class that is never instantiated, but from which all other "numbers" are derived as objects. (more below) > I'm sorry, but I am *still* unsure about what you mean by "context". > Python does not need a context object to switch between complex and real > number processing -- it's all done through operator overloading. > (more on this below, but I am thinking beyond overloading/ actually, virtual functions and templates) > Actually, modern Python is considered object-oriented. > Thank you. This has everything to do with this discussion, but allow me to take a humorous break and relate an anecdote... I have had numerous discussions with T.R. and C.A and S.D. and others where in the end python was object based. Ha! Nope, you have seen it right here, the BDfL says its "object oriented," and that's the end of the argument! (ok, tiny rhetoric) > > Unifying the >> number system on a computer does not have to be grounded in paradigms >> that serviced the >> industry (and academy) for the past 40 years; >> > {snip} > > Now you've got a specific proposal and we can discuss pros and cons. This > is a topic that comes up occasionally and it is indeed a pretty interesting > trade-off. I suspect the large (and fast-growing) SciPy community would > prefer to keep binary floating point, because that's what their libraries > written in Fortran or C++ want, and extra binary/decimal conversions at the > boundaries would just slow things down. But I'm really speculating here -- > maybe they don't need high-performance conversion between binary and > decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe > it varies per application. > Here is the main response for this discussion, and I will endeavor to keep the rhetoric down, I promise. This may even be easier to accomplish in 3.x without breaking anyone, or any third party software either; depends In my view numbers are objects (like circles, squares and triangles; in polymorphism discussions). Numbers are nothing more than objects which have "contexts" just like circles and squares and triangles are nothing more than objects that have "contexts". In the following discussion think in terms of object oriented (as in Grady Booch) rather than any specific language--- and certainly not python syntax. um, think C++ for now, if you must think of any. A *Number *is an abstract base class with "context" attributes and pure virtual functions--- it will never be instantiated---and *PythonNumber *will derive from that; also an abstract base class. Here is the Class hierarchy I am proposing: *Number* *PythonNumber* Decimal Real <======= this is the default BinaryIEEE85 Real_b &c (...) omitted for clarity Complex Quaternion Rational Fraction &c-------------------------------------------------- Well, there it is. The idea is that any function or method that takes a *Number* reference or a *PythonNumber *reference does not have to interrogate what type of *Number* it is... it just works (why?) glad you asked, because of polymorphism and a three pointer hop down a virtual function table! If a function or method gets a *Number *parm in as in method(num) then to get the pretty print representation of the number is simply num.pretty_print(), or to get an evaluation of the number is num.eval(), or to get the string rep of the number is num.str()... and on and on. The point is that other parts of the system no longer need to know what "type" of number object it is, the right code gets called because of virtual functions, operator overloading, and the beautiful virtual function table (all under the covers, not MAGIC, but polymorphism). Python at present is disjointed when it comes to number as a concept. In my view numbers need to be objects, and the Class hierarchy (see my simplified version above) needs to unify numbers across the python boards (so to speak) so that "types" are only important to the implementors, and so that adding new "types" is easy, concise, and coherent. I realize that this suggestion is over the top in terms of size and weight (Stefan's concern above). But not really, when you think of conceptualizing the unification of numbers under *Number *and *PythonNumber *. There may be other kinds of numbers too, besides these... like maybe *GmpyNumber* of *BC_Number*, or others. The bottom line is that we have an object oriented language in front of us that has the software engineering advantages which would allow this kind of hierarchy and conceptualization of a unified number system. Let's leverage our technology for mutual advantage across python communities? Thanks for your consideration, Sincerely, Kind regards, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3bbe3c10/attachment-0001.html> From greg.ewing at canterbury.ac.nz Thu Mar 6 23:38:42 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Mar 2014 11:38:42 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> Message-ID: <5318F8F2.5010209@canterbury.ac.nz> Tim Peters wrote: > The only difference between time and timedelta is in implementation > details. Only if you define it that way. To my way of thinking, a time-of-day is like a mod-24-hour datetime, not a mod-24-hour timedelta. It seems you're thinking of it as a mod-24-hour timedelta. Given that, I can see why you think that it's bizarre to attach a timezone to one. In fact, I'd say if that's really how a time object is meant to be interpreted, they shouldn't even be *allowed* to have a timezone, any more than a timedelta can. In any case, I don't see how this interpretation justifies the truthiness behaviour with timezones. From my reading of the docs, the *intention* seems to be for midnight UTC to be false and all other times to be true. I would never have suspected the actual behaviour with negative offsets, which looks like an outright bug to me. Or is there some reason for it that I'm not seeing? -- Greg From rosuav at gmail.com Thu Mar 6 23:41:08 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 09:41:08 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> Message-ID: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> On Fri, Mar 7, 2014 at 9:23 AM, Mark H. Harris <harrismh777 at gmail.com> wrote: > Well, there it is. The idea is that any function or method that > takes a Number reference > or a PythonNumber reference does not have to interrogate what type of > Number it is... it just > works (why?) glad you asked, because of polymorphism and a three pointer hop > down a virtual > function table! That's way too concrete for Pythonic style :) All you need to do is have each numeric type implement the same operations, and there you are, as the Lord Chancellor said, out of your difficulty at once! And that's what we already have. You can add two numbers and they'll simply add. In fact, a C++ style "hop down a virtual function table" couldn't handle that. I could make a class hierarchy like you describe, and have each one have an add() method or operator+(), but somewhere along the line, something still has to cope with the fact that it could be given any sort of number - there has to be code someplace that handles the adding of a (Decimal) Real and a Real_b, because some day, someone's going to do it. (The correct response might be to raise TypeError, on the basis that that action merges two separate inaccuracies. But more likely, the correct response is to convert one of them to the other type.) What does your proposed hierarchy offer that numbers.Number doesn't? ChrisA From rosuav at gmail.com Thu Mar 6 23:52:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 09:52:28 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318F8F2.5010209@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> Message-ID: <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> On Fri, Mar 7, 2014 at 9:38 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > It seems you're thinking of it as a mod-24-hour timedelta. > Given that, I can see why you think that it's bizarre > to attach a timezone to one. In fact, I'd say if that's > really how a time object is meant to be interpreted, > they shouldn't even be *allowed* to have a timezone, > any more than a timedelta can. Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00 America/New_York? If you add a date to it, you could get an aware datetime that represents an exact instant, and compare them. Can you compare them, as they are? Is it reliable? This is nothing to do with timedelta versus datetime. It's to do with attaching timezone information to a dateless time. ChrisA From barry at python.org Thu Mar 6 23:45:47 2014 From: barry at python.org (Barry Warsaw) Date: Thu, 6 Mar 2014 17:45:47 -0500 Subject: [Python-ideas] Add "goto fail" to Python? References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> Message-ID: <20140306174547.00eca988@anarchist.wooz.org> On Mar 06, 2014, at 10:36 PM, Georg Brandl wrote: >Aaand... we also don't have a policy to always use braces... Yeah, I've been having an interesting conversation on FB about this very thing (spurred by a code sample of the now-infamous Apple goto fail fail :). I personally wouldn't be opposed to changing PEP 7 to require it. But I remember us at least *discussing* it at the time and explicitly deciding against requiring those braces. -Barry (OTOH, I had to physically torture Emacs to write the Apple goto fail example, so I guess the answer is to just use a decent text editor. M-x wink RET). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/dfd3caf1/attachment.sig> From shai at platonix.com Thu Mar 6 23:55:29 2014 From: shai at platonix.com (Shai Berger) Date: Fri, 07 Mar 2014 00:55:29 +0200 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306152136.537d79e6@anarchist.wooz.org> References: <201403051216.12392.shai@platonix.com> <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp> <20140306152136.537d79e6@anarchist.wooz.org> Message-ID: <2750802.jzuIGEshV5@deblack> On Thursday 06 March 2014 15:21:36 Barry Warsaw wrote: > On Mar 07, 2014, at 02:40 AM, Stephen J. Turnbull wrote: > >I also value the psychological benefit of being precise here. I'm not > >even sure it's possible to fool 'is' into thinking some other object > >is None, but people do a great job of convincing themselves of things > >like "datetime.time objects can't be false" all the time. Use the > >"is None" idiom and you know exactly what you're doing. This latter > >is close to "just style", of course. > > Not just "you" (i.e. the author of the code), but everyone who comes after > you and reads it. Being more explicit with identity tests against None > assists future minions in understanding the intent of the code. When using > the Socratic Method Of Code Reviews, I always ask about whether the author > really expects the 'var' in "if var:" to be any truth-y value. The answer > is usually "no", in which case I encourage tightening up the conditional. > Wait -- you just argued for replacing if time_obj: not by if time_obj is not None: -- which is a more relaxed conditional -- but rather by if isinstance(time_obj, datetime.time): That seems quite un-pythonic, and *that* - from you - is quite unlikely, so I probably misunderstood. Do you mind clarifying? [A guess: The argument you intended to make was about an "if not var:" test changed to "if var is None:". It does not survive inversion of sense well.] Thanks, Shai. From dholth at gmail.com Thu Mar 6 23:56:14 2014 From: dholth at gmail.com (Daniel Holth) Date: Thu, 6 Mar 2014 17:56:14 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318EA28.3040305@stoneleaf.us> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <5318EA28.3040305@stoneleaf.us> Message-ID: <CAG8k2+6o7kezaet2m8jLCrPU3jnex2-K4wz=q4AOEyOK27F4Rg@mail.gmail.com> The obvious solution is just to fix the Python if statement to remove implicit bool coercion. So they would all evaluate to if isinstance (x, bool) and x. Problem solved. On Mar 6, 2014 5:27 PM, "Ethan Furman" <ethan at stoneleaf.us> wrote: > On 03/05/2014 07:42 AM, Paul Moore wrote: > >> On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote: >> >>> Let me show you an actual zero: >>> >>> if event.num_attendants: >>> prepare_cake() >>> else: >>> cancel_event() >>> >>> When no one is coming to your party, is is clearly a different >>> condition than if any number of people are coming to your event. When >>> you read "if num_attendants", you can clearly tell this is going to do >>> something depending on if people are coming or not. >>> >> >> I'm sorry, are you really trying to say that the above code is better than >> >> if event.num_attendants != 0: >> prepare_cake() >> else: >> cancel_event() >> >> ? >> > > Yes. This is Python. blah blah blah != 0 is unnecessary boiler-plate [1]. > > -- > ~Ethan~ > > > [1] In most cases. I have used `== 0` or `!= 0` when it makes the code > clearer, but that's pretty rare. > > -- > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/f135fa49/attachment-0001.html> From harrismh777 at gmail.com Thu Mar 6 22:37:31 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 13:37:31 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <loom.20140306T131407-724@post.gmane.org> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <loom.20140306T131407-724@post.gmane.org> Message-ID: <f5d71e93-c96b-471b-abde-04bec53feef3@googlegroups.com> On Thursday, March 6, 2014 6:36:46 AM UTC-6, Stefan Krah wrote: > > > As I understand, you are suggesting some kind of number system with > symbolic representations that can be converted to decimal if needed. > > The topic is so vast that the only chance of it happening is for > someone to provide an implementation on PyPI and see if people > like it. There are packages for symbolic mathematics like sympy, > perhaps that is a start. Yes, that is correct (in a way). I will respond to this better when I respond to Guido, so be sure to read that post. > > Regarding decimal literals: > > Possible in 3.x, but it would require some investigation if > people really want arbitrary precision arithmetic by default > rather than e.g. IEEE Decimal64. > Good. Dialogue is a good thing. Looking forward to it. Regarding decimal floats by default: > > Perhaps in 4.0, but IMO only with *full* backing by the large SciPy > and NumPy communities. My guess is that it's not going to happen, not > in the least because the literature for floating point algorithms > (with proofs!) has an overwhelming focus on binary floating point. > I know. Again, when I respond to Guido I'll address this a bit. Thanks Stefan, I appreciate your inputs, and your work... keep it up! marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/d5e20afd/attachment.html> From greg.ewing at canterbury.ac.nz Fri Mar 7 00:12:55 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Mar 2014 12:12:55 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <loom.20140306T120303-396@post.gmane.org> <6ED9830A-678A-445C-9C29-FC6A74B3F569@yahoo.com> Message-ID: <531900F7.1090101@canterbury.ac.nz> Andrew Barnert wrote: > I agree with Stefan here. Treating midnight as zero is not at all baffling. > There's a reason modular arithmetic is taught to kids as "clock arithmetic", I still say this is confusing times with timedeltas. It makes no conceptual sense to add two times of day together. You can only make it work if you abuse one type to mean two different things. We keep datetimes and timedeltas clearly separated. Why shouldn't we do the same for times of day? -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 7 00:48:06 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Mar 2014 12:48:06 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lfaigr$8t4$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org> Message-ID: <53190936.6030608@canterbury.ac.nz> Terry Reedy wrote: > What I find baffling and annoying is the endlessly > repeated false claim that there is nothing special about midnight, I think what people are really saying is that it's not special *enough* to be worth breaking people's intuition about the truthiness of structured data types in Python. Anyhow, it seems that the original motivation for the current truthiness behaviour isn't really "midnight is special", but rather "zero hours is special". In other words, if you turn your head sideways and view a time() object not as a time of day, but as a difference between two times of day, then a time() representing midnight morphs into a timedelta of zero hours, and it kind of sort of makes sense to treat that as false. If that's the intended interpretation of a time() object, the docs do a poor job of conveying it. A time() object is described as representing "a (local) time of day", not "an amount of time since midnight". The "local" even suggests that, if some midnight is to special, it would be *local* midnight -- but this is not the case for time() objects with a timezone. It seems to me that those claiming "midnight is special" are inventing a justification after the fact -- one that is not supported by the actual behaviour. Altogether, the whole thing seems to be a mess. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 7 00:51:50 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Mar 2014 12:51:50 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306152936.3c990c46@anarchist.wooz.org> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> Message-ID: <53190A16.4020201@canterbury.ac.nz> Barry Warsaw wrote: > On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote: > >>Instances of existing code that would be broken by the change: 0 > > You can't know this because you can't audit all the Python code in the world. I think he means instances that have been pointed out so far in this discussion. -- Greg From cs at zip.com.au Fri Mar 7 00:59:21 2014 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 7 Mar 2014 10:59:21 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140305120410.GY28804@ando> References: <20140305120410.GY28804@ando> Message-ID: <20140306235921.GA20568@cskk.homeip.net> Coming to this very late, but I was interested in the first thread... On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote: > On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote: > > I don't agree with Mark's proposal in this thread but I would like to > > have decimal literals e.g. 1.23d, > > +1 on that. Although that will depend on how big a slowdown it causes to > Python's startup time. +1 on 123d decimal literals. > > and I would also use Fraction > > literals if available e.g. 1/3F. > > Out of curiosity, why F rather than f? Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats in play in Python right now. If ever we get ambiguity on how numbers are done, this may be useful in the transition or experiment phase. So of course "F" is reasonable for fractions. But how hard does that make the grammar? Because the parser now has to grab the entire "1/3F" to construct the fraction. You can't just stick it in the lexer at that point. Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag which is treated differently on the right hand side of a division operation: if encoutered, division now returns a Fraction instead of an int or float; this feels like something that could easily have nasty side effects. And, speaking personally, a big -1 on Mark's "abstract all the numbers". Like other posters, I agree that a number's internal implementation/representation affects its semantics. You can't just wave a wand and say "it should all be abstract"; there will be side-effects. You can wave a wand and say "from here on in this code, unadorned numbers make Decimals, not IEEE floats". I'm +0.5 on something like: from __future__ import decimal_floats to imply making Decimals from unadorned literals. "__future__" may be the wrong pseudomodule name. Not least, it makes testing the side-effects much easier because you can tweak a nice big chunk of code without editing it internally. > (By the way, I think it is somewhat amusing that Python not only has a > built-in complex type, but also *syntax* for creating complex numbers, > but no built-in support for exact rationals.) Harder to parse (see above), smaller use case maybe. It is not a bad idea, just not yet done... Cheers, -- Cameron Simpson <cs at zip.com.au> The concept is interesting and well-formed, but in order to earn better than a 'C,' the idea must be feasible. --A Yale University management professor in response to Fred Smith's paper proposing reliable overnight delivery service. (Smith went on to found Federal Express Corp.) From greg.ewing at canterbury.ac.nz Fri Mar 7 01:04:56 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Mar 2014 13:04:56 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> Message-ID: <53190D28.50205@canterbury.ac.nz> Chris Angelico wrote: > Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00 > America/New_York? If you think of times of day as modular, then it doesn't make sense to ask whether one is earlier than another. But to clarify my position: It's okay to attach a timezone to a *time of day*. It's *not* okay to attach a timezone to a *difference* between times of day. This is one of the reasons it's imortant to keep the two concepts clearly separated, as we do for datetimes and timedeltas. Thinking that time(0, 0, 0) should be false seems to be a result of conflating the two. -- Greg From tim.peters at gmail.com Fri Mar 7 01:08:38 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 18:08:38 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53190936.6030608@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz> Message-ID: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com> [Greg Ewing] > ... > Altogether, the whole thing seems to be a mess. Although a mess that fits in a thimble that successfully hid in a dark corner unnoticed for a decade ;-) From rosuav at gmail.com Fri Mar 7 01:13:25 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 11:13:25 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53190D28.50205@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> <53190D28.50205@canterbury.ac.nz> Message-ID: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> On Fri, Mar 7, 2014 at 11:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Chris Angelico wrote: >> >> Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00 >> America/New_York? > > > If you think of times of day as modular, then it doesn't > make sense to ask whether one is earlier than another. It should be possible to ask if they're equal, at least. Are they? > But to clarify my position: It's okay to attach a > timezone to a *time of day*. It's *not* okay to attach > a timezone to a *difference* between times of day. What operations can you do on a time-of-day-with-timezone? Give me some examples, and I'll show you, with the above two examples, how quirky that can be. ChrisA From rosuav at gmail.com Fri Mar 7 01:17:55 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 11:17:55 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140306235921.GA20568@cskk.homeip.net> References: <20140305120410.GY28804@ando> <20140306235921.GA20568@cskk.homeip.net> Message-ID: <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com> On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote: > Aside: Unless you do something obtuse, like make "3F" be a regular > number with a special internal flag which is treated differently > on the right hand side of a division operation: if encoutered, > division now returns a Fraction instead of an int or float; this > feels like something that could easily have nasty side effects. Thanks, now I can take the paper bag off my head. I'm not the only one! I thought exactly what you say here, that 3F would have to be a magical type of integer. But actually, all it has to be is Fraction(3) and everything will work perfectly. >>> 1/Fraction(3) Fraction(1, 3) ChrisA From harrismh777 at gmail.com Fri Mar 7 01:17:55 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 16:17:55 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> Message-ID: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote: What does your proposed hierarchy offer that numbers.Number doesn't? > > I just had one more thought along this line; consider this: >>> from pdeclib import * >>> >>> s1=sqrt(2.01) >>> s1 Decimal('1.41774468787578244511883132198668766452744') >>> s2=sqrt(d(2.01)) >>> s2 Decimal('1.41774468787578252029556185427085779261123') >>> >>> s1**2 Decimal('2.00999999999999978683717927196994423866272') >>> s2**2 Decimal('2.01000000000000000000000000000000000000000') >>> >>> Which one is right, s1 or s2 ? Well, clearly s1 is wrong. And yet, s1 was coded by giving the system a normal human number, at a human level, and very innocently s1 is really badly broken but not immediately noticed until we try to square it... s2 on the other hand is correct, but we had to go through some hoops to make sure that the system received the correct type. The system should be able to make this determination without the user having to determine types. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/146e8ea0/attachment.html> From donald at stufft.io Fri Mar 7 01:20:48 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 19:20:48 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> <53190D28.50205@canterbury.ac.nz> <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> Message-ID: <9F5908A8-8B2B-4320-8CEE-B61F99483222@stufft.io> > On Mar 6, 2014, at 7:13 PM, Chris Angelico <rosuav at gmail.com> wrote: > >> On Fri, Mar 7, 2014 at 11:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> Chris Angelico wrote: >>> >>> Which is the earlier time: 18:00:00 Australia/Melbourne, or 03:00:00 >>> America/New_York? >> >> >> If you think of times of day as modular, then it doesn't >> make sense to ask whether one is earlier than another. > > It should be possible to ask if they're equal, at least. Are they? > >> But to clarify my position: It's okay to attach a >> timezone to a *time of day*. It's *not* okay to attach >> a timezone to a *difference* between times of day. > > What operations can you do on a time-of-day-with-timezone? Give me > some examples, and I'll show you, with the above two examples, how > quirky that can be. > It's not an operation but a time object with a timezone is perfect for representing an event which occurs everyday at 11:30 EST > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rosuav at gmail.com Fri Mar 7 01:29:34 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 11:29:34 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> Message-ID: <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> On Fri, Mar 7, 2014 at 10:58 AM, Mark H. Harris <harrismh777 at gmail.com> wrote: > But, it should not do this: > >>>> from pdeclib import * >>>> x=1 >>>> y=Decimal(.1) >>>> x+y > Decimal('1.10000000000000000555111512312578270211816') >>>> > > The policy hidden under the abstraction way down in the guts of the Class > hierarchy should make the decision to do this: > >>>> ====== RESTART ===== >>>> from pdeclib import * >>>> x=1 >>>> y=.1 >>>> >>>> def add_(x, y): > return d(x)+d(y) > >>>> add_(x, y) > Decimal('1.1') >>>> > > Please don't pick at this. The point is not the code or the syntax. The > point > is that the policy in a correctly related Class hierarchy can handle > somebody > asking the system to add a 1.23b with a 1.23d , and if setup correctly, > will > just work. If you had a literal syntax "0.1d" meaning Decimal("0.1"), this would be solved. The only reason your code seems to work is that you're rounding off your binary floats instead of using them with as much accuracy as they have. You're deceiving yourself that str(float('0.1')) seems to round-trip, and therefore that's the right way to get a decimal value from a float. But with floats completely out of the picture, there's no need to do any of this. >>> x=1 >>> y=Decimal(".1") # y=.1d >>> x+y Decimal('1.1') So... I'm +1 for adding a literal syntax for decimals. +1 for adding a stating-the-default for floats (do it straight away, then code that uses it can be backported all the way even when there's a change of default). +0.5 for adding a syntax for fractions; support in principle but the devil's in the details. -1 for trying to unify everything. ChrisA From python at mrabarnett.plus.com Fri Mar 7 01:32:16 2014 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 07 Mar 2014 00:32:16 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140306235921.GA20568@cskk.homeip.net> References: <20140305120410.GY28804@ando> <20140306235921.GA20568@cskk.homeip.net> Message-ID: <53191390.7050804@mrabarnett.plus.com> On 2014-03-06 23:59, Cameron Simpson wrote: > Coming to this very late, but I was interested in the first thread... > > On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote: >> On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote: >> > I don't agree with Mark's proposal in this thread but I would like to >> > have decimal literals e.g. 1.23d, >> >> +1 on that. Although that will depend on how big a slowdown it causes to >> Python's startup time. > > +1 on 123d decimal literals. > >> > and I would also use Fraction >> > literals if available e.g. 1/3F. >> >> Out of curiosity, why F rather than f? > > Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats > in play in Python right now. > > If ever we get ambiguity on how numbers are done, this may be useful > in the transition or experiment phase. > > So of course "F" is reasonable for fractions. But how hard does > that make the grammar? Because the parser now has to grab the entire > "1/3F" to construct the fraction. You can't just stick it in the lexer > at that point. > I think it would be confusing if there were "f" for "float" and "F" for "fraction". How about "r" for "rationals"? > Aside: Unless you do something obtuse, like make "3F" be a regular > number with a special internal flag which is treated differently > on the right hand side of a division operation: if encoutered, > division now returns a Fraction instead of an int or float; this > feels like something that could easily have nasty side effects. > > And, speaking personally, a big -1 on Mark's "abstract all the > numbers". Like other posters, I agree that a number's internal > implementation/representation affects its semantics. You can't just > wave a wand and say "it should all be abstract"; there will be > side-effects. > > You can wave a wand and say "from here on in this code, unadorned > numbers make Decimals, not IEEE floats". > > I'm +0.5 on something like: > > from __future__ import decimal_floats > > to imply making Decimals from unadorned literals. "__future__" may > be the wrong pseudomodule name. > > Not least, it makes testing the side-effects much easier because you > can tweak a nice big chunk of code without editing it internally. > >> (By the way, I think it is somewhat amusing that Python not only has a >> built-in complex type, but also *syntax* for creating complex numbers, >> but no built-in support for exact rationals.) > <pedantic>Python doesn't have a syntax for creating complex numbers; it has a syntax for creating imaginary numbers.</pedantic> > Harder to parse (see above), smaller use case maybe. It is not a > bad idea, just not yet done... > From rosuav at gmail.com Fri Mar 7 01:38:34 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 11:38:34 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> Message-ID: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> On Fri, Mar 7, 2014 at 11:17 AM, Mark H. Harris <harrismh777 at gmail.com> wrote: > Well, clearly s1 is wrong. And yet, s1 was coded by giving the system a > normal > human number, at a human level, and very innocently s1 is really badly > broken > but not immediately noticed until we try to square it... > > s2 on the other hand is correct, but we had to go through some hoops > to make sure that the system received the correct type. The system should > be able to make this determination without the user having to determine > types. Once again, the problem occurs only because you're using a float literal, and 2.01 can't perfectly round-trip. >>> Decimal(2.01) Decimal('2.0099999999999997868371792719699442386627197265625') That's pretty much the value you had (I have a few more digits there, the square rooting and squaring probably cost some accuracy), so I'd say Decimal correctly round-tripped. The problem is the conversion from string (source code) to float to Decimal. You think that str()ing a float gives you a better round-trip, but that works only because you're using relatively small numbers. Hmm. Is the rounding done by float.__str__() an attractive nuisance? Would it be better to show exactly what's stored, if only to remove the temptation to treat str(f) as "more correct" than f? ChrisA From rosuav at gmail.com Fri Mar 7 01:45:33 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 11:45:33 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <53191390.7050804@mrabarnett.plus.com> References: <20140305120410.GY28804@ando> <20140306235921.GA20568@cskk.homeip.net> <53191390.7050804@mrabarnett.plus.com> Message-ID: <CAPTjJmrmDHe740bMS+s+oArnBL7a+zFoP4dnu9Rvri=LEL7wQg@mail.gmail.com> On Fri, Mar 7, 2014 at 11:32 AM, MRAB <python at mrabarnett.plus.com> wrote: > I think it would be confusing if there were "f" for "float" and "F" for > "fraction". How about "r" for "rationals"? Since the string prefixes are all case insensitive, I'd be very much surprised if f and F did different things. > <pedantic>Python doesn't have a syntax for creating complex numbers; it > has a syntax for creating imaginary numbers.</pedantic> <pedantic> >>> type(1j) <class 'complex'> </pedantic> ChrisA From rob.cliffe at btinternet.com Fri Mar 7 01:44:00 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 07 Mar 2014 00:44:00 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53190A16.4020201@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> Message-ID: <53191650.5020208@btinternet.com> On 06/03/2014 23:51, Greg Ewing wrote: > Barry Warsaw wrote: >> On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote: >> >>> Instances of existing code that would be broken by the change: 0 >> >> You can't know this because you can't audit all the Python code in >> the world. > > I think he means instances that have been pointed out > so far in this discussion. > Quite. If you read my post, I was initialising totals. I invited the universe to add to them. From harrismh777 at gmail.com Fri Mar 7 00:58:11 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 15:58:11 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> Message-ID: <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote: > > What does your proposed hierarchy offer that numbers.Number doesn't? > > hi Chris, numbers.Number is the right concept, but not the correct relationship... in sixty's vernacular, its the right hoo hoo but the wrong tah tah... Here is the numbers.Number hierarchy: CLASSES builtins.object Number Complex Real Rational Integral Compare numbers.Number with my proposal. (I am not wanting to go into semantics at this point, but the numbers.Number model is not unified and has problems. If it were ok, we would not be having this discussion. ) The age old question, "How's that working for you?" Well, not too well. Chris, its not just the virtual function table, its also operator overloading, and its policy handled by (AI) that intelligently decides what to do if someone tries to add a real_b with a real_d. But, it should not do this: >>> from pdeclib import * >>> x=1 >>> y=Decimal(.1) >>> x+y Decimal('1.10000000000000000555111512312578270211816') >>> The policy hidden under the abstraction way down in the guts of the Class hierarchy should make the decision to do this: >>> ====== RESTART ===== >>> from pdeclib import * >>> x=1 >>> y=.1 >>> >>> def add_(x, y): return d(x)+d(y) >>> add_(x, y) Decimal('1.1') >>> Please don't pick at this. The point is not the code or the syntax. The point is that the policy in a correctly related Class hierarchy can handle somebody asking the system to add a 1.23b with a 1.23d , and if setup correctly, will just work. The idea is to unify numbers, so that efficient intelligent processing can occur without "types" or "limits". I'll say more later... after we see how folks respond. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/718b3c3d/attachment-0001.html> From skip at pobox.com Fri Mar 7 01:48:30 2014 From: skip at pobox.com (Skip Montanaro) Date: Thu, 6 Mar 2014 18:48:30 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz> <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com> Message-ID: <CANc-5Uxve4i40+hsv2B-SeQBvxUpt7uP=M2=4rj7kRmwMZ6O=Q@mail.gmail.com> > Although a mess that fits in a thimble that successfully hid in a dark > corner unnoticed for a decade ;-) +1 QOTW... Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/8d93a7df/attachment.html> From anikom15 at gmail.com Fri Mar 7 02:03:58 2014 From: anikom15 at gmail.com (=?iso-8859-1?Q?Westley_Mart=EDnez?=) Date: Thu, 6 Mar 2014 17:03:58 -0800 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> Message-ID: <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> Spaghetti? Seriously though, I don't see how this is any different than a regular goto. What's forcing users to use this only for errors? I understand that error- handling is what goto is most useful for in C, but if Python already has a simple and robust method for handling errors, what is goto going to add? > -----Original Message----- > From: Python-ideas [mailto:python-ideas-bounces+anikom15=gmail.com at python.org] > On Behalf Of Yann Kaiser > Sent: Thursday, March 06, 2014 1:42 PM > To: Georg Brandl > Cc: python-ideas > Subject: Re: [Python-ideas] Add "goto fail" to Python? > > What does this add that we can't do with control flow tools such as > try..finally or context managers? > > On 6 March 2014 22:36, Georg Brandl <g.brandl at gmx.net> wrote: > > Am 06.03.2014 11:11, schrieb Nick Coghlan: > >> > >> On 6 Mar 2014 13:55, "Sturla Molden" > >> <sturla.molden at gmail.com > >> <mailto:sturla.molden at gmail.com>> wrote: > >>> > >>> "goto fail" is a well-known error handling mechanism in open source > software, > >> widely reputed for its robusteness: > >>> > >>> > >> http://opensource.apple.com/source/Security/Security- > 55471/libsecurity_ssl/lib/sslKeyExchange.c > >>> > >>> > >> > https://www.gitorious.org/gnutls/gnutls/source/6aa26f78150ccbdf0aec1878a41c17c > 41d358a3b:lib/x509/verify.c > >>> > >>> I believe Python needs to add support for this superior paradigm. > >> > >> Unfortunately, we can't throw stones about those, since we currently have > >> hostname matching off by default and expect developers to turn it on. > > > > And on another level, we use the same error handling method in C code > > (which is not bad per se of course). The literal code "goto fail;" actually > > occurs about 200 times in the CPython sources :) > > > > Aaand... we also don't have a policy to always use braces... > > > > Georg > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From steve at pearwood.info Fri Mar 7 02:12:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 12:12:29 +1100 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> Message-ID: <20140307011229.GM28804@ando> Guys, Sturla's original email was quite subtle, but you may have missed that he is proposing this as a joke PEP for April Fools Day, a bit like the introduction of GOTO or the retirement of Guido: https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html http://legacy.python.org/dev/peps/pep-0401/ As a joke, it pokes fun at Apple's recent "goto fail" security breach. Unfortunately the Python C source code apparently uses exactly the same "goto fail" idiom, although hopefully not as poorly as Apple did. -- Steven From tim.peters at gmail.com Fri Mar 7 02:19:28 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 19:19:28 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318F8F2.5010209@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> Message-ID: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> [Tim] >> The only difference between time and timedelta is in implementation >> details. [Greg] > Only if you define it that way. Sorry, don't know what you mean there. But then I don't draw much distinction between "defining" something and "implementation details". `time` and `timedelta` objects are both exact values with microsecond resolution; the operations each supports is a matter of definition + implementation. > To my way of thinking, a time-of-day is like a mod-24-hour datetime, > not a mod-24-hour timedelta. Heh. I'll ponder that and see if I can make sense of it tomorrow ;-) > It seems you're thinking of it as a mod-24-hour timedelta. No. I'm saying: (1) it _can_ be thought of that way; and, (2) while that's not how `time` was initially released, there was some effort not to _preclude_ later extensions viewing it that way. As is, the datetime module offers no obvious solution to problems like "the clock reads 6:43 now; what will it read 57 minutes from now?". `time` objects support no arithmetic of any kind. The world wouldn't really collapse if time(6, 43) + time(minute=57), or if time(6, 43) + timedelta(minutes=57), didn't blow up but returned the obvious result as a time object. > Given that, I can see why you think that it's bizarre > to attach a timezone to one. In fact, I'd say if that's > really how a time object is meant to be interpreted, > they shouldn't even be *allowed* to have a timezone, > any more than a timedelta can. No, it's bizarre to attach a timezone to a time object because most tzinfo subclasses don't - and can't - know what to return for the UTC offset in the absence of - at least - month and day info too. Hour, minute, second and microsecond aren't usually enough to determine whether "daylight time" is in effect, and most tzinfo subclasses do attempt to model daylight time. And because daylight time is a political conceit, most tzinfo subclasses also need year info. That has nothing to do with whether times are viewed abstractly, concretely, etc - it has to do with that time zones generally aren't _defined_ in terms of isolated time-of-day info. It's 7:13 PM in US Central. Is that daylight (CDT) or standard (CST) time? It's impossible to answer. What's the corresponding UTC time? Ditto. While time objects don't support arithmetic they do support all comparison operators. And this is another case where the docs explain _how_ comparison of aware time objects works, but a close reading reveals that the result generally makes no sense. In effect, the time objects are converted to UTC before comparison, but correct conversion to UTC is generally impossible without month, day and year info too. A user intimately familiar with how all this works can define tzinfo subclasses that make time object conversions with the kinds of errors they prefer. but that too is ... bizarre. > In any case, I don't see how this interpretation > justifies the truthiness behaviour with timezones. > From my reading of the docs, the *intention* seems > to be for midnight UTC to be false and all other > times to be true. I would never have suspected the > actual behaviour with negative offsets, which looks > like an outright bug to me. Or is there some reason > for it that I'm not seeing? Truthiness would become useful if time() (i.e., the naive time object with all 0 fields) acted as an additive identity in some extension that allowed time object addition. That's all. I put more thought into writing that sentence than into what truthiness of aware time objects returned - and I bet it shows ;-) If more thought had gone into it, and assuming it remained the case that time objects had to support tzinfo members (despite that doing so makes little sense), and assuming it remained the case that we wanted to keep the _possibility_ of later time arithmetic open, I'd amend the truthiness rules to that False was equivalent to that the naive time converted to minutes minus the UTC offset (or 0 if tzinfo=None) is 0 mod 24*60. Which would be a long way of saying "midnight UTC", but a better way because: (1) again, it's not usually possible to do _correct_ conversion of a time object to UTC; and, (2) it's not "midnight" that's important to an additive identity, it's "0". From donald at stufft.io Fri Mar 7 02:25:10 2014 From: donald at stufft.io (Donald Stufft) Date: Thu, 6 Mar 2014 20:25:10 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> Message-ID: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> On Mar 6, 2014, at 8:19 PM, Tim Peters <tim.peters at gmail.com> wrote: > No, it's bizarre to attach a timezone to a time object because most > tzinfo subclasses don't - and can't - know what to return for the UTC > offset in the absence of - at least - month and day info too. It?s completely reasonable to attach a timezone to a time object. If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then absolutely that should be represented as a date time. But what if I want to schedule an event that occurs every day at 11:30AM EST? There?s no date associated with it (except an infinite set, or the span of my life I suppose!) it?s just 11:30AM EST. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/80dd4091/attachment-0001.sig> From kaiser.yann at gmail.com Fri Mar 7 02:32:10 2014 From: kaiser.yann at gmail.com (Yann Kaiser) Date: Fri, 7 Mar 2014 02:32:10 +0100 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <20140307011229.GM28804@ando> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> <20140307011229.GM28804@ando> Message-ID: <CANUJvPUsy2khWzAhBX+d2XhABk7z3=L4dVr5opZu6oitjEhisg@mail.gmail.com> Ah, my bad, I had received this message an hour before midnight in my timezone so my brain shorted out. Isn't this more of an insignificant-whitespace issue? We need more significant whitespace. On 7 March 2014 02:12, Steven D'Aprano <steve at pearwood.info> wrote: > Guys, > > Sturla's original email was quite subtle, but you may have missed that > he is proposing this as a joke PEP for April Fools Day, a bit like the > introduction of GOTO or the retirement of Guido: > > https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html > > http://legacy.python.org/dev/peps/pep-0401/ > > As a joke, it pokes fun at Apple's recent "goto fail" security breach. > Unfortunately the Python C source code apparently uses exactly the same > "goto fail" idiom, although hopefully not as poorly as Apple did. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From rosuav at gmail.com Fri Mar 7 02:34:37 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 12:34:37 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> Message-ID: <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> On Fri, Mar 7, 2014 at 12:25 PM, Donald Stufft <donald at stufft.io> wrote: > If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then > absolutely that should be represented as a date time. But what if I want > to schedule an event that occurs every day at 11:30AM EST? There?s > no date associated with it (except an infinite set, or the span of my life > I suppose!) it?s just 11:30AM EST. Repeating events need their own handling. Maybe it's safe to use 11:30AM local time, but what if you specify 1:30AM America/New_York? How can you render that onto a calendar? There'll be times when it breaks - one day a year when it's ambiguous, and one day a year when it doesn't exist. How can you work with a time that might not exist? ChrisA From abarnert at yahoo.com Fri Mar 7 02:32:45 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 17:32:45 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> Message-ID: <1394155965.8575.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Mark H. Harris <harrismh777 at gmail.com> Sent: Thursday, March 6, 2014 2:23 PM > Here is the Class hierarchy I am proposing: > >Number >? ? ? ?PythonNumber >? ? ? ? ? ? ? Decimal >? ? ? ? ? ? ? ? ? ? ?Real ? ? ? ? <======= this is the ?default >? ? ? ? ? ? ? BinaryIEEE85 >? ? ? ? ? ? ? ? ? ? ?Real_b >? ? ? ? ? ? ? ?&c (...) ? omitted for clarity >? ? ? ? ? ? ? ?Complex >? ? ? ? ? ? ? ? ? ? ? ?Quaternion >? ? ? ? ? ? ? ? Rational >? ? ? ? ? ? ? ? ? ? ? ? Fraction >? ? ? ? ? ? ? ? &c-------------------------------------------------- This kind of hierarchy doesn't work.?Most importantly,?you can have complex decimal floats and complex binary floats. This is why complex is a class template rather than a class in C++.?Also, this is ignoring the numerical tower?rationals are a specialization of reals, just as floats are. >? ? ? ?Well, there it is. ? The idea is that any function or method that takes a ?Number?reference >or a ?PythonNumber ?reference does not have to interrogate what type of Number?it is... it just >works (why?) glad you asked, because of polymorphism and a three pointer hop down a virtual >function table! 99% of the time you don't need these abstract base classes at all, because duck typing and operator overloading. But when you need it, it's already there, and does everything you're asking for. See below. >? ? ? ?If a function or method gets a Number parm in as in ? ?method(num) ? ?then to get the pretty >print representation of the number ?is simply ? ?num.pretty_print(), ? or to get an evaluation? >of the number is ? num.eval(), or to get the string rep of the number is ?num.str()... We already have the first and third as str() and repr(), and I'm not sure what the second is supposed to do. What does evaluating a number return other than the number itself? >? ? ? ?Python at present is disjointed when it comes to number as a concept. In my view numbers? >need to be objects, and the Class hierarchy (see my simplified version above) needs to unify numbers >across the python boards (so to speak) so that "types" are only important to the implementors, and >so that adding new "types" is easy, concise, and coherent.? This is all just plainly false. Numbers are objects. There is a hierarchy of abstract base classes for them?done correctly?in the numbers module. And they handle everything you're asking for, except for one thing:?There are no ABCs to distinguish binary vs. decimal floats. But I doubt that's useful for anything. What kind of code can you imagine that does need to distinguish, say, float vs. decimal.Decimal, but doesn't need to distinguish decimal.Decimal from mylib.MyDecimal? They're both inexact floating-point representations of reals. If the motivation for your whole idea is that you want a class hierarchy like the one in the numbers module but you didn't know that it already existed, then we're done. From tim.peters at gmail.com Fri Mar 7 02:36:29 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 19:36:29 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> Message-ID: <CAExdVNkjZB5rwV5c8KruVfV0c==i+uC_LLoFa=wkddWsVf+kBA@mail.gmail.com> [Donald Stufft] > It's completely reasonable to attach a timezone to a time object. > > If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then > absolutely that should be represented as a date time. But what if I want > to schedule an event that occurs every day at 11:30AM EST? There's > no date associated with it (except an infinite set, or the span of my life > I suppose!) it's just 11:30AM EST. That's a stretch, Donald. Can you present an example of such an event? For someone living on the US East Coast, some days that event would occur when their clock said 11:30AM, and on other days when it occurred at - figure it out! - 10:30AM or 12:30PM? Depending on whether, in the real world, daylight time wasn't or was in effect. In any case, you keep ignoring the context of what's written to present some caricature to shoot down. Here: [Tim] > No, it's bizarre to attach a timezone to a time object because most > tzinfo subclasses don't - and can't - know what to return for the UTC > offset in the absence of - at least - month and day info too. You didn't see the "most"? Or the later repeated elaborations that most tzinfo subclasses (in real life) _do_ attempt to model daylight time? An EST tzinfo subclass is a fixed-offset subclass, making no attempt to model daylight time. Those aren't the problem. But fixed-offset tzinfo subclasses are relatively rare. From tim.peters at gmail.com Fri Mar 7 02:51:48 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 19:51:48 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> Message-ID: <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com> [Donald Stufft] >> If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then >> absolutely that should be represented as a date time. But what if I want >> to schedule an event that occurs every day at 11:30AM EST? There's >> no date associated with it (except an infinite set, or the span of my life >> I suppose!) it's just 11:30AM EST. [Chris Angelico <rosuav at gmail.com>] > Repeating events need their own handling. Maybe it's safe to use > 11:30AM local time, but what if you specify 1:30AM America/New_York? > How can you render that onto a calendar? There'll be times when it > breaks - one day a year when it's ambiguous, and one day a year when > it doesn't exist. How can you work with a time that might not exist? Filling in the blanks, I presume that Donald's "EST" is what the docs call a "fixed-offset" tzinfo subclass: it represents a fixed offset from UTC. It's what a US East Coast clock would look like if the idiotic ;-) "daylight saving" rules were repealed, and the East Coast stayed on "standard" time forever. Such classes are utterly problem-free. No ambiguities, no clock leaping ahead or leaping back, no "missing" hours, no "duplicate" hours, they just march on forever in perfect lockstep with UTC. Alas, such classes are also rare. From abarnert at yahoo.com Fri Mar 7 02:52:38 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 17:52:38 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> Message-ID: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: Chris Angelico <rosuav at gmail.com> Sent: Thursday, March 6, 2014 4:29 PM > So... I'm +1 for adding a literal syntax for decimals. +1 for adding a > stating-the-default for floats (do it straight away, then code that > uses it can be backported all the way even when there's a change of > default). This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more. Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"? > +0.5 for adding a syntax for fractions; support in principle > but the devil's in the details. What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right? From breamoreboy at yahoo.co.uk Fri Mar 7 02:51:04 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 07 Mar 2014 01:51:04 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> Message-ID: <lfb8m3$v6m$2@ger.gmane.org> On 07/03/2014 01:25, Donald Stufft wrote: > > On Mar 6, 2014, at 8:19 PM, Tim Peters <tim.peters at gmail.com> wrote: > >> No, it's bizarre to attach a timezone to a time object because most >> tzinfo subclasses don't - and can't - know what to return for the UTC >> offset in the absence of - at least - month and day info too. > > It?s completely reasonable to attach a timezone to a time object. > > If I schedule an event in a calendar for Mar 7th, at 12pm Eastern, then > absolutely that should be represented as a date time. But what if I want > to schedule an event that occurs every day at 11:30AM EST? There?s > no date associated with it (except an infinite set, or the span of my life > I suppose!) it?s just 11:30AM EST. > > ----------------- > Donald Stufft Hum, thinking out loud as I don't know the answer. Assume that for your sins in a past life at 11:30AM EST every day you have to talk to me. How do I set my calendar up to have an infinite set of dates, I've never known an application let me do this? How do we sync our calendars so that your EST or (I guess) EDT ties in with my GMT or BST? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From guido at python.org Fri Mar 7 03:09:02 2014 From: guido at python.org (Guido van Rossum) Date: Thu, 6 Mar 2014 18:09:02 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> Message-ID: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> Mark, it feels like you do not understand Python well enough to be able to make sweeping proposals about its reform. Hopefully the responses you are getting will help you make more informed proposals in the future. On Thu, Mar 6, 2014 at 2:23 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > > > On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote: > {snip} > > hi Guido, thanks for your responses, your candor, and your kindness; I > think > dialogue will work, and I promise to keep the rhetoric to a bare minimum, > if at all. :) > > There really are only two questions I am going to try to answer here, and > some of > the answer goes back to Stefan's (and others) questions as well, so I hope > its helpful. > > > >> So, by context you really seem to be referring to some specific state of >> the program. Perhaps you are even talking about extending the existing >> notion of numeric context used by Python's Decimal type. But it's not very >> clear because "you change the context" is also a common term for changing >> the topic of a discussion. >> > > This is what I mean by "context," for this entire discussion, > mostly having to do with > Decimal and the "context manager" and these two lines: > > getcontext() > > with localcontext(ctx=None) as context_manager > > Allow me to back up a bit... the pdeclib module was not difficult > for me because of the > mathematics (no brag) I can code transcendental functions in my sleep. The > hard part of > that project last week was coming to grips with the context manger, and > the parameterization > of context attributes in the Decimal class. Back in the day when I did > this work for Rexx at IBM > I used NUMERIC DIGITS and NUMERIC FUZZ to emulate (in a crude way) what > Decimal is > doing correctly with its context, with the context manager. I credit Oscar > Benjamin for helping > me get up to speed with this concept (and I am so glad he did too, because > its key for this > entire discussion. > > Before we go on, allow me to explain my italics of *Number *and > *PythonNumber.* These are > each an abstract base class. They are a class that is never instantiated, > but from which all > other "numbers" are derived as objects. (more below) > > >> I'm sorry, but I am *still* unsure about what you mean by "context". >> Python does not need a context object to switch between complex and real >> number processing -- it's all done through operator overloading. >> > > (more on this below, but I am thinking beyond overloading/ > actually, virtual functions and templates) > > >> Actually, modern Python is considered object-oriented. >> > > Thank you. This has everything to do with this discussion, but > allow me to take a humorous > break and relate an anecdote... I have had numerous discussions with T.R. > and C.A and S.D. and > others where in the end python was object based. Ha! Nope, you have > seen it right here, the > BDfL says its "object oriented," and that's the end of the argument! > (ok, tiny rhetoric) > >> >> Unifying the >>> number system on a computer does not have to be grounded in paradigms >>> that serviced the >>> industry (and academy) for the past 40 years; >>> >> {snip} > >> >> Now you've got a specific proposal and we can discuss pros and cons. This >> is a topic that comes up occasionally and it is indeed a pretty interesting >> trade-off. I suspect the large (and fast-growing) SciPy community would >> prefer to keep binary floating point, because that's what their libraries >> written in Fortran or C++ want, and extra binary/decimal conversions at the >> boundaries would just slow things down. But I'm really speculating here -- >> maybe they don't need high-performance conversion between binary and >> decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe >> it varies per application. >> > > Here is the main response for this discussion, and I will endeavor > to keep the > rhetoric down, I promise. > This may even be easier to accomplish in 3.x without breaking > anyone, or any third party software either; depends > > In my view numbers are objects (like circles, squares and > triangles; in polymorphism discussions). > Numbers are nothing more than objects which have "contexts" just like > circles and squares and triangles > are nothing more than objects that have "contexts". In the following > discussion think in terms of object > oriented (as in Grady Booch) rather than any specific language--- and > certainly not python syntax. um, > think C++ for now, if you must think of any. A *Number *is an abstract > base class with "context" attributes > and pure virtual functions--- it will never be instantiated---and *PythonNumber > *will derive from that; also > an abstract base class. Here is the Class hierarchy I am proposing: > > *Number* > > *PythonNumber* > > Decimal > > Real <======= this is the default > > BinaryIEEE85 > > Real_b > > &c (...) omitted for clarity > > Complex > > Quaternion > > Rational > > Fraction > > &c-------------------------------------------------- > > Well, there it is. The idea is that any function or method that > takes a *Number* reference > or a *PythonNumber *reference does not have to interrogate what type of > *Number* it is... it just > works (why?) glad you asked, because of polymorphism and a three pointer > hop down a virtual > function table! > If a function or method gets a *Number *parm in as in > method(num) then to get the pretty > print representation of the number is simply num.pretty_print(), or > to get an evaluation > of the number is num.eval(), or to get the string rep of the number is > num.str()... and on and on. > The point is that other parts of the system no longer need to know what > "type" of number object > it is, the right code gets called because of virtual functions, operator > overloading, and the beautiful > virtual function table (all under the covers, not MAGIC, but polymorphism). > Python at present is disjointed when it comes to number as a > concept. In my view numbers > need to be objects, and the Class hierarchy (see my simplified version > above) needs to unify numbers > across the python boards (so to speak) so that "types" are only important > to the implementors, and > so that adding new "types" is easy, concise, and coherent. > > I realize that this suggestion is over the top in terms of size and > weight (Stefan's concern above). > But not really, when you think of conceptualizing the unification of > numbers under *Number *and > *PythonNumber *. There may be other kinds of numbers too, besides > these... like maybe *GmpyNumber* > of *BC_Number*, or others. > > The bottom line is that we have an object oriented language in > front of us that has the > software engineering advantages which would allow this kind of hierarchy > and conceptualization > of a unified number system. Let's leverage our technology for mutual > advantage across python > communities? > > Thanks for your consideration, > > Sincerely, > > Kind regards, > > marcus > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3be4381e/attachment-0001.html> From abarnert at yahoo.com Fri Mar 7 03:09:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 18:09:23 -0800 (PST) Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> Message-ID: <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Tim Peters <tim.peters at gmail.com> Sent: Thursday, March 6, 2014 5:19 PM > No, it's bizarre to attach a timezone to a time object because most > tzinfo subclasses don't - and can't - know what to return for the UTC > offset in the absence of - at least - month and day info too.? Hour, > minute, second and microsecond aren't usually enough to determine > whether "daylight time" is in effect, and most tzinfo subclasses do > attempt to model daylight time.? And because daylight time is a > political conceit, most tzinfo subclasses also need year info.? That > has nothing to do with whether times are viewed abstractly, > concretely, etc - it has to do with that time zones generally aren't > _defined_ in terms of isolated time-of-day info.? It's 7:13 PM in US > Central.? Is that daylight (CDT) or standard (CST) time?? It's > impossible to answer.? What's the corresponding UTC time?? Ditto. Well, a time in Central is useless, but a time in CDT or CST is not, and you can design a library that's smart enough to give you a CDT or CST (as appropriate) time from a Central datetime. However, I built something like that in a C++ datetime library, and we never found a use for it anywhere. Instead, we just carried the full datetime or recurring-datetime object around as late as possible, and rendered to naive times for display. From greg.ewing at canterbury.ac.nz Fri Mar 7 03:12:36 2014 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 07 Mar 2014 15:12:36 +1300 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> <53190D28.50205@canterbury.ac.nz> <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> Message-ID: <53192B14.4070304@canterbury.ac.nz> On 7/03/2014 1:13 p.m., Chris Angelico wrote: > What operations can you do on a time-of-day-with-timezone? Give me > some examples, and I'll show you, with the above two examples, how > quirky that can be. I'm not particularly attached to the idea of times of day with timezones; I wouldn't mind if they didn't exist. The main point is that they definitely don't make sense for time differences. An hour in New York is the same length as an hour in Melbourne, on any day of the year, as far as I know. (At least in modern times. It seems the Romans divided daytime and nighttime into 12 equal hours each, so that the length of an hour varied with latitude and time of year, and daytime hours were a different length from nighttime hours! I hate to think what a Python datetime module would have looked like back then.) -- Greg From python at mrabarnett.plus.com Fri Mar 7 03:12:59 2014 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 07 Mar 2014 02:12:59 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <53192B2B.1010205@mrabarnett.plus.com> On 2014-03-07 01:52, Andrew Barnert wrote: > From: Chris Angelico <rosuav at gmail.com> > > Sent: Thursday, March 6, 2014 4:29 PM > > >> So... I'm +1 for adding a literal syntax for decimals. +1 for >> adding a stating-the-default for floats (do it straight away, then >> code that uses it can be backported all the way even when there's a >> change of default). > > This makes sense if you also add an optional suffix for binary floats > at the same time. Otherwise, it would be confusing to talk about the > "default" when there's no way to make it explicit, and it would delay > any potential change of the default by at least one version, if not > more. > > Of course there's a lot of room for bikeshedding the suffix. The "f" > suffix from C implies 32-bit float as opposed to 64-bit double, which > is obviously wrong. The "d" suffix from C might be confusing as > distinguishing "binary, not decimal". Maybe "b"? > Calling the floating-point class "float" implies 32-bit float as opposed to 64-bit double, doesn't it? :-) >> +0.5 for adding a syntax for fractions; support in principle but >> the devil's in the details. > > What details, other than bikeshedding the exact suffix? If 3r == > fraction.Fraction(3), we're done, right? > From greg.ewing at canterbury.ac.nz Fri Mar 7 03:16:13 2014 From: greg.ewing at canterbury.ac.nz (Greg) Date: Fri, 07 Mar 2014 15:16:13 +1300 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com> References: <20140305120410.GY28804@ando> <20140306235921.GA20568@cskk.homeip.net> <CAPTjJmr=jGk2LT5nF72x8=jJmqHXjiV2As2BEhYHy0Wj1MiCpg@mail.gmail.com> Message-ID: <53192BED.6090909@canterbury.ac.nz> On 7/03/2014 1:17 p.m., Chris Angelico wrote: > On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote: >> Aside: Unless you do something obtuse, like make "3F" be a regular >> number with a special internal flag > > I thought exactly what you say here, that 3F would have to be a > magical type of integer. But actually, all it has to be is Fraction(3) > and everything will work perfectly. I think Cameron was talking about using 'F' for both 'binary floating point' *and* 'fraction', which would indeed lead to madness. -- Greg From tim.peters at gmail.com Fri Mar 7 03:16:53 2014 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 6 Mar 2014 20:16:53 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> [Tim] >> No, it's bizarre to attach a timezone to a time object because most >> tzinfo subclasses don't - and can't - know what to return for the UTC >> offset in the absence of - at least - month and day info too. Hour, >> minute, second and microsecond aren't usually enough to determine >> whether "daylight time" is in effect, and most tzinfo subclasses do >> attempt to model daylight time. And because daylight time is a >> political conceit, most tzinfo subclasses also need year info. That >> has nothing to do with whether times are viewed abstractly, >> concretely, etc - it has to do with that time zones generally aren't >> _defined_ in terms of isolated time-of-day info. It's 7:13 PM in US >> Central. Is that daylight (CDT) or standard (CST) time? It's >> impossible to answer. What's the corresponding UTC time? Ditto. [Andrew Barnert <abarnert at yahoo.com>] > Well, a time in Central is useless, but a time in CDT or CST is not, and you can > design a library that's smart enough to give you a CDT or CST (as appropriate) time > from a Central datetime. Tell me how. You have, in context, a Python `time` object with a "Central" tzinfo member. That's all you get from the user. You're given absolutely no information about day, month, or year. What's your algorithm for implementing picking CDT or CST "as appropriate"? Note that we're NOT talking about datetime.datetime objects here. These are datetime.time objects. This isn't a problem for datetime.datetime objects. A "Central" tzinfo timeclass has no problem at all picking CDT or CST as appropriate given all the info in a datetime.datetime object. From harrismh777 at gmail.com Fri Mar 7 03:08:28 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 18:08:28 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> Message-ID: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com> On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote: Once again, the problem occurs only because you're using a float > literal, and 2.01 can't perfectly round-trip. > > >>> Decimal(2.01) > Decimal('2.0099999999999997868371792719699442386627197265625') > > That's pretty much the value you had (I have a few more digits there, > the square rooting and squaring probably cost some accuracy), > hi Chris, yes, you are completely missing the point. We are talking past each other. I DO NOT need you to explain to me why the is working just like its supposed to... its broken and should not work this way ! If you write a number on paper do you write down '2.01' ? Do you write down d(2.01) ? Do you write down 2.01d ? (or) do you write down 2.01 ? When you punch a number into your TI89 do you punch in {'} {2} {.} {0} {1} {'} ? (or) do you punch in {2} {.} {0} {1} ? When I want a square-root from python I want to enter s1=sqrt(2.01) <====== this is normal (why?) glad you asked, because that is the human normal thing to do. Python numbers don't work correctly, because the underlying design is not correct. Everyone sees it, nobody really likes it, but everyone wants to avoid the problem like the proverbial ostrich (only worse , because this ostrich wants me to believe that everything is fine, " Don't pay any attention to that man behind the current... I... am the great and powerful OZ" marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/61c380ba/attachment-0001.html> From harrismh777 at gmail.com Fri Mar 7 03:19:56 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 18:19:56 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> Message-ID: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote: Mark, it feels like you do not understand Python well enough to be able to > make sweeping proposals about its reform. > hi Guido, ouch. You can't be serious. You have not commented yet... are you happy with this: >>> from decimal import Decimal >>> a=Decimal(1) >>> b=Decimal(.1) >>> a+b Decimal('1.100000000000000005551115123') <==== does this not bother you at all ? ... even though we both know why its doing this, doesn't it bother you a little? marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/047747b6/attachment.html> From rymg19 at gmail.com Fri Mar 7 03:21:13 2014 From: rymg19 at gmail.com (Ryan) Date: Thu, 06 Mar 2014 20:21:13 -0600 Subject: [Python-ideas] Add "goto fail" to Python? In-Reply-To: <20140307011229.GM28804@ando> References: <lf8rh3$suv$1@ger.gmane.org> <CADiSq7dhh6XAQ7DV04uMBJAOT3nSwTE+a_X=xxYZdgXL=s5Szg@mail.gmail.com> <lfapo9$t5i$1@ger.gmane.org> <CANUJvPUdQM1CRTumz=C-BOkZ1U7S-YN5Vk9qwRrM1=1X0QVGYQ@mail.gmail.com> <003601cf39a1$21aa23b0$64fe6b10$@gmail.com> <20140307011229.GM28804@ando> Message-ID: <ab60b651-e71c-49bc-a4af-d190c04aaed5@email.android.com> Wow, those links are hilarious. Steven D'Aprano <steve at pearwood.info> wrote: >Guys, > >Sturla's original email was quite subtle, but you may have missed that >he is proposing this as a joke PEP for April Fools Day, a bit like the >introduction of GOTO or the retirement of Guido: > >https://mail.python.org/pipermail/python-announce-list/2004-April/002982.html > >http://legacy.python.org/dev/peps/pep-0401/ > >As a joke, it pokes fun at Apple's recent "goto fail" security breach. >Unfortunately the Python C source code apparently uses exactly the same > >"goto fail" idiom, although hopefully not as poorly as Apple did. > > > >-- >Steven >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/10cf5d6b/attachment.html> From abarnert at yahoo.com Fri Mar 7 03:19:40 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 18:19:40 -0800 (PST) Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com> Message-ID: <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Tim Peters <tim.peters at gmail.com> Sent: Thursday, March 6, 2014 5:51 PM > Filling in the blanks, I presume that Donald's "EST" is what the > docs > call a "fixed-offset" tzinfo subclass:? it represents a fixed offset > from UTC.? It's what a US East Coast clock would look like if the > idiotic ;-) "daylight saving" rules were repealed, and the East Coast > stayed on "standard" time forever. Except that it still wouldn't work for dates before 2014. Or, of course, if DST were reinstated in the future. If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm? From abarnert at yahoo.com Fri Mar 7 03:26:29 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 18:26:29 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <53192B2B.1010205@mrabarnett.plus.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> <53192B2B.1010205@mrabarnett.plus.com> Message-ID: <1394159189.4605.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: MRAB <python at mrabarnett.plus.com> Sent: Thursday, March 6, 2014 6:12 PM > On 2014-03-07 01:52, Andrew Barnert wrote: >> Of course there's a lot of room for bikeshedding the suffix. The >> "f" >> suffix from C implies 32-bit float as opposed to 64-bit double, which >> is obviously wrong. The "d" suffix from C might be confusing as >> distinguishing "binary, not decimal". Maybe "b"? >> > Calling the floating-point class "float" implies 32-bit float as > opposed to 64-bit double, doesn't it? :-) Sure. It's not as strong an implication as an "f" suffix, because many languages (and applications) have types named float, while "0.3f" is a lot more C-family-specific. Which means "float" wasn't a perfect choice?but it was still probably the best choice. And it's certainly possible that "f" is the best choice for the suffix despite this problem.?If there were an obvious best answer that had no problems, there wouldn't be any room for bikeshedding. From abarnert at yahoo.com Fri Mar 7 03:35:31 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 18:35:31 -0800 (PST) Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> Message-ID: <1394159731.26118.YahooMailNeo@web181003.mail.ne1.yahoo.com> From: Tim Peters <tim.peters at gmail.com> Sent: Thursday, March 6, 2014 6:16 PM >> Well, a time in Central is useless, but a time in CDT or CST is not, and > you can >> design a library that's smart enough to give you a CDT or CST (as > appropriate) time >> from a Central datetime. > > Tell me how.? You have, in context, a Python `time` object with a > "Central" tzinfo member.? That's all you get from the user.? > You're > given absolutely no information about day, month, or year.? What's > your algorithm for implementing picking CDT or CST "as appropriate"? > Note that we're NOT talking about datetime.datetime objects here. > These are datetime.time objects. Of course that's inherently ambiguous and therefore not solvable. If your app wants unambiguous times, it can't allow users to enter times in Central with no additional information. But that's a separate problem from getting a CDT or CST time from a Central datetime (or from other problems, like handling a Central time that's known to be today). A datetime library could solve _that_ problem. > This isn't a problem for datetime.datetime objects.? A "Central" > tzinfo timeclass has no problem at all picking CDT or CST as > appropriate given all the info in a datetime.datetime object. Right, but if you get its tztime, you get a Central time, and no longer have the information that lets you use it as CDT or CST as appropriate. That is a problem?but, as I said before, it's probably not a problem in practice, and therefore probably not worth solving. (Again, I did solve it in a different timezone library, and never found a use for that feature.) From harrismh777 at gmail.com Fri Mar 7 03:51:18 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 18:51:18 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> Message-ID: <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote: Mark, it feels like you do not understand Python well enough to be able to > make sweeping proposals about its reform. > hi Guido, I'm sorry, but isn't that the whole point? The person who notices that something isn't working quite right (that would be me) comes to court, as it were, before those who are supposed to know intimately how everything works (that would be you) and in good faith asks politely for the "people in the know" to correct the difficulty. You know what, you're right--- I'm a dumb cluck who doesn't know anything about the intricacies of python yadda yadda yadda... but you know what else ? > I hate this: >>> from decimal import Decimal >>> a=Decimal(1) >>> b=Decimal(.1) >>> a+b Decimal('1.100000000000000005551115123') >>> ... and I think the people who DO know intimately how python works, should do something to fix it. I'd like to help, if you'd let me. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/5e698c43/attachment.html> From cs at zip.com.au Fri Mar 7 03:52:24 2014 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 7 Mar 2014 13:52:24 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <53191390.7050804@mrabarnett.plus.com> References: <53191390.7050804@mrabarnett.plus.com> Message-ID: <20140307025224.GA17812@cskk.homeip.net> On 07Mar2014 00:32, MRAB <python at mrabarnett.plus.com> wrote: > On 2014-03-06 23:59, Cameron Simpson wrote: > >Coming to this very late, but I was interested in the first thread... > > > >On 05Mar2014 23:04, Steven D'Aprano <steve at pearwood.info> wrote: > >>On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote: > >>> and I would also use Fraction > >>> literals if available e.g. 1/3F. > >> > >>Out of curiosity, why F rather than f? > > > >Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats > >in play in Python right now. [...] > >So of course "F" is reasonable for fractions. But how hard does > >that make the grammar? Because the parser now has to grab the entire > >"1/3F" to construct the fraction. You can't just stick it in the lexer > >at that point. > > > I think it would be confusing if there were "f" for "float" and "F" for > "fraction". How about "r" for "rationals"? I'm good with 'r'. -- Cameron Simpson <cs at zip.com.au> If you do not read the paper, you are uninformed. If you do read the paper, you are misinformed. - Mark Twain From abarnert at yahoo.com Fri Mar 7 03:53:35 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 18:53:35 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> Message-ID: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Mark H. Harris <harrismh777 at gmail.com> Sent: Thursday, March 6, 2014 6:19 PM [snipping out of order] >?>>> from decimal import Decimal >>>> a=Decimal(1) >>>> b=Decimal(.1) >>>> a+b >Decimal('1.100000000000000005551115123') ? ? ?<==== does this not bother you at all ? > >? ? ... even though we both know why its doing this, doesn't it bother you a little? That's not a problem with Python's number system, it's that Decimal(.1) is not the right way to write what you want. The only solution without changing Python is to train end-users to write something correct, like Decimal('.1'). The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. For example, a decimal suffix, as already proposed before this thread, would completely solve the problem: ? ? >>> a = 1d ? ? >>> b = .1d ? ? >>> a+b ? ? 1.1d Of course the exact suffix (or other syntax) is up for bikeshedding, as is the possibility of one day changing the default from binary floats to decimal floats, but other than those trivial details, tada.?But there's no need for anything more radical, like some amorphous idea to "unify Python numbers". >On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote: > >>Mark, it feels like you do not understand Python well enough to be able to make sweeping proposals about its reform.? > >hi Guido, ?ouch. You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python. You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem. You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading. (And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to.?It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.) There are some ideas in your posts that are worth responding to, but I think it's perfectly fair for Guido to decide it's not worth digging through the mass of ignorance about Python's basic design to find the nuggets that can be rejected for more specific reasons instead of just dismissed. From ben+python at benfinney.id.au Fri Mar 7 04:12:32 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 07 Mar 2014 14:12:32 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> Message-ID: <85k3c6bt27.fsf@benfinney.id.au> "Mark H. Harris" <harrismh777 at gmail.com> writes: > On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote: > > > Mark, it feels like you do not understand Python well enough to be > > able to make sweeping proposals about its reform. > > hi Guido, I'm sorry, but isn't that the whole point? The person who > notices that something > isn't working quite right (that would be me) comes to court, as it were, > before those who > are supposed to know intimately how everything works (that would be you) > and in good > faith asks politely for the "people in the know" to correct the difficulty. No, that's not the point of this forum. The point of this forum is to present ideas for improvement to Python and to *defend* those ideas robustly against critique, in order to converge on concrete proposals. To do that, you need to have a pretty good understanding of Python as it currently is, especially in the domain of the change you're proposing, in order to robustly defend an idea for improvement. > I hate this: [?] If you want a forum for general discussion where complaints lacking a good understanding of Python can be bounced around, this isn't it. You can try the general discussion forum for Python <URL:https://mail.python.org/mailman/listinfo/python-list>. But beware that even there, complaints about how Python behaves aren't going to be well received if you show a wilful ignorance of that behaviour. Mere ignorance is fine, of course: we all start out ignorant of any given topic. But wilful ignorance isn't a good foundation for expecting change. It's better to take the complaint as motivation to *learn* why Python behaves this way, to see if your assumptions may not be flawed. > ... and I think the people who DO know intimately how python works, > should do something to fix it. You'll need to form a much more concrete understanding of why it is the way it is, and exactly what behavioural change you want, and an understanding of the costs and benefits of that change, before such an expectation would in any way imply an obligation on others to do what you want. This forum isn't the place to begin with that. -- \ ?You say ?Carmina?, and I say ?Burana?, You say ?Fortuna?, and | `\ I say ?cantata?, Carmina, Burana, Fortuna, cantata, Let's Carl | _o__) the whole thing Orff.? ?anonymous | Ben Finney From cs at zip.com.au Fri Mar 7 03:58:21 2014 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 7 Mar 2014 13:58:21 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <53192BED.6090909@canterbury.ac.nz> References: <53192BED.6090909@canterbury.ac.nz> Message-ID: <20140307025821.GA19433@cskk.homeip.net> On 07Mar2014 15:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > On 7/03/2014 1:17 p.m., Chris Angelico wrote: > >On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson <cs at zip.com.au> wrote: > >>Aside: Unless you do something obtuse, like make "3F" be a regular > >>number with a special internal flag > > > >I thought exactly what you say here, that 3F would have to be a > >magical type of integer. But actually, all it has to be is Fraction(3) > >and everything will work perfectly. > > I think Cameron was talking about using 'F' for both 'binary > floating point' *and* 'fraction', which would indeed lead to > madness. Well, only because I hadn't thought through to having it be a literal Fraction, which it turns out already has exactly this cool semantic. -- Cameron Simpson <cs at zip.com.au> The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. - George Bernard Shaw From cs at zip.com.au Fri Mar 7 04:29:08 2014 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 7 Mar 2014 14:29:08 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com> References: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com> Message-ID: <20140307032908.GA46124@cskk.homeip.net> On 06Mar2014 18:08, Mark H. Harris <harrismh777 at gmail.com> wrote: > On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote: > Once again, the problem occurs only because you're using a float > > literal, and 2.01 can't perfectly round-trip. > > > > >>> Decimal(2.01) > > Decimal('2.0099999999999997868371792719699442386627197265625') > > > > That's pretty much the value you had (I have a few more digits there, > > the square rooting and squaring probably cost some accuracy), > > hi Chris, yes, you are completely missing the point. We are talking past > each other. I DO NOT need you to explain to me why the is working just > like its supposed to... its broken and should not work this way ! > > If you write a number on paper do you write down '2.01' ? > > Do you write down d(2.01) ? > > Do you write down 2.01d ? > > (or) do you write down 2.01 ? [...] > When I want a square-root from python I want to enter s1=sqrt(2.01) > <====== this is normal So essentially you are saying: when I write Python code I do not want to have to put in all this special notation to get the actual value which I naturally mean. And if you write 2.01, you mean 2 + 1/100, without roundoff because you do not want to be working in a system which would round that off. i.e. a Decimal internal representation because you're writing a number in base 10. To my mind you want a Python _parser_ mode, because inside python, given the various numeric classes abounding, the actual _mechanisms_ already exist. So really you want to have a way of saying to the parser: in the code, 2.01 is not transformed into an IEEE float with some loss of precision, it is transformed into a Decimal floating point number, with no loss of precision. There are, it seems to me, two basic ways to approach this. The easy one is give Python something like: from __the_far_far_future__ import Decimal_numbers and have decimal literals converted into those instead of Python "floats". The alternative is the implement a Python class, let us call it "AbtractNumber", which stores the literal text you put in the programme code "2.01" and converts it, _when used_, to a suitable python type with the right semantics. Either would get you code where: x = 2.01 is stored without loss of precision, and may get you the behaviour you're after. Which of these more closely matches your desires? Please try to be precise. Examples don't quite do it without some rather precise accompanying lagunage which says what specific things the example is meant to illustrate. If one of my two suggestions above is very close to what you're after, please present an example where one suggestion does what you want and where the other would not, with an explaination of why. Just hoping to get at things more precisely and clearly, -- Cameron Simpson <cs at zip.com.au> Careful and correct use of language is a powerful aid to straight thinking, for putting into words precisely what we mean necessitates getting our own minds quite clear on what we mean. - W.I.B. Beveridge From stephen at xemacs.org Fri Mar 7 04:30:55 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Mar 2014 12:30:55 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5318EA28.3040305@stoneleaf.us> References: <201403051216.12392.shai@platonix.com> <CACac1F9YB8p-m+Q9wnYGiGVkhvTf0jCCYkAtWg+MFrhyKU0f4g@mail.gmail.com> <16619289-2CDC-4C0A-B2FC-9B1EB58A29CD@masklinn.net> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <CANUJvPWND9Zx-NEA-J-6bqPomsJzVteQr1MyyncQm9W9eu5VEw@mail.gmail.com> <CACac1F-+KeouYM-1sy7iDCSP7zBAvVVzb+sg30U__V42j9eh1A@mail.gmail.com> <5318EA28.3040305@stoneleaf.us> Message-ID: <87eh2eptw0.fsf@uwakimon.sk.tsukuba.ac.jp> Ethan Furman writes: > On 03/05/2014 07:42 AM, Paul Moore wrote: > > On 5 March 2014 15:08, Yann Kaiser <kaiser.yann at gmail.com> wrote: > >> Let me show you an actual zero: > >> > >> if event.num_attendants: > >> prepare_cake() > >> else: > >> cancel_event() > > I'm sorry, are you really trying to say that the above code is better than > > > > if event.num_attendants != 0: > > prepare_cake() > > else: > > cancel_event() > > > > ? > Yes. This is Python. blah blah blah != 0 is unnecessary boiler-plate. I wouldn't call that *better*, just a different style. I agree I prefer "if blah.count:" to "if blah.count != 0:", but I might very well write "if blah.count > 0:". I don't see anything "wrong" with using an explicit comparison, but I don't have a problem reading the abbreviated style in this situation.[1] Personally I would *always* use the "implied boolean" style for emptiness tests (I'm an old Lisper), but I might very well use explicit comparisons against zero. For example, I would be likely to write the party example with a reversed test (to keep 'if' and 'else' close together): if event.num_attendants == 0: # nobody's coming cancel_event() else: # several-pages-long suite of preparations prepare_cake() and I think that looks better than if not event.num_attendants: # it's a count, not a boolean # etc But actually I'd probably design the event class differently, and write it as: if not event.list_of_attendants: # attendance list is empty # etc or perhaps if event.list_of_attendants: # attendance list is non-empty # reverse the suites (depending on the phase of the moon and whether the trout are running in Esopus Creek to choose between the last two :-). Footnotes: [1] I do have a real issue with the "if x:" to test for a sentinel meaning "uninitialized" that happens to evaluate to false. There's objectively a probability that it would pass an error that would be caught if the "is None" form were used, but that probability is admittedly low. Tastes may differ regarding "importance" here. From stephen at xemacs.org Fri Mar 7 04:37:02 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Mar 2014 12:37:02 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53190936.6030608@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz> Message-ID: <87d2hyptlt.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > It seems to me that those claiming "midnight is special" > are inventing a justification after the fact -- one > that is not supported by the actual behaviour. > > Altogether, the whole thing seems to be a mess. Mirroring the real world, no? From steve at pearwood.info Fri Mar 7 04:45:47 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 14:45:47 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> Message-ID: <20140307034547.GO28804@ando> On Thu, Mar 06, 2014 at 04:17:55PM -0800, Mark H. Harris wrote: > I just had one more thought along this line; consider this: > > >>> from pdeclib import * > >>> s1=sqrt(2.01) > >>> s1 > Decimal('1.41774468787578244511883132198668766452744') > >>> s2=sqrt(d(2.01)) > >>> s2 > Decimal('1.41774468787578252029556185427085779261123') > >>> > >>> s1**2 > Decimal('2.00999999999999978683717927196994423866272') > >>> s2**2 > Decimal('2.01000000000000000000000000000000000000000') If you skip the conversion to Decimal, you actually get the right answer using floats: py> (2.01**0.5)**2 2.01 So the problem here isn't the binary float, but that Decimal by default has *too much precision* and consequently it ends up keeping digits that the user doesn't care about: py> from decimal import Decimal as D py> (D.from_float(2.01)**D("0.5"))**2 Decimal('2.009999999999999786837179272') Floats have about 14 significant base-10 figures of precision (more in base-2), so if we tell Decimal to use the same, we should get the same result: py> import decimal py> ct = decimal.getcontext() py> ct.prec = 14 py> (D.from_float(2.01)**D("0.5"))**2 Decimal('2.0100000000000') Decimal is not a panacea. Both Decimal and binary floats have the same limitations, they just occur in different places for different numbers. All floating point numbers have these same issues. Fixed point numbers have different issues, rationals have their own issues, and symbolic computations have a different set of issues. Computer maths is a leaky abstraction. No matter what you do, how you implement it, the abstraction leaks. Not even Mathematica can entirely hide the fact that it is computing rather than performing a Platonic ideal of mathematics. -- Steven From harrismh777 at gmail.com Fri Mar 7 04:53:15 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 19:53:15 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote: > > > The only solution without changing Python is to train end-users to write > something correct, like Decimal('.1'). > hi Andrew, yes, that is the problem. And, to be fair, that example is not really the worst because it might be expected that the user should know how to construct Decimals, and could be educated to construct them properly--> Decimal('0.1'); I concur. It is worse in these two scenarios (and others too): sqrt(.1) sin(.1) No one would expect that the user should know to quote the number---when is that ever done? QED this is broken. Again, we know perfectly well why its happening (I am not ignorant) but its not right. > > The obvious solution for changing Python is to make it easier to create > Decimal numbers correctly and/or harder to create them incorrectly. For > example, a decimal suffix, as already proposed before this thread, would > completely solve the problem: > > >>> a = 1d > >>> b = .1d > >>> a+b > 1.1d > Yes, and at a bare minimum, that is the immediate change I have been asking for, for now; nothing more. I answered Guido's questions in hopes that he might be willing to dialogue --- not dismiss. > You proposed that Python should handle numbers in an OO way, with numbers > being real objects, instances of classes, with a hierarchy including > abstract base classes; all of this is already there in Python. > Yes, it is... but its not designed to use decimal floating point by default... in order to do that the entire OO setup will have to be changed (what Guido called a sweeping reform). Its like if we want to use floating point decimals in the 21st century, using python, we have to duck tape modules on and then educate users in the correct input of numbers. Seems convoluted to me. > > You went off on a long digression about how you could implement this using > the details of C++-style inheritance, when Python has a completely (and > more powerful) different solution to inheritance that has already been used > to solve this problem. > No, I did not. I answered Guido's questions regarding context as clearly as I could. If python has a more powerful way to handle this situation, gladly do it! I will be happy as a clam to beta test or help with the coding. > > You proposed some complicated AI-based solution to solve the problem of > using separate number classes in a single expression, even though Python > (almost exactly like C++, in this case) has already solved that problem > with operator overloading. > No, I did not. I suggested that unifying numbers in an (AI) way could solve this problem (conceptually) by regarding all numbers as *PythonNumbers. *Decimals should not only be default, they should be integrated, not tacked on with duck tape. > > (And note that Python is flexible enough that third-party libraries can > easily insert new types like quaternions, matrices, symbolic expressions, > etc. into the hierarchy in a way that's transparent to end users. I can > multiply a NumPy matrix of float64 values by the builtin in 2 just by > writing "m * 2", and it works exactly the way you'd want it to. It's hard > to imagine that would be even feasible with an AI-based solution, but with > the current design, that's the easiest part of NumPy.) > That's nice for you. Because sqrt(.23709) does not behave as I expect, sadly, I have to train my users to enter sqrt('0.23709'). > > There are some ideas in your posts that are worth responding to, > Thank you. If a user goes to the time and trouble to present an idea clearly, I would expect the responders to respect the effort and respond to the points that make sense. Andrew, I respect you for taking the time to dialogue, I appreciate it. Thanks. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3c21e039/attachment-0001.html> From ethan at stoneleaf.us Fri Mar 7 04:50:35 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Mar 2014 19:50:35 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <20140306030002.GH28804@ando> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <20140306030002.GH28804@ando> Message-ID: <5319420B.3000004@stoneleaf.us> On 03/05/2014 07:00 PM, Steven D'Aprano wrote: > On Wed, Mar 05, 2014 at 09:23:37PM -0500, Donald Stufft wrote: > >> It?s hard to do any sort of search for this, however in an informal poll where I?ve shown >> people this code not a single person thought it made sense, and most of them responded >> with ?wtf??. > > Well I don't know who these people are, what their background is, or > exactly how you phrased the question. But in my experience, most > programmers have extremely strongly held opinions about the sensibility > of certain features that have little or nothing with any rational > analysis of the pros and cons and far more likely to be "that's > different from the first language I learned, therefore it's rubbish". As part of learning Python, I learned one of its core tenets was the divide between something and nothing (some of that from you, I believe ;), midnight is most certainly something, so I was very surprised to learn that bool(midnight) is False. -- ~Ethan~ From stephen at xemacs.org Fri Mar 7 04:58:45 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Mar 2014 12:58:45 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53190A16.4020201@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> Message-ID: <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Barry Warsaw wrote: > > On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote: > > > >>Instances of existing code that would be broken by the change: 0 > > > > You can't know this because you can't audit all the Python code > > in the world. > > I think he means instances that have been pointed out > so far in this discussion. Well, either way it's a pretty careless claim. Alexander and Tim both have stated that they have used "if time:" to test for a zero time (per the documentation) in actual applications. I gather from the fact that Tim is "-0" on the change that his code was never widely distributed, but I don't know about Alexander's. Probably not, as he didn't post a URL, either. AFAIK the code doesn't have to be distributed for gratuitous code breakage to be considered a Very Bad Thing, though. One of Alexander's use cases (changing date in a simulation at midnight using naive times) is particularly salient here. Unlike any of the other cases described so far by either side, it's clear that his simulation would be completely broken by the proposed incompatible change in behavior, as the date would never change. (Granted, it's not clear to me whether he actually implemented this in production, but I also can't say from what I've read that he didn't.) From rosuav at gmail.com Fri Mar 7 05:16:27 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 15:16:27 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <624af90f-50cd-4ac8-9401-f3023b2735f3@googlegroups.com> <CAPTjJmo8DqgLTm_S1b1yr_zZLANbCUt07vGiLquVuSTXgd+LZA@mail.gmail.com> <1394157158.94258.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <CAPTjJmpj-DNtzxbk8Ae-zQomRd65Jfe5CxXeUUpnvGm+RCcrVQ@mail.gmail.com> On Fri, Mar 7, 2014 at 12:52 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > From: Chris Angelico <rosuav at gmail.com> > > Sent: Thursday, March 6, 2014 4:29 PM > > >> So... I'm +1 for adding a literal syntax for decimals. +1 for adding a >> stating-the-default for floats (do it straight away, then code that >> uses it can be backported all the way even when there's a change of >> default). > > This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more. > > Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"? > Yep. That's what I mean (when I said "float"s up there, I meant the current type 'float', aka binary floating point). The suffix won't have any effect; it'll be like u"string" in Py3, explicitly stating the default, and added for the exact same reason. >> +0.5 for adding a syntax for fractions; support in principle >> but the devil's in the details. > > What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right? > Precisely that detail. Is it 3r? 3F? Something else? Would it look tidier as a prefix instead of a suffix? But if that can be resolved, it'd be good to have a syntax for fractions. I'm only +0.5 on that, as rationals aren't as big an advantage over the status quo as decimal is. It's a lot less common to see: >>> Fraction(1/3) Fraction(6004799503160661, 18014398509481984) than the oddities of decimal.Decimal construction that we're seeing here. Having a numeric suffix for decimal literals will be much more beneficial to the language, imo; if the bikeshedding of Fraction literals is problematic, I'd not be against doing decimal (and binary float) tags one version, and leaving Fraction for another version. ('Course, it might all work out perfectly, in which case great! Add 'em all at once.) ChrisA From harrismh777 at gmail.com Fri Mar 7 05:17:33 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 20:17:33 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <85k3c6bt27.fsf@benfinney.id.au> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> <85k3c6bt27.fsf@benfinney.id.au> Message-ID: <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com> On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote: > But beware that even there, complaints about how Python behaves aren't > going to be well received if you show a wilful ignorance of that > behaviour. Mere ignorance is fine, of course: we all start out ignorant > of any given topic. But wilful ignorance isn't a good foundation for > expecting change. > Ben, excuse me, but you are out of line here. I have not shown will full ignorance in any of this discussion (None). I am not ignorant of the system, nor am I ignorant of the underlying caveats. I have studied here, and I have a pretty good handle on how things are working. When I have been confused I asked questions, read, experimented, and researched. I will not have it here said that I have displayed will full ignorance, nor should anyone believe it. I have a good understanding of why it is... I just don't like why it is... those are two very different things. I am fully aware of the issues created by IEEE 754 1985 floats and doubles. This is 2014. I would expect that python would be using IEEE 754 2008 not only for specification, but also for moving towards decimal floating point arithmetic by default. Continuing to stick with a 1985 standard, or even an IEEE 854 1987 standard, is putting all python communities behind the eight ball going forward into the 21st Century. This is not will full ignorance. Its eyes-wide-open thoroughly knowledgable concern with a genuine "will" to see the right thing done... because its the right thing to do. marcus > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/d172b262/attachment.html> From ben+python at benfinney.id.au Fri Mar 7 05:27:39 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 07 Mar 2014 15:27:39 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> <85k3c6bt27.fsf@benfinney.id.au> <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com> Message-ID: <85d2hybpl0.fsf@benfinney.id.au> "Mark H. Harris" <harrismh777 at gmail.com> writes: > On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote: > > > But beware that even there, complaints about how Python behaves aren't > > going to be well received if you show a wilful ignorance of that > > behaviour. Mere ignorance is fine, of course: we all start out ignorant > > of any given topic. But wilful ignorance isn't a good foundation for > > expecting change. > Ben, excuse me, but you are out of line here. I have not shown > will full ignorance in any of this discussion (None). I didn't mean to imply that, though I can see how the above can be read that way. My apologies for giving that impression. Nevertheless, the warning stands for those who wish to bring a complaint in this forum or the ?python-list? forum. -- \ ?Probably the toughest time in anyone's life is when you have | `\ to murder a loved one because they're the devil.? ?Emo Philips | _o__) | Ben Finney From rosuav at gmail.com Fri Mar 7 05:29:15 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 15:29:15 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> <85k3c6bt27.fsf@benfinney.id.au> <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com> Message-ID: <CAPTjJmp+UEOho5-0Ku22orMvw9bKmnsjdccQ2==xS3u5-h9jjQ@mail.gmail.com> On Fri, Mar 7, 2014 at 3:17 PM, Mark H. Harris <harrismh777 at gmail.com> wrote: > On Thursday, March 6, 2014 9:12:32 PM UTC-6, Ben Finney wrote: > >> >> But beware that even there, complaints about how Python behaves aren't >> going to be well received if you show a wilful ignorance of that >> behaviour. Mere ignorance is fine, of course: we all start out ignorant >> of any given topic. But wilful ignorance isn't a good foundation for >> expecting change. > > Ben, excuse me, but you are out of line here. I have not shown > will full ignorance in any of this discussion (None). Hey hey, I'd be careful of calling someone out like that... might want to be a little humble and respectful here :) The main problem here is that you and the Python interpreter have different expectations about a string of digits in the source code. You expect them to be represented exactly; the language specifies that they be represented with the float type. Changing how the language interprets something as fundamental as "number with decimal point in it", even if unassailably correct, is a backward-incompatible change. (The full unification of the int and long types wasn't effected until Python 3, even though that change is unlikely to break much code.) If someone wants to push for it, write the PEP, go through all the details, then it'd be possible to have this for Python 3.5: >>> 0.1d == Decimal("0.1") True But you're not going to have that effect for untagged numbers without a long deprecation period; probably we're talking Python 4000, which may not ever even exist. It'd be possible to have a __future__ directive that switches around the two types, so decimal is the default and binary floats have to be tagged, but that could cause so much confusion that it'd itself be the subject of massive controversy. (Although there would be precedent for it. "from __future__ import unicode_literals" didn't destroy Py2 code readability.) ChrisA From cs at zip.com.au Fri Mar 7 05:32:33 2014 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 7 Mar 2014 15:32:33 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> References: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> Message-ID: <20140307043233.GA14220@cskk.homeip.net> On 06Mar2014 19:53, Mark H. Harris <harrismh777 at gmail.com> wrote: > On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote: [... lots of stuff, snipped along with Mark's replies ...] > > You proposed some complicated AI-based solution to solve the problem of > > using separate number classes in a single expression, even though Python > > (almost exactly like C++, in this case) has already solved that problem > > with operator overloading. > > No, I did not. I suggested that unifying numbers in an (AI) way > could solve this problem (conceptually) by > regarding all numbers as *PythonNumbers. *Decimals should not only be > default, they should be integrated, not > tacked on with duck tape. This sounds to me like keeping the original numeric text around and only turning it into some efficient-for-a-machine representation when it comes time to work on it. One issue with that is that when the work occurs, it can be far from where the number was defined, and the correct internal representation might be poorly chosen. > > (And note that Python is flexible enough that third-party libraries can > > easily insert new types like quaternions, matrices, symbolic expressions, > > etc. into the hierarchy in a way that's transparent to end users. I can > > multiply a NumPy matrix of float64 values by the builtin in 2 just by > > writing "m * 2", and it works exactly the way you'd want it to. It's hard > > to imagine that would be even feasible with an AI-based solution, but with > > the current design, that's the easiest part of NumPy.) > > That's nice for you. Because sqrt(.23709) does not behave as I > expect, sadly, I have to train my users to enter sqrt('0.23709'). That's partly because .23709, in current python, becomes an IEEE float with some loss of precision because binary fractions and base-10 fractions do not round trip. But also partly because sqrt() (in the pure mathematical sense) often produces transcendental numbers, which are not representation by any fixed precision intergal base notation - effectively IEEE floats and Decimal floats are fractions/rationals. So sqrt() will, for almost all numbers, involve loss of precision no matter what base your backing storage is: base 2 as in IEEE float or base 10 as in a Decimal. Steven (or Stephen) has already pointed this out to you; perhaps it has been missed. And going back (eg sqrt(2.01)*sqrt(2.01) ==> not-quite-2.01) just extends this loss of precision. This is an inherent problem unless sqrt() doesn't convert to a "concrete" type, and instead just lurks as some symbolic representation, so that any expression involving it becomes a progressively more elaboration representation of an algebraic expression. Which may be correct, but only until you try to evaluate it to get a concrete number again. > > There are some ideas in your posts that are worth responding to, > > Thank you. If a user goes to the time and trouble to present an > idea clearly, I would expect the responders > to respect the effort and respond to the points that make sense. Many people have. You should see the short shrift some ideas receive. I think part of the problem is not your wish for a "number" but that lack of a concrete proposal to implement it. Cheers, -- Cameron Simpson <cs at zip.com.au> The Usenet is not the real world. The Usenet usually does not even resemble the real world. - spaf at cs.purdue.edu From rosuav at gmail.com Fri Mar 7 05:38:02 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 15:38:02 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com> <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com> On Fri, Mar 7, 2014 at 1:19 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm? > Fortunately, you don't have to be dictator of the whole world. As admin of Minstrel Hall (minstrelhall.com), I simply made the declaration that all campaign times would be listed in UTC. That effectively bans DST and timezones. The in-game clock uses 24-hour time, so a lot of people follow that. Most things happen on a weekly cycle, so I haven't done anything about MDY (the in-game clock currently reads "Fri 04:36:05"), but again, I would personally use YMD most likely, and a lot of people will just follow that. Let's start creating pockets like that all over the place. Train people to manage international events in UTC. Eventually the benefits will start to be more visible :) ChrisA From rosuav at gmail.com Fri Mar 7 05:43:26 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 15:43:26 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <53192B14.4070304@canterbury.ac.nz> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAPTjJmpaNEDpyzHo=U0oOLEmy8O1yL=Rw60Pi_Yv5Y_myJLWtQ@mail.gmail.com> <53190D28.50205@canterbury.ac.nz> <CAPTjJmoCz80Lhc56Ftrnd3bEmXPvig14mXkGpr5UbCrLK5YDEg@mail.gmail.com> <53192B14.4070304@canterbury.ac.nz> Message-ID: <CAPTjJmoFyF8qbbX7VWRWFmBRFk0EzACNtgNZOCOkhKbwKuY=1g@mail.gmail.com> On Fri, Mar 7, 2014 at 1:12 PM, Greg <greg.ewing at canterbury.ac.nz> wrote: > On 7/03/2014 1:13 p.m., Chris Angelico wrote: >> >> What operations can you do on a time-of-day-with-timezone? Give me >> some examples, and I'll show you, with the above two examples, how >> quirky that can be. > > > I'm not particularly attached to the idea of times of day with > timezones; I wouldn't mind if they didn't exist. > > The main point is that they definitely don't make sense for > time differences. An hour in New York is the same length as > an hour in Melbourne, on any day of the year, as far as I know. Sure. So a timedelta with timezone makes no sense. But we're not talking about timedelta here, we're talking about time. (And we're also ostriching leap seconds away. Don't wanna know! Don't wanna know!) A calendar that supports repeating events is going to need some kind of rule system anyway. I don't think a "time with timezone" is sufficient for that. I haven't used that kind of system (ever!), but from Lennart Regebro's PyCon 2013 talk "Blame it on Caesar", I gather that there is such a thing in Python. So... from this thread, it seems like there's only one real use for time-with-timezone: fracturing a datetime-with-timezone into two parts, date and time, and recombining losslessly. Anyone have other uses? ChrisA From harrismh777 at gmail.com Fri Mar 7 05:51:16 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 20:51:16 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <85d2hybpl0.fsf@benfinney.id.au> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> <85k3c6bt27.fsf@benfinney.id.au> <95630dc1-695a-4ab0-89be-8ed1c116768d@googlegroups.com> <85d2hybpl0.fsf@benfinney.id.au> Message-ID: <c6e1f8ff-a95d-4d2c-a9ff-49b5c4ee55e3@googlegroups.com> On Thursday, March 6, 2014 10:27:39 PM UTC-6, Ben Finney wrote: > > > Ben, excuse me, but you are out of line here. I have not shown > > will full ignorance in any of this discussion (None). > > I didn't mean to imply that, though I can see how the above can be read > that way. My apologies for giving that impression. > > Nevertheless, the warning stands for those who wish to bring a complaint > in this forum or the ?python-list? forum. > hi Ben, forgive me, I misunderstood you. No problem, nor hard feelings. Chris is absolutely correct when he asks for humble respect. I absolutely respect you guys, and I'm here to learn as much as anything. Thank you for your insights. Kind regards, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/a67aaf91/attachment-0001.html> From ethan at stoneleaf.us Fri Mar 7 06:02:54 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Mar 2014 21:02:54 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> <CAP7h-xZLHgeP_VqjXbJE64at7BiDwzuvWCpY1U0jj=fd=sm9Rw@mail.gmail.com> Message-ID: <531952FE.5050205@stoneleaf.us> On 03/06/2014 08:13 AM, Alexander Belopolsky wrote: > > On Thu, Mar 6, 2014 at 8:21 AM, M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>> wrote: > > > Are they all False? No, no they're not (unless your local timezone is UTC): > > > >>>> bool(utcmidnight) > > False > >>>> bool(naivemidnight) > > False > >>>> bool(localmidnight) > > True > > Now this is a what I consider a valid argument for making a change. > > > FWIW, I would be +0 for making a change so that t.tzinfo is not None > implies bool(t.timetz()) is True. > > My intuition goes like this. Given > >>>> from datetime import * >>>> t1 = datetime(2014, 1, 1) >>>> t2 = datetime(2014, 1, 1, 12) >>>> t3 = datetime(2014, 1, 1, tzinfo=timezone.utc) > > Instance t1 has no time information, so t1.time() is false-y. If datetime actually worked like that I would agree with you; but it doesn't -- it says you have the time of midnight on 2014, 1, 1, not that you don't have a time at all. -- ~Ethan~ From steve at pearwood.info Fri Mar 7 06:25:21 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 16:25:21 +1100 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com> References: <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <F19C8FE3-750E-4E22-BBF8-59AD0877826F@stufft.io> <CAPTjJmrY82dU8f102PtQ7xn93wD5xW+jiK_dDS+yGvyD0DBNEg@mail.gmail.com> <CAExdVN=H8xeXz2bmjaHYPnWBL62DcZA6xL8z_yh5FPDhpKtzjw@mail.gmail.com> <1394158780.26537.YahooMailNeo@web181002.mail.ne1.yahoo.com> <CAPTjJmr9h1=g8Ugiu2_qkxax8EBEsy0C_Sv8ROq-gV35_t1BpA@mail.gmail.com> Message-ID: <20140307052521.GP28804@ando> On Fri, Mar 07, 2014 at 03:38:02PM +1100, Chris Angelico wrote: > On Fri, Mar 7, 2014 at 1:19 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > > If I were dictator of the world, I would ban DST. And 12-hour times. And MDY date formats. In fact, I would probably ban timezones. How hard would it be to learn that on the east coast you work from 0400-1200, not 9:00am-5:00pm? > > > > Fortunately, you don't have to be dictator of the whole world. As > admin of Minstrel Hall (minstrelhall.com), I simply made the > declaration that all campaign times would be listed in UTC. That > effectively bans DST and timezones. No it doesn't. People still have to convert from the listed timezone to their own local timezone. The only difference is that instead of privileging some semi-arbitrary timezone on the basis of where your servers are, you're privileging an even more arbitrary timezone in Europe. Depending on the distribution of your users, chances are good that you're actually inconveniencing more people by your decision than if you used the local timezone. [...] > Let's start creating pockets like that all over the place. Train > people to manage international events in UTC. Eventually the benefits > will start to be more visible :) There are no benefits, not even to programmers, since they will still have to convert a local time to UTC. The only differences will be: - people whose local time happens to be UTC won't have to do anything; - everyone else will suffer more, as even local (to them) events will need to be converted from UTC. Really, all these (hopefully tongue-in-cheek) suggestions to ban timezones are as useful as those (unfortunately serious) suggestions to avoid all the issues with Unicode by going back to "good ol' ASCII" and insist that everybody write in American English. (Actually a subset of American English, since ASCII is missing some common American symbols.) -- Steven From abarnert at yahoo.com Fri Mar 7 06:30:47 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 6 Mar 2014 21:30:47 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> Message-ID: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> On Mar 6, 2014, at 19:53, "Mark H. Harris" <harrismh777 at gmail.com> wrote: > On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote: >> >> >> The only solution without changing Python is to train end-users to write something correct, like Decimal('.1'). > > hi Andrew, yes, that is the problem. And, to be fair, that example is not really the worst > because it might be expected that the user should know how to construct Decimals, and > could be educated to construct them properly--> Decimal('0.1'); I concur. > It is worse in these two scenarios (and others too): > sqrt(.1) > sin(.1) No, that's not the same problem but worse, it's a completely different problem. In the first case, the user is trying to specify 0.1 as a decimal number, which is exactly representable, just not the way he's entered it. In these cases, the numbers are irrational, and therefore inherently impossible to represent exactly. It doesn't matter whether you use decimal floats or binary floats. > No one would expect that the user should know to quote the number---when is that ever done? What does quoting have to do with anything? Do you not understand why Decimal('.1') works? It's not because Python is a weakly-typed language that allows you to use strings as numbers, but because Decimal has a constructor that takes strings, for a specific and well-documented reason. Quoting here would just give you a TypeError. And nothing in your proposal(s) would change that. > QED this is broken. Again, we know perfectly well why its happening (I am not ignorant) but its not right. The only way to fix this second problem is to use a symbolic representation instead of a numeric one. The good news is that Python's design makes that pretty easy to add on--as SymPy demonstrates. Your proposal would actually make this kind of add-on much harder. >> The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. > > Yes, and at a bare minimum, that is the immediate change I have been asking for, for now; nothing more. And people have agreed with that, and proposed feasible extensions to it. Presenting it as the first step toward some radical and ill-formed transformation of the whole language weakens the case for this suggestion. Implying that it would solve problems (like handling irrational numbers) that it obviously can't also weakens the case. > I answered Guido's questions in hopes that he might be willing to dialogue --- not dismiss. He was willing to dialogue, as evidenced by his initial replies. It was only after you demonstrated your ignorance of basics fundamentals of Python (as a user, not even about its implementation) and math/numerics, and implied that you had no interest in correcting that ignorance, that he dismissed you. And he has every right to do so. He's the one donating his free time to make a great language for you (and many others) to use, not the other way around. >> You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python. > > Yes, it is... but its not designed to use decimal floating point by default... in order to do that the entire OO > setup will have to be changed (what Guido called a sweeping reform). Nonsense. Python already has classes for both binary and decimal floats. They both fit into the hierarchy properly. They both interact with other types the way they should. Changing which one you get from the literal "0.1" would be a simple change to the parser, and have no effect whatsoever to the OO setup. >> You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem. > > No, I did not. I answered Guido's questions regarding context as clearly as I could. If python has a more powerful > way to handle this situation, gladly do it! I will be happy as a clam to beta test or help with the coding. It's already been done, years ago, so nobody has to do it. There's already an OO system that works, with abstract and concrete classes, and with a rich operator overloading system. And it's already been used to give you all of the abstract and concrete numeric types you want, and they do the things you asked for in this section, all without having to do any jumps through virtual pointer tables. >> You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading. > > No, I did not. I suggested that unifying numbers in an (AI) way could solve this problem (conceptually) by > regarding all numbers as PythonNumbers. So you didn't suggest a complicated AI-based solution, you suggested a complicated AI-based solution? > Decimals should not only be default, they should be integrated, not > tacked on with duck tape. How does that have anything to do with the first half of this paragraph? And in what way are Decimals "tacked on with duck tape"? They're instances of Number, and of Real. They act like numbers of other types, including interacting properly with other types like int. What is missing from the Decimal type and the ABCs in Number that makes you think we need a radical change? If all you're suggesting is moving Decimal from the decimal module to builtins and/or adding parser support for decimal literals, those are not sweeping changes, they're both very simple changes. (That doesn't necessarily mean they're _desirable_ changes, but that's another argument--an argument nobody can actually begin until you make it clear whether or not that's what you're suggesting, which can't happen until you learn enough about using Python to know what you're suggesting.) >> (And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to. It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.) > > That's nice for you. Because sqrt(.23709) does not behave as I expect, sadly, I have to train my users to enter sqrt('0.23709'). How do you expect it to behave? Again, using decimal floats here would make no difference. Instead of needing an infinite number of binary digits, you need an infinite number of decimal digits. Dividing infinite by log2(10) still leaves infinity. If you're trying to fix that, you're not trying to fix Python, you're trying to fix irrational numbers. This list cannot help you with that. Prayer is the only option, but medieval mathematicians tried that and didn't get very far, and eventually we had to accept that there are numbers that cannot be represented finitely. >> There are some ideas in your posts that are worth responding to, > > Thank you. If a user goes to the time and trouble to present an idea clearly, I would expect the responders > to respect the effort and respond to the points that make sense. > > > Andrew, I respect you for taking the time to dialogue, I appreciate it. Thanks. > > marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/2c86a9b0/attachment-0001.html> From ethan at stoneleaf.us Fri Mar 7 06:11:55 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Mar 2014 21:11:55 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <lfaigr$8t4$1@ger.gmane.org> <53190936.6030608@canterbury.ac.nz> <CAExdVNnwQAmm3appUX58oD8RhWpp74-pM9HSLwZPmP7Wk-B16g@mail.gmail.com> Message-ID: <5319551B.6000000@stoneleaf.us> On 03/06/2014 04:08 PM, Tim Peters wrote: > [Greg Ewing] >> ... >> Altogether, the whole thing seems to be a mess. > > Although a mess that fits in a thimble that successfully hid in a dark > corner unnoticed for a decade ;-) If the decade is now, it didn't go that long. I discovered this irritation within a year of learning Python, and at least three years ago brought it up on one of the mailing lists. I ended up making my own Date, Time, and DateTime classes. -- ~Ethan~ From harrismh777 at gmail.com Fri Mar 7 05:05:02 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 20:05:02 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307034547.GO28804@ando> References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <20140307034547.GO28804@ando> Message-ID: <5a9524af-1d00-4663-9bca-1d04bbbc4fea@googlegroups.com> On Thursday, March 6, 2014 9:45:47 PM UTC-6, Steven D'Aprano wrote: > Decimal is not a panacea. Both Decimal and binary floats have the same > limitations, they just occur in different places for different numbers. > All floating point numbers have these same issues. Fixed point numbers > have different issues, rationals have their own issues, and symbolic > computations have a different set of issues. > hi Steven, yes, I concur. My primary objection (for immediate concern) is that we have now a very fast module for doing decimal floating point math with less of the issues you speak of. Doing decimal floating point math by default, or at a minimum, providing a way to enter decimal numbers 1.234d would do a lot to alleviate much of the perceived difficulty with float issues. > > Computer maths is a leaky abstraction. No matter what you do, how you > implement it, the abstraction leaks. Not even Mathematica can entirely > hide the fact that it is computing rather than performing a Platonic > ideal of mathematics. > Again, I concur. Leaky is one thing... but being mired in 40-year-old paradigms because IEEE 754 1985 floats |doubles are so entrenched just doesn't make sense to me, and should be corrected. I am not insisting on Guido's "sweeping" reform. I'm just trying to make like a little easier for number munching and get folks to be forward thinking about moving to a decimal based floating point system of number for python by default---sometime. Thank you for your response, Steven. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/59f344cc/attachment.html> From ethan at stoneleaf.us Fri Mar 7 05:47:26 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 06 Mar 2014 20:47:26 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> <53187670.6070302@egenix.com> <87mwh3pfbt.fsf@uwakimon.sk.tsukuba.ac.jp> <lfa1as$8bm$1@ger.gmane.org> <87lhwnp6n9.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <53194F5E.8070103@stoneleaf.us> On 03/06/2014 09:40 AM, Stephen J. Turnbull wrote: > Antoine Pitrou writes: > > Le 06/03/2014 15:33, Stephen J. Turnbull a ?crit : > > > > never get fixed, and that "if var:" which is actually testing for > > > identity with a false-y sentinel is still strongly discouraged. > > > > Why "strongly discouraged"? This is more of a style issue than anything > > else? > > No, it's not just a style issue. As Skip points out, it's a > *semantic* issue as well. Indeed, I have been bitten by using the lazy method of `if var` instead of `if var is not None`, and then been briefly puzzled as why my empty string | container | whatever that I passed in did not give me the correct results. I'm still all for having truthy midnights. :) -- ~Ethan~ From steve at pearwood.info Fri Mar 7 07:11:21 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 17:11:21 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> Message-ID: <20140307061121.GQ28804@ando> On Thu, Mar 06, 2014 at 06:19:56PM -0800, Mark H. Harris wrote: > >>> from decimal import Decimal > >>> a=Decimal(1) > >>> b=Decimal(.1) > >>> a+b > Decimal('1.100000000000000005551115123') <==== does this not bother > you at all ? Of course it bothers people a little. It bothers me. It also bothers me when I'm too hot wearing a coat and too cold when I take it off, but sometimes that's how the universe works. For the first few releases of Decimal, it prohibited direct conversion of floats specifically to avoid that issue: # Python 2.5 py> decimal.Decimal(2.01) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.5/decimal.py", line 648, in __new__ "First convert the float to a string") TypeError: Cannot convert float to Decimal. First convert the float to a string But that turned out to be more of a nuisance than what it was trying to protect from, so starting in Python 2.7 Decimal now supports direct and exact conversion from float. The problem is, you are focused on one application for numeric computing to the exclusion of all else, specifically using Python as an interactive calculator. But in practice, for many uses, nobody typed in 2.01 and nobody has any expectation that it will be exactly the value 2.01. Rather, the value will be the result of some calculation, and there is *absolutely no reason to think* that the decimal number 2.01 will be more accurate than the binary number 10.101000111101011100001010001111010111000010100, or for that matter, the base-7 number 2.0462046204620462. The difference between those three representations is usually minor compared to the actual measurement errors of the initial data and the rounding errors from the calculation. -- Steven From harrismh777 at gmail.com Fri Mar 7 07:24:47 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 22:24:47 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> Message-ID: <b63af00a-ebea-4f2e-89bb-59f74b98b506@googlegroups.com> On Thursday, March 6, 2014 11:30:47 PM UTC-6, Andrew Barnert wrote: > > I answered Guido's questions in hopes that he might be willing to > dialogue --- not dismiss. > > > He was willing to dialogue, as evidenced by his initial replies. It was > only after you demonstrated your ignorance of basics fundamentals of Python > (as a user, not even about its implementation) and math/numerics, and > implied that you had no interest in correcting that ignorance, that he > dismissed you. And he has every right to do so. He's the one donating his > free time to make a great language for you (and many others) to use, not > the other way around. > Andrew, I demonstrated no such thing. This is the second time you have accused me of being ignorant of basics and fundamentals. My answers in no way (implied or otherwise) ignorant, they were well defined, carefully articulated, and implied an ernest interest in understanding and cooperation. Your *ad hominem* comments about my so-called ignorance do not become you and are beginning to annoy me... I don't see any reason for it. I know how the system works... and I think it can be improved. My ideas are unpopular, I realize that, but that does not mean that I am ignorant, nor does it mean that I don't understand python fundamentals. I had a professor one time who told us, "You know when you have won the argument when they attack you personally (or call you names)". Why don't we keep the discussion on the details relevant to the topic and leave the personal attacks alone? marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/75b55e08/attachment-0001.html> From harrismh777 at gmail.com Fri Mar 7 07:39:35 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 22:39:35 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307061121.GQ28804@ando> References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <20140307061121.GQ28804@ando> Message-ID: <31615f5e-5429-4027-915d-8ceea251465e@googlegroups.com> On Friday, March 7, 2014 12:11:21 AM UTC-6, Steven D'Aprano wrote: > For the first few releases of Decimal, it prohibited direct conversion > of floats specifically to avoid that issue: > > # Python 2.5 > py> decimal.Decimal(2.01) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/local/lib/python2.5/decimal.py", line 648, in __new__ > "First convert the float to a string") > TypeError: Cannot convert float to Decimal. First convert the float to a > string > > > But that turned out to be more of a nuisance than what it was > trying to protect from, so starting in Python 2.7 Decimal now > supports direct and exact conversion from float. hey Steven, whoa...! that is useful, and very interesting news. That almost makes me want to cry.... Wait a minute, I'm going to check it out, ... son of a gun, you're right. My Py2.6.1 does not allow me to enter this sqrt( .789 ). What-da-ya-know........ Here is the output: >>> from pdeclib import * >>> sqrt(.789) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "pdeclib.py", line 243, in sqrt sqr=Decimal(x).sqrt() File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/decimal.py", line 652, in __new__ "First convert the float to a string") TypeError: Cannot convert float to Decimal. First convert the float to a string >>> So, "ignorant boy" finds out that the community has come to court with unclean hands ! Everyone (even the core devs) know that this little problem sucks, and they've tried to fix it in different ways, and yet when I bring it up its like (hypocrisy) I'm the one with the problem--- when in fact all along everyone knows that this little IED was going to go off and make a mess. ~nice. <sigh> Steven, I honor you and thank you for being honest and coming clean on this. Geeze... I'm going to bed. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/3e251deb/attachment.html> From harrismh777 at gmail.com Fri Mar 7 07:01:51 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Thu, 6 Mar 2014 22:01:51 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> Message-ID: <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> On Thursday, March 6, 2014 11:30:47 PM UTC-6, Andrew Barnert wrote: > > No one would expect that the user should know to quote the > number---when is that ever done? > > > What does quoting have to do with anything? Do you not understand why > Decimal('.1') works? It's not because Python is a weakly-typed language > that allows you to use strings as numbers, but because Decimal has a > constructor that takes strings, for a specific and well-documented reason. > Quoting here would just give you a TypeError. And nothing in your > proposal(s) would change that. > > I understand precisely what it happening. It is just the opposite of what you are implying (that I'm ignorant) which you actually state later. I full well know that the quoted string is forcing the right constructor to be called for the Decimal object. I am an experienced OO programmer and have many years under my belt with C++, Smalltalk, and Java. Please don't be pedantic with me, nor imply that I am ignorant. It does not become you, and it irritates me... besides, it gets in the way of discussion. My point is not about loose typing, nor about Decimal constructors, ... my point is that an average user entering a number into a sqrt() function is not going to realize that they need to enter a quoted string in order to force the string constructor on the Decimal Class !! They are going to want to enter this --> sqrt( .789 ) not this sqrt( '0.789' ) I know how it works. I'm asking for a reasonable response to the need for normal usability. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140306/9e3a2147/attachment.html> From stefan at bytereef.org Fri Mar 7 09:11:44 2014 From: stefan at bytereef.org (Stefan Krah) Date: Fri, 7 Mar 2014 09:11:44 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> References: <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> Message-ID: <20140307081144.GA6899@sleipnir.bytereef.org> Mark H. Harris <harrismh777 at gmail.com> wrote: > You can't be serious. You have not commented yet... are you happy with > this: > > >>> from decimal import Decimal > >>> a=Decimal(1) > >>> b=Decimal(.1) > >>> a+b > Decimal('1.100000000000000005551115123') <==== does this not bother you at > all ? You can set a trap: >>> getcontext().traps[FloatOperation] = True >>> b = Decimal(.1) Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.FloatOperation: [<class 'decimal.FloatOperation'>] >>> Stefan Krah From mertz at gnosis.cx Fri Mar 7 10:17:26 2014 From: mertz at gnosis.cx (David Mertz) Date: Fri, 7 Mar 2014 01:17:26 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> Message-ID: <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com> On Thu, Mar 6, 2014 at 6:51 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > I hate this: > >>> from decimal import Decimal > >>> a=Decimal(1) > >>> b=Decimal(.1) > >>> a+b > Decimal('1.100000000000000005551115123') > >>> > ... and I think the people who DO know intimately how python works, should > do something to fix it. > The problem here--and the problem with the "default number type is decimal" is that what you want fixed is "unfixable." As a number of other people have noted, what you are demanding is that numbers have finite sized and exact representations, which they simply don't and can't in computers with finite memory. Your example, Mark, is merely one where you have misspelled your Python code: >>> from decimal import Decimal as D >>> D('1') + D('.1') Decimal('1.1') Fix your spelling error, and in this case you get what you want. However, what about this: >>> D(1/3) * 3 Decimal('0.9999999999999999444888487687') There's no way to fix that spelling. Yes, you might be able to play with a decimal.Context in a way that gets this one example to happen to round the way you want it to, but no context will do so generically for every similar expected identity under reciprocal operators. By coincidence, this works under binary FB *because* of the limited precision of floats: >>> 1/3 * 3 1.0 But that's just luck, basically, it's not because we've managed to precisely represent 1/3 as a binary float. We don't have to look all that far for a problem case: >>> 1/49 * 49 0.9999999999999999 Of course, we do have Fractions, which will always do the right thing under the reciprocal division and multiplication operators: >>> F('1/3') * 3 Fraction(1, 1) Don't misspell this either, of course: >>> F(1/3) * 3 Fraction(18014398509481983, 18014398509481984) However, even if Fraction were to become the default numeric type for Python, and preserve the reciprocity of div/mul, that wouldn't help us any under the hoped reciprocals of sqrt() and square. Similarly, of course, for trigonometric identities and various other numeric functions that produce irrational answers. E.g.: >>> from fractions import Fraction as F >>> sqrt(F('1/3'))**2 0.3333333333333333 >>> F(sqrt(F('1/3'))**2) Fraction(6004799503160661, 18014398509481984) Well, that answer is a binary floating point; we leave Fraction land pretty easily in Python. The float we get, of course, is something that's not exactly 1/3 (but is pretty close). Even if we imagined some future version of Python that did sqrt() purely as a Fraction without converting to IEEE 754, we STILL couldn't ever have an exact identity. There simply does not exist any Fraction that can accurately represent sqrt(F('1/3')); there are only rational numbers that are "pretty close." Maybe this one, for example: >>> F(sqrt(F('1/3'))) Fraction(1300077228592327, 2251799813685248) Maybe this hypothetical Python 4000 that didn't do the conversion to floating point would produce a different approximation, but it would always be an approximation, not the Real (irrational) answer. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/f5546c95/attachment-0001.html> From saghul at gmail.com Fri Mar 7 10:37:58 2014 From: saghul at gmail.com (=?ISO-8859-1?Q?Sa=FAl_Ibarra_Corretg=E9?=) Date: Fri, 07 Mar 2014 10:37:58 +0100 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode Message-ID: <53199376.40007@gmail.com> Hi, (I hope this list is the right place for this) I was browsing around for accept4 usage in Python and I saw it's already used internally in socket.accept (on Linux only), albeit it's not exposed, so accepting an incoming connection and setting it to be non-blocking takes 3 syscalls. accept4 allows us to do this with a single syscall, and it made it into FreeBSD 10, so that's another system that could benefit from this optimization. Would it be desirable to extend socket.accept to something like: socket.accept(set_non_blocking=False) which would use accept4's flags on supported systems and fallback to regular fcntl on others? The idea is to accept an incoming connection and make it non-blocking at the same time, asyncio and other frameworks would benefit from this by doing one function call and 2 syscalls less (on supported systems, that is). Cheers, -- Sa?l Ibarra Corretg? bettercallsaghul.com From steve at pearwood.info Fri Mar 7 11:12:35 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 21:12:35 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> Message-ID: <20140307101235.GR28804@ando> On Thu, Mar 06, 2014 at 07:53:15PM -0800, Mark H. Harris wrote: [...] > because it might be expected that the user should know how to construct > Decimals, and > could be educated to construct them properly--> Decimal('0.1'); I concur. > It is worse in these two scenarios (and others too): > sqrt(.1) > sin(.1) Decimal won't save you in those two cases. For square roots, the result is usually a surd, an irrational number. The only way to store surds exactly is to store it symbolically. Should we try to turn Python into Mathematica? In the case of sines, the same applies. While it is possible to calculate exact rational values for some angles, but for most the calculation rapidly becomes untenable, and in general trigonometric values are irrational, like surds. > No one would expect that the user should know to quote the > number---when is that ever done? OF COURSE WE DO. This is a programming language, not a calculator. We expect users -- programmers -- to know how to write functions, define classes, import modules, use regular expressions, call external processes, and so on. In other words, we expect them to know how to program. > QED this is broken. No it is not. Another factor you are not taking into account is that there is a reason that the computer industry has standardised on base-2 floats rather than 10. 30 years ago, it was quite common to find applications using some variant of Decimal, perhaps using BCD (binary coded decimal) as their native numeric format. Going back even further, computers were built using decimal-based hardware: http://en.wikipedia.org/wiki/Decimal_computer But people moved to binary for multiple reasons: - it's cheaper; - it's faster; - it's less surprising. For example, between 1 and 100000, about 12% of integer-valued floats fail the "1/(1/n) == n" test, but over 51% of the integer-valued Decimals: py> from decimal import Decimal as D py> sum(1/(1/n) != n for n in range(1, 100001)) 11846 py> sum(1/(1/D(n)) != n for n in range(1, 100001)) 51665 Likewise we can test how many fail the "(1/n)*n == 1" test: py> sum((1/n)*n != 1 for n in range(1, 100001)) 13117 py> sum((1/D(n))*n != 1 for n in range(1, 100001)) 36806 13% for floats versus 36% for Decimals. One more to prove it isn't a fluke: the "sqrt(n)**2 == n" test: py> sum((n**0.5)**2 != n for n in range(1, 100001)) 49544 py> sum((n**D("0.5"))**2 != n for n in range(1, 100001)) 71303 That's three for three in favour of binary floats. [...] > Yes, and at a bare minimum, that is the immediate change I have been > asking for, for now; nothing more. You're not going to get an immediate change, no matter how solid the case you ask for, no matter how logical the argument. Python 3.4 is not receiving any new features. The earliest this could come out is 3.5, in about 18 months or so. > I answered Guido's questions in hopes that he might be willing to > dialogue --- not dismiss. You've had at least three emails from Guido. That's three more than most radical or wild ideas receive. The fact that so many people are continuing to discuss this with you is proof that we recognise that computer maths is hard and unintuitive. But just because you have identified a problem doesn't mean that your answer is a solution. [...] > > You proposed some complicated AI-based solution to solve the problem of > > using separate number classes in a single expression, even though Python > > (almost exactly like C++, in this case) has already solved that problem > > with operator overloading. > > No, I did not. I suggested that unifying numbers in an (AI) way > could solve this problem (conceptually) by > regarding all numbers as *PythonNumbers. *Decimals should not only be > default, they should be integrated, not > tacked on with duck tape. It's statements like this that strongly suggest that you don't really understand how numbers work in computer programming. Or the practical limitations of what is *possible*. And as soon as you start talking about using an AI to decide what value the user intended, you've entered crackpot territory. Speaking of AIs, perhaps you ought to see what Wolfram Alpha is capable of: https://www.wolframalpha.com/input/?i=%28square+root+of+2.01%29+to+the+power+of+2 Not surprisingly for something with Mathematica behind it, it gets the exact right answer. Do you understand why Python the language cannot duplicate Mathematica's abilities with symbolic maths? Hint: it's not because it isn't technically possible. See, for example, libraries like sympy. -- Steven From steve at pearwood.info Fri Mar 7 11:16:22 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 21:16:22 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307061121.GQ28804@ando> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <20140307061121.GQ28804@ando> Message-ID: <20140307101622.GS28804@ando> On Fri, Mar 07, 2014 at 05:11:21PM +1100, Steven D'Aprano wrote: [...] > there is *absolutely no reason to think* that the decimal number 2.01 > will be more accurate than the binary number > 10.101000111101011100001010001111010111000010100, or for that matter, > the base-7 number 2.0462046204620462. The difference between those three > representations is usually minor compared to the actual measurement > errors of the initial data and the rounding errors from the calculation. Er, apparently I cannot convert fractions into base 7 even using Python. That should be 2.003300330033 in base 7. The number I gave was actually 2.1. -- Steven From steve at pearwood.info Fri Mar 7 11:39:56 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 21:39:56 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> Message-ID: <20140307103956.GT28804@ando> On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote: > Hmm. Is the rounding done by float.__str__() an attractive nuisance? It's not *rounding* precisely. Starting in Python 3.1, the float __repr__ will display the shortest decimal string that doesn't change the float's value. Reading the issue tracker history for this change is informative: http://bugs.python.org/issue1580 To the guys who worked on that, I take my hat off to you all. -- Steven From solipsis at pitrou.net Fri Mar 7 11:40:39 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 7 Mar 2014 11:40:39 +0100 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode References: <53199376.40007@gmail.com> Message-ID: <20140307114039.720efff9@fsol> On Fri, 07 Mar 2014 10:37:58 +0100 Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote: > > accept4 allows us to do this with a single syscall, and it made it into > FreeBSD 10, so that's another system that could benefit from this > optimization. If it doesn't, then perhaps the configure script needs to be fixed. > The idea is to accept an incoming connection and make it non-blocking at > the same time, asyncio and other frameworks would benefit from this by > doing one function call and 2 syscalls less (on supported systems, that is). That's not likely to do a significant difference (benchmarks welcome). Regards Antoine. From rob.cliffe at btinternet.com Fri Mar 7 11:51:12 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 07 Mar 2014 10:51:12 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <5319A4A0.10905@btinternet.com> On 07/03/2014 03:58, Stephen J. Turnbull wrote: > Greg Ewing writes: > > Barry Warsaw wrote: > > > On Mar 06, 2014, at 11:56 AM, Rob Cliffe wrote: > > > > > >>Instances of existing code that would be broken by the change: 0 > > > > > > You can't know this because you can't audit all the Python code > > > in the world. > > > > I think he means instances that have been pointed out > > so far in this discussion. > > Well, either way it's a pretty careless claim. > > Please read what I actually said: Let's _start_ a survey. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/bfe69403/attachment-0001.html> From steve at pearwood.info Fri Mar 7 11:57:32 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Mar 2014 21:57:32 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> <862f6205-6b49-483a-a941-345f30bf60fb@googlegroups.com> Message-ID: <20140307105732.GU28804@ando> On Thu, Mar 06, 2014 at 06:08:28PM -0800, Mark H. Harris wrote: > If you write a number on paper do you write down '2.01' ? > Do you write down d(2.01) ? > Do you write down 2.01d ? Sometimes I write down 2.01 subscript 10. That's just a different way of spelling 2.01d. (That's the trouble with rhetorical questions. Sometimes people will give an answer other than what you were expecting.) > (or) do you write down 2.01 ? > > When you punch a number into your TI89 do you punch in {'} {2} {.} {0} > {1} {'} ? > (or) do you punch in {2} {.} {0} {1} ? For what it's worth, the TI-89 stores numbers using a decimal floating point format. -- Steven From rosuav at gmail.com Fri Mar 7 12:05:09 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 22:05:09 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307103956.GT28804@ando> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> <20140307103956.GT28804@ando> Message-ID: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> On Fri, Mar 7, 2014 at 9:39 PM, Steven D'Aprano <steve at pearwood.info> wrote: > On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote: > >> Hmm. Is the rounding done by float.__str__() an attractive nuisance? > > It's not *rounding* precisely. Starting in Python 3.1, the float > __repr__ will display the shortest decimal string that doesn't change > the float's value. > > Reading the issue tracker history for this change is informative: > > http://bugs.python.org/issue1580 > > To the guys who worked on that, I take my hat off to you all. That effect, yes. Let's call it "the magic of float.__str__", because it really is pretty amazing. But it's still post-processing magic. It means that strings appear to round-trip through floats, as long as you're a long way within the available precision; but as soon as you do operations, that ceases to be the case. I think it's great for display, but is putting that into __repr__ (at least, they do appear to be the same) an attractive nuisance, in that it encourages people to treat float("...") as a true representation? Don't get me wrong, it's a really *awesome* feature. It just happens to have been partially responsible for this thread, which I think is approaching 300 posts. ChrisA From rosuav at gmail.com Fri Mar 7 12:06:03 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 22:06:03 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> <20140307103956.GT28804@ando> <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> Message-ID: <CAPTjJmqhDea4Qj=aUDb1Po6i2GQYsr-Sby6MmQ2kc3EiNA7H8Q@mail.gmail.com> On Fri, Mar 7, 2014 at 10:05 PM, Chris Angelico <rosuav at gmail.com> wrote: > Don't get me wrong, it's a really *awesome* feature. It just happens > to have been partially responsible for this thread, which I think is > approaching 300 posts. Whoops, no. This one's only approaching 100. It's the "should midnight be true or false" that's approaching 300. Sorry! ChrisA From rosuav at gmail.com Fri Mar 7 12:08:23 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Mar 2014 22:08:23 +1100 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode In-Reply-To: <53199376.40007@gmail.com> References: <53199376.40007@gmail.com> Message-ID: <CAPTjJmrcQQPL46G8urY1B-nweoJz0XkbhiUpXcOq5tHA2LdVhA@mail.gmail.com> On Fri, Mar 7, 2014 at 8:37 PM, Sa?l Ibarra Corretg? <saghul at gmail.com> wrote: > The idea is to accept an incoming connection and make it non-blocking at the > same time, asyncio and other frameworks would benefit from this by doing one > function call and 2 syscalls less (on supported systems, that is). Is the benefit that it's simply more efficient, or is there a specific benefit from atomicity? The latter is a pretty strong argument (look at close-on-exec for instance), if it applies. ChrisA From saghul at gmail.com Fri Mar 7 12:14:04 2014 From: saghul at gmail.com (=?UTF-8?B?U2HDumwgSWJhcnJhIENvcnJldGfDqQ==?=) Date: Fri, 07 Mar 2014 12:14:04 +0100 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode In-Reply-To: <20140307114039.720efff9@fsol> References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol> Message-ID: <5319A9FC.1010609@gmail.com> On 03/07/2014 11:40 AM, Antoine Pitrou wrote: > On Fri, 07 Mar 2014 10:37:58 +0100 > Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> > wrote: >> >> accept4 allows us to do this with a single syscall, and it made it into >> FreeBSD 10, so that's another system that could benefit from this >> optimization. > > If it doesn't, then perhaps the configure script needs to be fixed. > Probably not, I'll have a look. >> The idea is to accept an incoming connection and make it non-blocking at >> the same time, asyncio and other frameworks would benefit from this by >> doing one function call and 2 syscalls less (on supported systems, that is). > > That's not likely to do a significant difference (benchmarks welcome). > Actually, after http://hg.python.org/cpython/rev/5f0d1aad7322/ it's 2 function calls (accept + set_blocking)+ 2 syscalls (accept + ioctl FIONBIO) vs 1 function call (accept) + 1 syscall (accept4). -- Sa?l Ibarra Corretg? bettercallsaghul.com From rob.cliffe at btinternet.com Fri Mar 7 12:13:29 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 07 Mar 2014 11:13:29 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> Message-ID: <5319A9D9.8000302@btinternet.com> On 07/03/2014 02:16, Tim Peters wrote: > [Tim] >>> No, it's bizarre to attach a timezone to a time object because most >>> tzinfo subclasses don't - and can't - know what to return for the UTC >>> offset in the absence of - at least - month and day info too. Hour, >>> minute, second and microsecond aren't usually enough to determine >>> whether "daylight time" is in effect, and most tzinfo subclasses do >>> attempt to model daylight time. And because daylight time is a >>> political conceit, most tzinfo subclasses also need year info. That >>> has nothing to do with whether times are viewed abstractly, >>> concretely, etc - it has to do with that time zones generally aren't >>> _defined_ in terms of isolated time-of-day info. It's 7:13 PM in US >>> Central. Is that daylight (CDT) or standard (CST) time? It's >>> impossible to answer. What's the corresponding UTC time? Ditto. > [Andrew Barnert <abarnert at yahoo.com>] >> Well, a time in Central is useless, but a time in CDT or CST is not, and you can >> design a library that's smart enough to give you a CDT or CST (as appropriate) time >> from a Central datetime. > Tell me how. You have, in context, a Python `time` object with a > "Central" tzinfo member. That's all you get from the user. You're > given absolutely no information about day, month, or year. What's > your algorithm for implementing picking CDT or CST "as appropriate"? > Note that we're NOT talking about datetime.datetime objects here. > These are datetime.time objects. > > This isn't a problem for datetime.datetime objects. A "Central" > tzinfo timeclass has no problem at all picking CDT or CST as > appropriate given all the info in a datetime.datetime object. > _______________________________________________ > Guys, please. This discussion is getting increasingly esoteric and has become irrelevant to the original proposition. What it does show is that the current truthy behaviour of time objects is incomprehensible to the average programmer. And therefore useless to her. Rob Cliffe From p.f.moore at gmail.com Fri Mar 7 14:00:10 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 7 Mar 2014 13:00:10 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5319A9D9.8000302@btinternet.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> <5319A9D9.8000302@btinternet.com> Message-ID: <CACac1F_DLxJSPF=j9Nr3DW+7cg4kA1+Dv28kN9ch35G6a6rZ_A@mail.gmail.com> On 7 March 2014 11:13, Rob Cliffe <rob.cliffe at btinternet.com> wrote: > Guys, please. This discussion is getting increasingly esoteric and has > become irrelevant to the original proposition. Everything here is irrelevant to the original proposition. The bug report has been reopened. The next (relevant) step is to create a patch and post it to the bug report. Everything else is just people having a chat... Paul From dickinsm at gmail.com Fri Mar 7 14:07:15 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 7 Mar 2014 13:07:15 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <1F349B3B-7D36-402F-B9FE-93D7EC11FEF8@stufft.io> <5318336F.7030009@egenix.com> <lf9gft$jnh$1@ger.gmane.org> <53184EE4.80105@egenix.com> <lf9jl7$ole$3@ger.gmane.org> <53185658.5080507@egenix.com> <CADiSq7dL5OFaSLhgkmUPOcp2GB4quWGme6u7UZbCjgjYZHs0Fw@mail.gmail.com> Message-ID: <CAAu3qLXgOt0FNrigGjHMW3+S0hbd8kkLUJUrguYDsQmhmW8kAw@mail.gmail.com> On Thu, Mar 6, 2014 at 12:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > Now, let's consider the time with the *best* possible claim to being > false: timestamp zero. How does that behave? > > [... snipped ...] > So, unless I happen to live in UTC, it's highly unlikely that I'm > going to infer from Python's *behaviour* that datetime.time() (unlike > datetime.date() and datetime.datetime()) belong in the "number" > category, rather than the "arbitrary object" category. > This one's even *more* fun if you live in the UK. I live on the Greenwich Meridian (and I mean that quite literally: it goes through the middle of my living room), and since the epoch was in January, well away from British Summer Time, I shouldn't see any of those pesky timezone-related problems that those poor underprivileged people living in non-UTC timezones see. Right? Here's Python 2.7 on my Mac laptop, whose timezone is set to London time: >>> import datetime as dt >>> bool(dt.datetime.fromtimestamp(0).time()) True And on Windows (again with timezone set to the UK): >>> import datetime as dt >>> bool(dt.datetime.fromtimestamp(0).time()) False Wait, what? It turns out that in January 1970, the UK government was in the middle of an experimental change to a year-round GMT+1 timezone. Some operating systems seem to be aware of that fact; some aren't. Count me in the bool(time(0)) should be True camp, by the way. -- Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/5179e860/attachment.html> From breamoreboy at yahoo.co.uk Fri Mar 7 15:41:18 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 07 Mar 2014 14:41:18 +0000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5319A9D9.8000302@btinternet.com> References: <201403051216.12392.shai@platonix.com> <loom.20140305T220007-633@post.gmane.org> <CAP7h-xYBUCSHr=Ls+2Ya3rtBXG0EcaXrMo9vcuK7RRtoQpPyYw@mail.gmail.com> <53179F55.3080501@egenix.com> <85siqwcmvt.fsf@benfinney.id.au> <5317A696.8040605@egenix.com> <CAExdVNkv+S--nqQqqKyM8=Rw=a3T7yCUM03HCvZchyrAWwug3g@mail.gmail.com> <53181ADA.2030401@canterbury.ac.nz> <CAExdVNkjrs9Kt+MdgyY+0oHSPK=qzf8_X0DoR49FWH0ADgKMgA@mail.gmail.com> <5318F8F2.5010209@canterbury.ac.nz> <CAExdVN=izRxLkDTN+pYV4SG-jJT07XCJvWjYR=SMu3Zu7sBP7g@mail.gmail.com> <1394158163.96645.YahooMailNeo@web181005.mail.ne1.yahoo.com> <CAExdVNkvECuTeBkXt1ZqdyP7pfCOYD60nOcxt=iuH=WWxz7b8g@mail.gmail.com> <5319A9D9.8000302@btinternet.com> Message-ID: <lfclqa$jhd$1@ger.gmane.org> On 07/03/2014 11:13, Rob Cliffe wrote: > > On 07/03/2014 02:16, Tim Peters wrote: >> [Tim] >>>> No, it's bizarre to attach a timezone to a time object because most >>>> tzinfo subclasses don't - and can't - know what to return for the UTC >>>> offset in the absence of - at least - month and day info too. Hour, >>>> minute, second and microsecond aren't usually enough to determine >>>> whether "daylight time" is in effect, and most tzinfo subclasses do >>>> attempt to model daylight time. And because daylight time is a >>>> political conceit, most tzinfo subclasses also need year info. That >>>> has nothing to do with whether times are viewed abstractly, >>>> concretely, etc - it has to do with that time zones generally aren't >>>> _defined_ in terms of isolated time-of-day info. It's 7:13 PM in US >>>> Central. Is that daylight (CDT) or standard (CST) time? It's >>>> impossible to answer. What's the corresponding UTC time? Ditto. >> [Andrew Barnert <abarnert at yahoo.com>] >>> Well, a time in Central is useless, but a time in CDT or CST is not, >>> and you can >>> design a library that's smart enough to give you a CDT or CST (as >>> appropriate) time >>> from a Central datetime. >> Tell me how. You have, in context, a Python `time` object with a >> "Central" tzinfo member. That's all you get from the user. You're >> given absolutely no information about day, month, or year. What's >> your algorithm for implementing picking CDT or CST "as appropriate"? >> Note that we're NOT talking about datetime.datetime objects here. >> These are datetime.time objects. >> >> This isn't a problem for datetime.datetime objects. A "Central" >> tzinfo timeclass has no problem at all picking CDT or CST as >> appropriate given all the info in a datetime.datetime object. >> _______________________________________________ >> > Guys, please. This discussion is getting increasingly esoteric and has > become irrelevant to the original proposition. > What it does show is that the current truthy behaviour of time objects > is incomprehensible to the average programmer. And therefore useless to > her. > Rob Cliffe I disagree. People here have stated that they use it. I could use it if I wanted to, having been raised to do some strange things, as in read docs, so I'm firmly in favour of sticking with the status quo. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From stephen at xemacs.org Fri Mar 7 15:59:38 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Mar 2014 23:59:38 +0900 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> <20140307103956.GT28804@ando> <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> Message-ID: <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Angelico writes: > That effect, yes. Let's call it "the magic of float.__str__", because > it really is pretty amazing. > > But it's still post-processing magic. It means that strings appear to > round-trip through floats, as long as you're a long way within the > available precision; but as soon as you do operations, that ceases to > be the case. I think it's great for display, but is putting that into > __repr__ (at least, they do appear to be the same) an attractive > nuisance, in that it encourages people to treat float("...") as a true > representation? What makes you think they need more encouragment? Seriously, as one data point, I don't think having more "human" representations encourages me the think of floating point results as the product of arithmetic on real numbers. I don't think anybody who knows how tricky "floating point" arithmetic can be is going to be fooled by the "pretty eyes" of a number represented as "2.0" rather than "1.99999999999999743591". From rosuav at gmail.com Fri Mar 7 16:09:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Mar 2014 02:09:12 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp> References: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAPTjJmp=UaJbQGbbY3+9srWd4qyJRDSmKPusYHmLVWv+tTk8HA@mail.gmail.com> <9a4504fb-a5a9-4a9b-a07e-d004ebee96e7@googlegroups.com> <CAPTjJmrpvMyRL-mHybbXXTy1qvrkAtmCSZBBF_x1CABx=xRMJw@mail.gmail.com> <20140307103956.GT28804@ando> <CAPTjJmo_fe=4ObzXsaohiygL7gL+FZPF4cuSXTgy07uw6WK55g@mail.gmail.com> <878usmoy05.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <CAPTjJmp4YhMrix5p6B9=4o6sWQW5wdMmGM0RNz2i6E0D__ZckQ@mail.gmail.com> On Sat, Mar 8, 2014 at 1:59 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > Seriously, as one data point, I don't think having more "human" > representations encourages me the think of floating point results as > the product of arithmetic on real numbers. I don't think anybody who > knows how tricky "floating point" arithmetic can be is going to be > fooled by the "pretty eyes" of a number represented as "2.0" rather > than "1.99999999999999743591". Fair enough. I just remember reading, back in my really REALLY early days with GW-BASIC, an explanation of why 3.2# (the hash made it double-precision) came out as whatever-it-did. Went into a full explanation of the nature of binary floating point, and the issue was forced to your attention because just about _any_ value that wasn't a neat multiple of a (negative) power of two would do that. You can lead a programmer to docs, but you can't make him understand. ChrisA From stephen at xemacs.org Fri Mar 7 16:53:55 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 08 Mar 2014 00:53:55 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <5319A4A0.10905@btinternet.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> Message-ID: <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> Rob Cliffe writes: > Please read what I actually said: > ??? Let's start a survey. Ah, OK. You started a survey, but don't care if it is accurate. I remain deeply unimpressed with your methodology. From guido at python.org Fri Mar 7 18:17:45 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 09:17:45 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> If I had to do it over again I would *definitely* never make a time value "falsy". The datetime module was conceived long ago, when the dangers of falsy objects that weren't clearly trying to be "like numbers" or "like collections" weren't so clear. Today I would argue that times aren't enough "like numbers" to qualify. The only question left in my mind is how safe it is to change the current behavior, given that it's been documented and implemented for over a decade. If this was for Python 3.0 (or even 3.1) I would definitely vote for fixing it, damn the torpedoes. But given that we're talking about 3.5, I'm more hesitant. It's also not something for which a deprecation warning would work well. (When would it be issued? Whenever bool(<time>) is called? Or only when it's about to return False?) Still, my intuition tells me that it's pretty much a nuisance feature and few people have actually relied on it, so I'd be okay with fixing (and breaking!) this in 3.5, perhaps after a thorough search for how much the feature is actually relied upon and how legitimate or important those uses are. A search should *also* try to unearth code that is technically *broken* by the current behavior. I would expect there to be quite a lot of the latter kind, and very few of the former. Fearing I have said nothing new, -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/ce710a46/attachment.html> From ncoghlan at gmail.com Fri Mar 7 18:30:52 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 8 Mar 2014 03:30:52 +1000 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> Message-ID: <CADiSq7d3FrsFzk=DW=fta6EFh7S_7oX3=-efukc0O0edva=rPQ@mail.gmail.com> On 8 Mar 2014 03:19, "Guido van Rossum" <guido at python.org> wrote: > > If I had to do it over again I would *definitely* never make a time value "falsy". The datetime module was conceived long ago, when the dangers of falsy objects that weren't clearly trying to be "like numbers" or "like collections" weren't so clear. Today I would argue that times aren't enough "like numbers" to qualify. > > The only question left in my mind is how safe it is to change the current behavior, given that it's been documented and implemented for over a decade. If this was for Python 3.0 (or even 3.1) I would definitely vote for fixing it, damn the torpedoes. But given that we're talking about 3.5, I'm more hesitant. It's also not something for which a deprecation warning would work well. (When would it be issued? Whenever bool(<time>) is called? Or only when it's about to return False?) I was proposing a deprecation warning when it would be about to return False, advising the use of an explicit comparison instead. My rationale was that people that actually rely on it are at least somewhat likely to be testing it and would trigger the warning and have a chance to change their code. However, on reflection, just changing with a note in the porting guide would actually have a similar consequence. Regards, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/b4b68e06/attachment.html> From stephen at xemacs.org Fri Mar 7 18:50:26 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 08 Mar 2014 02:50:26 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> Message-ID: <874n39q4nx.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > It's also not something for which a deprecation warning would work > well. (When would it be issued? Whenever bool(<time>) is called? Yes! Yes! Yes! Just joking -- that's got to be the wrong thing to do, because it would warn on precisely the construct (None as sentinel for "uninitialized") that's the only reason anybody wants this. > Or only when it's about to return False?) That seems reasonable to me. People who are testing for this are presumably *constructing* instances of midnight, either with a crontab of "* * * * *" or in a simulation -- they'll hit it frequently. The other case that seems plausible is using a zero time as a sentinel for "uninitialized" or "unspecified", and that *must* get a warning. > Fearing I have said nothing new, IIRC, you're the only person to have mentioned the pragmatic issues about the DeprecationWarning. From ethan at stoneleaf.us Fri Mar 7 19:07:19 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 07 Mar 2014 10:07:19 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <lf7q2a$vpl$1@ger.gmane.org> References: <201403051216.12392.shai@platonix.com> <CACac1F_qkYQRSbm-vL3OdX0a4mYr0XSXVZDS5SkUe+Px77NCdg@mail.gmail.com> <CANc-5UzH_r-S92fJPAuajhHgcWgeJ_EW9cOGEFCJngrZ6z7JLQ@mail.gmail.com> <201403051715.29773.shai@platonix.com> <CANc-5Uze4nB1FPrS5sCz7GvFeSDChJe_LU8t-9Ef_DXjP9qWoA@mail.gmail.com> <CAE0SK644pDu0hk7BnFsA-PH+4wofnRcLo7XZdxVHywKNbxUOxw@mail.gmail.com> <CACac1F-wk9ifRO0qMrAe=yrALA4xR0j+wjy-JRC7EMLhSZApHQ@mail.gmail.com> <CAE0SK66cwz4d1bP3oKaJ_n4t_2+x56gMCFRtR6N94U+SriibGw@mail.gmail.com> <lf7q2a$vpl$1@ger.gmane.org> Message-ID: <531A0AD7.6020504@stoneleaf.us> On 03/05/2014 10:23 AM, Mark Lawrence wrote: > On 05/03/2014 18:12, Amber Yust wrote: >> >> I think this is the wrong stage to evaluate cost. After all, one of the >> tenets of open source is that the core devs don't have to be the source >> of all change. >> > > But the core devs have to review the change and ultimately decide whether > or not to commit it. In this case it looks as if a patch would not be > accepted, so why keep going on about it, especially when there's a known > work around? The point of discussion is to find out what all the issues are, not to give up with "oh well, the core devs aren't going to accept it anyway..." -- ~Ethan~ From harrismh777 at gmail.com Fri Mar 7 20:01:01 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 11:01:01 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307101235.GR28804@ando> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> Message-ID: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> On Friday, March 7, 2014 4:12:35 AM UTC-6, Steven D'Aprano wrote: It's statements like this [ AI language ] that strongly suggest that you > don't really > understand how numbers work in computer programming. Or the practical > limitations of what is *possible*. And as soon as you start talking > about using an AI to decide what value the user intended, you've entered > crackpot territory. hi Steven, fair enough. When I refer to AI all I'm really stating is that some form of computational intelligence needs to be employed beyond simple copying, &c. An intelligent decision needs to be used; that's all. I'll stop using it (AI) , its not helpful. Also, thank you for continuing to discuss this, it helps me grow, and it may lead to an understanding or change. That's a good thing. Here is the main post for today, from me, having slept on this and putting some things back into historical timeframe. Sometime before Py2.7.x it was not possible to promote a binary float to a decimal float; the system threw an exception (convert to string). First, that was the correct way to handle this issue from the inception of decimal.Decimal. Throw an exception and let everyone know that there is a problem; then let the user figure out how to solve it. It is not possible to correctly promote a binary float to a decimal float. The decision (in Py2.7?) was to NOT throw an exception and to copy the binary float "exactly" into the decimal float. Second, that was a bad decision--really--IMHO. It was the wrong decision for two reasons 1) it denies what is inherently poor about binary floating point representation in the first place, and 2) further compounds the problem by denying what is the benefit of using Decimal floating point in the first place. Not to mention, it gives the wrong answer; from a decimal floating point perspective it gives a VERY wrong answer. (I realize most science and engineering solutions only require 3-4 digits of accuracy---heck, I used to do that work on a slide rule). The point here is that binary floats should not be promoted to decimal floats by using "exact" copy from binary float representation to decimal float representation. This reminds me of the guys at three mile island who turned off the alarms because they were annoying. Steven, I am going to try to go a little further without AI speak... What needs to happen is that binary floats need to be "correctly" promoted to decimal floats as appropriate. This must not happen by simple copying (that does not work). There needs to be some policy in place that will correctly "set" the decimal float value based on intelligence, not just simple copying. ( It may not be possible, or it may require some funky copy b float-->reduce context round-->set decimal . I'm still thinking about it. To make decimal the default, and to handle all numbers correctly giving the SciPy community the ability to use binary floats easily too (without breaking codes) will require a complete rewrite of numbers.Number and maybe some other things. Unifying the numbers concept in python is desirable but not practical at this point... I understand that; I get it. I am not asking anyone for this kind of "sweeping" reform at this point... just think forward to the possibility. Please. Stefan's suggestion to set a trap is an excellent one, and I will play around with that a bit. Although I was initially asking for a method to more easily enter a literal decimal, and that is still important, I think really what is called for here considering the history is to rescind the decision to promote binary floats to decimal floats through an "exact" copy without flagging an error, or even giving a warning. I am remembering my Zen of python... special cases aren't special enough to break the rules, although practicality beats purity, errors should never pass silently. Anyway, I know you wonderful guys will be thinking hard about this (evidenced by the discussion so far) and I leave it in your competent hands. I too will be thinking about it, perhaps adding some coding in the near future that will be helpful. Again, I wish to thank Stefan Krah for his fine work on decimal.Decimal, without which none of this discussion would be worth having in the first place. Kind regards, marcus > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/d5fac344/attachment-0001.html> From guido at python.org Fri Mar 7 20:20:00 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 11:20:00 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> Message-ID: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> Mark, I don't know if it has occurred to you, but several of the rhetorical forms you have used are pretty upsetting to many Python devs. (I can give you a list offline.) It would really help the discussion if you tried to keep your rhetoric in check. (A few other participants might also want to focus more on the topic and less on Mark.) Now I'd like to restate the core issue in ways that should appeal to Python devs. I am describing the status quo in Python 3.4, since that's the starting point for any evolution of Python. Please bear with me as I describe the status quo -- I find it necessary as a context for any future proposals. Python's parser recognizes a few different types of number literals: integers (e.g. 42), "decimal point" notation (e.g. 3.14), "exponential notation" (e.g. 2.1e12), and any of these followed by 'j'. There are also binary, octal and hex notations for integers (e.g. 0b1010, 0o12, 0xa). The parser, by design, does not have any understanding of the context in which these numbers are used -- just as it doesn't know about the types of variables (even when it's obvious -- e.g. after seeing "x = 4" the parser does not keep track of the fact that x is now an integer with the value 4, although it does remember it has seen an assignment to x). What the parser does with those number literals (as with string literals and with the keywords None, True, False) is that it creates an object of an appropriate type with an appropriate value. This object is then used as a value in the expression in which it occurs. The current algorithm for creating the object is to create an int (which is implemented as an exactly represented arbitrary-precision integer) when the literal is either binary/octal/hex or decimal without point or exponent, a Python float (which is an object wrapping an IEEE 754 double) when a decimal point or exponent is present, and a Python complex (which wraps *two* IEEE doubles) when a trailing 'j' is seen. Note that negative numbers are not literals -- e.g. -10 is an expression which applies the unary minus operator to an integer object with the value 10. (Similarly, IEEE inf and NaN don't have literals but requires evaluating an expression, e.g. float('inf').) Not all numbers come from literals -- most come from expressions such as x+y, some are read from files or other input media. (E.g. the struct module can "read" IEEE floats and doubles from byte strings containing their "native" binary encoding, and float() with a string argument can interpret decimal numbers with optional decimal point and/or exponent.) The clever thing about expressions (in Python as in many languages) is that the overloading of operators can make the "right" think happen when numbers of different types are combined. Consider x+y. If x and y are both ints, the result is an int (and is computed exactly). If either is a float and the other an int, the result is a float. If either is a complex and the other is an int or float, the result is a float. If you are defining your own number-ish type in Python or C code, you can make it play this game too, and this is how the current Decimal and Fraction types work. Python's operator overloading machinery is flexible enough so that e.g. Fraction can say "if a Fraction and an int are added, the result is a Fraction; but if a Fraction and a float are added, the result is a float." (Because Python is dynamic, it would be *possible* to define a Fraction type that returns a plain int for results whose denominator is 1, but the stdlib's Fraction type doesn't do this.) Most of the time, the built-in types are conservative in their return type (by which I mean that adding two ints returns an int, not a float), but there are a few exceptions: int/int returns a float (because returning an int would require truncation or a compound (div, mod) result), and the ** (power) operator also has a few special cases (int**negative_int and negative_float**non_integer). Now let's move on to the problems, which begin with the current float type. Given the representation it is obvious that various limitations exist; results may have to be truncated to fit into 64 bits, and conversion to and from decimal cannot always preserve the exact value. It turns out that the truncation of results doesn't bother most users (everyone knows what a typical hand calculator does for 1/3), but the issues around conversion to and from decimal often trip over users on their initial forays into learning the language. The improvement we made to the output conversion (dropping digits that don't affect round-tripping) take some of the sting out of this, but Python newbies still excitedly point out some of the unavoidable anomalies like 1.1 + 2.2 ==> 3.3000000000000003 when they first discover it. So what to do about it? If we aren't too worried about strict backwards compatibility, there's an easy answer that should shut up the newbies: change the parser so that when it sees a number with a decimal point or an exponent it returns a Decimal instance, and make a bunch of other adjustments to match. (E.g repr() of the Decimal 3.14 should return '3.14' instead of "Decimal('3.13')".) The float() builtin should also return a Decimal (we should probably just rename the whole type to 'float'). The math module will have to be rewritten. We should probably preserve the existing binary float under some other name, perhaps requiring an import. We'll have to decide what to do about complex numbers. (One option would be to remove supporting them in the parser.) The details can be worked out in a PEP. I don't expect this process to be easy or without controversies, but I'm confident we could come up with a good design. But there are big problems with such a proposal. Those newbies perhaps aren't Python's most important user population. (And for many of them it is actually a moment of enlightenment on their way to becoming experts, once they hear the explanation.) The problems with binary floats don't affect most actual calculations (never mind 1.1+2.2, once you've seen what 1/3 looks like you switch all your output to some kind of rounded format). Many of the problems with floating point that actually matter (such as truncation, or infinities, or NaN) don't go away just by switching everything to Decimal. For many Python programs the switch to decimal would not matter at all. For example, in all my own recent coding (which focuses on network I/O), the only uses I've had for float is for timekeeping (which is inherently approximate) and to prints some percentages in a report. Such code wouldn't be affected at all -- it wouldn't break, but it wouldn't be any simpler either. Decimal/binary simply isn't an issue here. However, for the growing contingent of scientists who use Python as a replacement for Matlab (not Mathematica!), it could be a big nuisance. They don't care about decimal issues (they actually like binary better) and they write lots of C++ code that interfaces between CPython's internal API and various C++ libraries for numeric data processing, all of which use IEEE binary. (If anything, they'd probably want a 32-bit binary float literal more than a decimal float literal.) Python already defines some macros for converting between standard Python float objects and C++ doubles (e.g. PyFloat_AS_DOUBLE), so technically we could support source-code compatibility for these users, but performance would definitely suffer (currently that macro just returns the C++ double value already present in the object; it would have to be changed to call a costly decimal-to-binary macro). There is also the huge effort of actually implementing such a proposal. It's not insurmountable, but I estimate it would be at least as much work as the str->unicode conversion we did for Python 3. All in all I just don't see the stars aligned for this proposal. (Nor for anything even more sweeping like representing 1/3 as a fraction or some other symbolic representation.) So what to do instead? A few simper proposals have been made. Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.) I'm not excited about a notation for Fraction -- the use cases are too esoteric. Maybe we can fix the conversion between Decimal and float (if this is really all that matters to Mark, as it appears to be from his last email -- I'd already written most of the above before it arrived). Could it be as simple as converting the float to a string using repr()? Given the smarts in the float repr() that should fix the examples Mark complained about. Are there any roadblocks in the implementation or assumptions of the Decimal type here? Perhaps the default Decimal context being set for a much higher precision makes it philosophically unacceptable? I think I've said as much as I can, for now. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/547dd34f/attachment.html> From mertz at gnosis.cx Fri Mar 7 21:28:41 2014 From: mertz at gnosis.cx (David Mertz) Date: Fri, 7 Mar 2014 12:28:41 -0800 Subject: [Python-ideas] Fwd: Python Numbers as Human Concept Decimal System In-Reply-To: <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <2f5ad6d3-d9e5-4866-a565-9b7a80977a1d@googlegroups.com> <CAEbHw4Z328DxAr4nB7qD3+-GpwnGqw5Agpv6ayjMxFVMpiy77g@mail.gmail.com> Message-ID: <CAEbHw4aNgFuGV+JYz1Z_T4XNnMg_ftpkZmnVE3RGvfEwZSEpow@mail.gmail.com> On Thu, Mar 6, 2014 at 6:51 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > I hate this: > >>> from decimal import Decimal > >>> a=Decimal(1) > >>> b=Decimal(.1) > >>> a+b > Decimal('1.100000000000000005551115123') > >>> > ... and I think the people who DO know intimately how python works, should > do something to fix it. > The problem here--and the problem with the "default number type is decimal" is that what you want fixed is "unfixable." As a number of other people have noted, what you are demanding is that numbers have finite sized and exact representations, which they simply don't and can't in computers with finite memory. Your example, Mark, is merely one where you have misspelled your Python code: >>> from decimal import Decimal as D >>> D('1') + D('.1') Decimal('1.1') Fix your spelling error, and in this case you get what you want. However, what about this: >>> D(1/3) * 3 Decimal('0.9999999999999999444888487687') There's no way to fix that spelling. Yes, you might be able to play with a decimal.Context in a way that gets this one example to happen to round the way you want it to, but no context will do so generically for every similar expected identity under reciprocal operators. By coincidence, this works under binary FB *because* of the limited precision of floats: >>> 1/3 * 3 1.0 But that's just luck, basically, it's not because we've managed to precisely represent 1/3 as a binary float. We don't have to look all that far for a problem case: >>> 1/49 * 49 0.9999999999999999 Of course, we do have Fractions, which will always do the right thing under the reciprocal division and multiplication operators: >>> F('1/3') * 3 Fraction(1, 1) Don't misspell this either, of course: >>> F(1/3) * 3 Fraction(18014398509481983, 18014398509481984) However, even if Fraction were to become the default numeric type for Python, and preserve the reciprocity of div/mul, that wouldn't help us any under the hoped reciprocals of sqrt() and square. Similarly, of course, for trigonometric identities and various other numeric functions that produce irrational answers. E.g.: >>> from fractions import Fraction as F >>> sqrt(F('1/3'))**2 0.3333333333333333 >>> F(sqrt(F('1/3'))**2) Fraction(6004799503160661, 18014398509481984) Well, that answer is a binary floating point; we leave Fraction land pretty easily in Python. The float we get, of course, is something that's not exactly 1/3 (but is pretty close). Even if we imagined some future version of Python that did sqrt() purely as a Fraction without converting to IEEE 754, we STILL couldn't ever have an exact identity. There simply does not exist any Fraction that can accurately represent sqrt(F('1/3')); there are only rational numbers that are "pretty close." Maybe this one, for example: >>> F(sqrt(F('1/3'))) Fraction(1300077228592327, 2251799813685248) Maybe this hypothetical Python 4000 that didn't do the conversion to floating point would produce a different approximation, but it would always be an approximation, not the Real (irrational) answer. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/c4ea31e5/attachment-0001.html> From random832 at fastmail.us Fri Mar 7 21:39:03 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 07 Mar 2014 15:39:03 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394007018.19177.YahooMailNeo@web181004.mail.ne1.yahoo.com> <CAHVvXxSdQnvAXorpdR6uTj_Kzo==iCoE=piXtx2otsMyLdQz-w@mail.gmail.com> <20140305120410.GY28804@ando> <CAHVvXxSsYX1PCkv2WOwBwZx=yp-edhZxv6q_4QcpaoOQgFbzPA@mail.gmail.com> Message-ID: <1394224743.8914.91895745.755B4C9A@webmail.messagingengine.com> On Wed, Mar 5, 2014, at 7:30, Oscar Benjamin wrote: > In C and some other languages the f suffix indicates a numeric literal > that has type "float" e.g. "6.0f". You can use upper case there as > well but the convention is lower case and to include the .0 when it's > an integer. I just though that 3F looks sufficiently distinct from the > way it's typically done in C. How about 3r? On the subject of suffixes, it might be worth considering that they often use "d" for "double" as well. C# uses "m" (for money) for decimal. From random832 at fastmail.us Fri Mar 7 21:45:51 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 07 Mar 2014 15:45:51 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> Message-ID: <1394225151.11384.91897925.4D8F1ECA@webmail.messagingengine.com> On Wed, Mar 5, 2014, at 21:21, Mark H. Harris wrote: > "Then, and only then, a function that requires PI or half_PI > will first check to see if __gpi__ has been cached and if so pull it > back, or if not calculate it (once). If the context > does not change then the function can take half of some whopping value of > PI, for the duration of the context, and > use that, but if not needed for the context will never bind it ! What if you want a number type to represent a fraction of pi exactly, so you can take its sine and get the square root of a rational number, then square it to get the rational number? Or to take the sine of an arc tangent of a rational number. From random832 at fastmail.us Fri Mar 7 21:48:04 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 07 Mar 2014 15:48:04 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com> On Thu, Mar 6, 2014, at 7:46, Stephen J. Turnbull wrote: > Steven D'Aprano writes: > > > The problem is that fractions can be unbounded in memory, > > Seminumerical Algorithms has a few problems on "floating slash", ie a > representation of numbers where the total memory allocated to a > fraction is fixed, but the amounts allocated to numerator and > denominator are variable. I don't know if it has ever been tried in > practice, though. And of course any reasonable fixed size would have > very limited ability to express very large or very small magnitudes. Or, for that matter, you could limit both the numerator and the denominator to some maximum value - when an intermediate result exceeds it (won't be more than the square of the maximum for most operations), you find the closest representable value. From abarnert at yahoo.com Fri Mar 7 22:24:06 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 7 Mar 2014 13:24:06 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> Message-ID: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> On Mar 7, 2014, at 11:01, "Mark H. Harris" <harrismh777 at gmail.com> wrote: > > Here is the main post for today, from me, having slept on this and putting some things > back into historical timeframe. Sometime before Py2.7.x it was not possible to promote > a binary float to a decimal float; the system threw an exception (convert to string). > > First, that was the correct way to handle this issue from the inception of decimal.Decimal. Throw > an exception and let everyone know that there is a problem; then let the user figure out how to > solve it. The decision was discussed at the time, and all the pros and cons were hashed out. If you're not willing to read that discussion, your opinion that the change was a mistake is worth exactly as much as that of any individual user who asked for the change. > It is not possible to correctly promote a binary float to a decimal float. The decision (in Py2.7?) > was to NOT throw an exception and to copy the binary float "exactly" into the decimal float. There is no such thing as copying a binary float exactly into a decimal float. Decimal float values are not a superset of binary float values; in either direction, you have to round. Which is what Python 2.7+ does. > Steven, I am going to try to go a little further without AI speak... What needs to happen is that > binary floats need to be "correctly" promoted to decimal floats as appropriate. This must not happen > by simple copying (that does not work). There needs to be some policy in place that will correctly > "set" the decimal float value based on intelligence, not just simple copying. ( It may not be possible, or > it may require some funky copy b float-->reduce context round-->set decimal . I'm still thinking about it. Python is _already_ doing exactly what you're asking for. Each binary float is rounded to the closest possible decimal float. And it's hard to imagine what else it _could_ be doing. There is no closer decimal than the closest decimal, so any "smarter" algorithm would either do the exact same thing, or be worse. It's true that this "doesn't work" to solve your problem, but that's only because you're trying to solve an impossible problem. Once you've rounded a number inexactly (whether to binary or decimal), you can't get the lost data back. > To make decimal the default, and to handle all numbers correctly giving the SciPy community the > ability to use binary floats easily too (without breaking codes) will require a complete rewrite of > numbers.Number and maybe some other things. Why do you keep saying that even after it's been repeatedly explained to you that it wouldn't? Number and its subclasses do not care about the representation of the number. Guido explained to you exactly what would and wouldn't need to be changed, and the abstract base classes do not enter into it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/14713e49/attachment.html> From harrismh777 at gmail.com Fri Mar 7 23:18:57 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 14:18:57 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> Message-ID: <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com> On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote: > > It is not possible to correctly promote a binary float to a decimal float. > The decision (in Py2.7?) > was to NOT throw an exception and to copy the binary float "exactly" into > the decimal float. > > > There is no such thing as copying a binary float exactly into a decimal > float. Decimal float values are not a superset of binary float values; in > either direction, you have to round. Which is what Python 2.7+ does. > > Andrew, you are deliberately side-stepping the point. You apparently disagree with Steven D, and others, who have already stated repeatedly on this list that on 2.7+ binary float literals are copies "exactly" to decimal float representation! That is what is causing this entire problem, sir. That is the reason for this: >>> from decimal import Decimal >>> Decimal(.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') <====== this is not a rounding problem >>> It is a copy problem, and it should not be allowed. It was a bad decision to approach the problem this way, and the correct response to it is to not allow it, throw an exception, and let the user fix it. Otherwise, as I've stated earlier, there needs to be a policy that promotes the binary floating point representation to decimal floating point representation without the mess from the 16th digit out.... and, that may not be possible (needs to be discussed, and researched). It is clear from Guido's response earlier this afternoon that he understands the problem, and has some good ideas for how to proceed. I am happy with that, and will keep monitoring the news to see how things go. Thanks for your feedback. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/1e718cc8/attachment-0001.html> From cf.natali at gmail.com Fri Mar 7 23:33:27 2014 From: cf.natali at gmail.com (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=) Date: Fri, 7 Mar 2014 22:33:27 +0000 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode In-Reply-To: <5319A9FC.1010609@gmail.com> References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol> <5319A9FC.1010609@gmail.com> Message-ID: <CAH_1eM0UJTPXTueg9+kRRAxBgnZTsZd38pFEjmHE-K9qWQY9KQ@mail.gmail.com> accept4() was added for atomicity (O_CLOEXEC), not performance. For pipe2() vs pipe(), this could make a measurable difference because the base syscall is so fast, but for accept(), which is a blocking syscall, shaving one or two syscalls are unlikely to yield any performance gain. IMO that's not a reason good enough to make accept() signature more complex. cf 2014-03-07 11:14 GMT+00:00 Sa?l Ibarra Corretg? <saghul at gmail.com>: > On 03/07/2014 11:40 AM, Antoine Pitrou wrote: > >> On Fri, 07 Mar 2014 10:37:58 +0100 >> Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> >> wrote: >> >>> >>> accept4 allows us to do this with a single syscall, and it made it into >>> FreeBSD 10, so that's another system that could benefit from this >>> optimization. >>> >> >> If it doesn't, then perhaps the configure script needs to be fixed. >> >> > Probably not, I'll have a look. > > > The idea is to accept an incoming connection and make it non-blocking at >>> the same time, asyncio and other frameworks would benefit from this by >>> doing one function call and 2 syscalls less (on supported systems, that >>> is). >>> >> >> That's not likely to do a significant difference (benchmarks welcome). >> >> > Actually, after http://hg.python.org/cpython/rev/5f0d1aad7322/ it's 2 > function calls (accept + set_blocking)+ 2 syscalls (accept + ioctl FIONBIO) > vs 1 function call (accept) + 1 syscall (accept4). > > > -- > Sa?l Ibarra Corretg? > bettercallsaghul.com > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/9ee8b132/attachment.html> From ckaynor at zindagigames.com Fri Mar 7 23:33:18 2014 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Fri, 7 Mar 2014 14:33:18 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> Message-ID: <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com> On Fri, Mar 7, 2014 at 1:24 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > There is no such thing as copying a binary float exactly into a decimal > float. Decimal float values are not a superset of binary float values; in > either direction, you have to round. Which is what Python 2.7+ does. Presuming enough digits of precision, every binary float could be represented precisely in decimal. Decimal is base 10, which factors to 2 and 5, meaning all powers of 1/2 and 1/5 can be precisely represented. Binary floats can only be powers of 1/2. Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/92bf222e/attachment.html> From mertz at gnosis.cx Sat Mar 8 00:12:00 2014 From: mertz at gnosis.cx (David Mertz) Date: Fri, 7 Mar 2014 15:12:00 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <CALvWhxvKs0JtqErmeW7UfFVGrp6KwkuKdWvKQ9Li2fsc6=rBWg@mail.gmail.com> Message-ID: <CAEbHw4Yavjb1x71dpTAZrjRaYk6302pm=BV2wWtT96RYtyqvsQ@mail.gmail.com> On Fri, Mar 7, 2014 at 2:33 PM, Chris Kaynor <ckaynor at zindagigames.com>wrote: > Presuming enough digits of precision, every binary float could be > represented precisely in decimal. Decimal is base 10, which factors to 2 > and 5, meaning all powers of 1/2 and 1/5 can be precisely represented. > Binary floats can only be powers of 1/2. > This is half true. Yes, the factors of 10 are 2 and 5. So every number that can be exactly represented in base-2 or in base-5 can also be represented exactly in base-10 (given enough digits. The problem is that the converse is not the case. So in the case Mark presents several times: >>> from decimal import Decimal >>> a = 0.1 # store a binary float >>> Decimal(a) Decimal('0.1000000000000000055511151231257827021181583404541015625') That decimal number *is* an exact representation of the value referenced by 'a'. So if that's all you want, no problem exists. But the problem that does exist is that there is *no* binary floating point representation that is exactly the same numeric value as Decimal('0.1'). This would be true, not only for float32 or float64 values that Python has (depending on the machine architecture and compilation switches), it would also be true if we used float128, or float256, or float8192 numbers. And again, while that one example might lead you to think that decimal floating point is simply better, we don't need to go far to find a value like Fraction('1/3') that simply cannot have an exact representation in decimal either, no matter how many digits of precision we decide to use. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/aefcef70/attachment.html> From abarnert at yahoo.com Sat Mar 8 00:14:59 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 7 Mar 2014 15:14:59 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <31145a81-79e1-4072-ad0e-bddf30947f55@googlegroups.com> Message-ID: <5A794BFD-C3D8-4E09-9FAE-B693AD664B8F@yahoo.com> On Mar 7, 2014, at 14:18, "Mark H. Harris" <harrismh777 at gmail.com> wrote: > Andrew, you are deliberately side-stepping the point. You apparently disagree with Steven D, and others, > who have already stated repeatedly on this list that on 2.7+ binary float literals are copies "exactly" to > decimal float representation! That is what is causing this entire problem, sir. What's causing the problem is that 0.1 and float(0.1) can both be represented exactly in Decimal--and that they are different. You've lost information in converting to float (I may have stated that the wrong way around last time; if so, apologies), and you cannot regain that information when converting back. If you assume people will only ever try to work with numbers that are representable in a few decimal digits, you can use that additional information to retrieve the list data. But that assumption clearly doesn't make sense generally, and if you treat all float values that way, you'll be adding more error on average. > That is the reason for this: > >>> from decimal import Decimal > >>> Decimal(.1) > Decimal('0.1000000000000000055511151231257827021181583404541015625') <====== this is not a rounding problem Yes it is, it's a rounding problem converting the ".1" literal to a float. There is no exact decimal->float->decimal conversion, which is why the right way to solve this is to avoid the float conversion--e.g., with your decimal literal syntax, or just constructing decimals from stings. From harrismh777 at gmail.com Sat Mar 8 00:39:59 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 15:39:59 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> Message-ID: <1c6f1e58-f30d-4b12-b7bf-abee9a801215@googlegroups.com> On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote: > The decision was discussed at the time, and all the pros and cons were > hashed out. If you're not willing to read that discussion, your opinion > that the change was a mistake is worth exactly as much as that of any > individual user who asked for the change. > > hi Andrew, I am satisfied that the problem is well understood and that we no longer need to discuss the root cause, nor the computer science of trying to represent numbers in memory that are not representable. I admire you guys too much to keep going with the previous discussion. I am completely satisfied with Guido's last answer (thank you, by the way), and I look forward to seeing how the community will address the main issue. In response to your statement quoted above, I would like to suggest seeing if there might be any way you could think of wording that, in future dialogue? The response quoted above is not only another *ad hominem* attack, it is also a straw man attack. Your presumption that I am unwilling to read the discussion archive is unfounded and faulty thinking. In fact, I began a thorough attempt to understand the previous discussion as soon as I discovered the need through Steven's revelation last evening. In fact, I am very interested in the discussion which led to the decision in question. I not only find it interesting philosophically I also am interested in who was involved and how they were thinking at the time. Of course none of that changes my opinion at this time that the decision was not correct. I don't fully know yet, but I think it was made the way many decisions are made by young men; like lightning, ---shortest path to ground. It would be helpful going forward in future discussions with me that you presume not to put words in my mouth, nor to presume to know how I think or feel, nor presume to attack my character or my person. I would really appreciate it. Kind regards, marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/1768c691/attachment.html> From oscar.j.benjamin at gmail.com Sat Mar 8 00:42:14 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 7 Mar 2014 23:42:14 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> Message-ID: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> On 7 March 2014 19:20, Guido van Rossum <guido at python.org> wrote: > > Maybe we can fix the conversion between Decimal and float (if this is really > all that matters to Mark, as it appears to be from his last email -- I'd > already written most of the above before it arrived). Could it be as simple > as converting the float to a string using repr()? Given the smarts in the > float repr() that should fix the examples Mark complained about. Are there > any roadblocks in the implementation or assumptions of the Decimal type > here? The current Decimal constructor comes with a guarantee of exactness. It accepts int, str and float (and tuple) and creates a Decimal object having the exact value of the int or float or the exact value that results from a decimal interpretation of the str. This exactness is not mandated by any of the standards on which the decimal module is based but I think it is a good thing. I strongly oppose a change that would have it use heuristics for interpreting other numeric types. >Perhaps the default Decimal context being set for a much higher > precision makes it philosophically unacceptable? The context is ignored by the Decimal constructor which will always create an exact value. If you want the created value rounded to context use unary +: >>> from decimal import Decimal as D >>> D(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') >>> +D(0.1) Decimal('0.1000000000000000055511151231') Oscar From guido at python.org Sat Mar 8 01:10:34 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 16:10:34 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> Message-ID: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>wrote: > On 7 March 2014 19:20, Guido van Rossum <guido at python.org> wrote: > > > > Maybe we can fix the conversion between Decimal and float (if this is > really > > all that matters to Mark, as it appears to be from his last email -- I'd > > already written most of the above before it arrived). Could it be as > simple > > as converting the float to a string using repr()? Given the smarts in the > > float repr() that should fix the examples Mark complained about. Are > there > > any roadblocks in the implementation or assumptions of the Decimal type > > here? > > The current Decimal constructor comes with a guarantee of exactness. > But Decimal(<float>) is relatively new (it was an error in 2.6). I know it's annoying to change it again, but I also have the feeling that there's not much use in producing a Decimal with 53 (or more?) *decimal digits* of precision from a float with 53 *bits* -- at least not by default. Maybe we can relent on this guarantee and make the constructor by default use fewer digits (e.g. using the same algorithm used by repr(<float>)) and have an optional argument to produce the full precision? Or reserve Decimal.from_float() for that? > It accepts int, str and float (and tuple) and creates a Decimal object > having the exact value of the int or float or the exact value that > results from a decimal interpretation of the str. This exactness is > not mandated by any of the standards on which the decimal module is > based but I think it is a good thing. I strongly oppose a change that > would have it use heuristics for interpreting other numeric types. > Well, given that most floating point values are the result of operations that inherently rounded to 53 bits anyway, I'm not sure that this guarantee is useful. Sure, I agree that there should be *a* way to construct the exact Decimal for any IEEE 754 double (since it is always possible, 10 being a multiple of 2), but I'm not sure how important it is that this is the default constructor. > >Perhaps the default Decimal context being set for a much higher > > precision makes it philosophically unacceptable? > > The context is ignored by the Decimal constructor which will always > create an exact value. If you want the created value rounded to > context use unary +: > > >>> from decimal import Decimal as D > >>> D(0.1) > Decimal('0.1000000000000000055511151231257827021181583404541015625') > >>> +D(0.1) > Decimal('0.1000000000000000055511151231') I guess that's too late. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/fdc61c24/attachment-0001.html> From oscar.j.benjamin at gmail.com Sat Mar 8 01:27:42 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 8 Mar 2014 00:27:42 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> Message-ID: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote: > On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> > wrote: >> >> The current Decimal constructor comes with a guarantee of exactness. > > But Decimal(<float>) is relatively new (it was an error in 2.6). I know it's > annoying to change it again, but I also have the feeling that there's not > much use in producing a Decimal with 53 (or more?) *decimal digits* of > precision from a float with 53 *bits* -- at least not by default. Maybe we > can relent on this guarantee and make the constructor by default use fewer > digits (e.g. using the same algorithm used by repr(<float>)) and have an > optional argument to produce the full precision? Or reserve > Decimal.from_float() for that? I'd rather a TypeError than an inexact conversion. The current behaviour allows to control how rounding occurs. (I don't use the decimal module unless this kind of thing is important). >> It accepts int, str and float (and tuple) and creates a Decimal object >> having the exact value of the int or float or the exact value that >> results from a decimal interpretation of the str. This exactness is >> not mandated by any of the standards on which the decimal module is >> based but I think it is a good thing. I strongly oppose a change that >> would have it use heuristics for interpreting other numeric types. > > Well, given that most floating point values are the result of operations > that inherently rounded to 53 bits anyway, I'm not sure that this guarantee > is useful. Sure, I agree that there should be *a* way to construct the exact > Decimal for any IEEE 754 double (since it is always possible, 10 being a > multiple of 2), but I'm not sure how important it is that this is the > default constructor. It's hard to reason about the accuracy of calculations that use the decimal module. It's harder than with a fixed-width type because of the possible presence of higher-than-context precision Decimals. The invariant that the constructor is exact (or a TypeError) is of significant benefit when using Decimal for code that accepts inputs of different numeric types. Oscar From donald at stufft.io Sat Mar 8 01:48:46 2014 From: donald at stufft.io (Donald Stufft) Date: Fri, 7 Mar 2014 19:48:46 -0500 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> Message-ID: <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io> On Mar 7, 2014, at 12:17 PM, Guido van Rossum <guido at python.org> wrote: > If I had to do it over again I would *definitely* never make a time value "falsy". The datetime module was conceived long ago, when the dangers of falsy objects that weren't clearly trying to be "like numbers" or "like collections" weren't so clear. Today I would argue that times aren't enough "like numbers" to qualify. > > The only question left in my mind is how safe it is to change the current behavior, given that it's been documented and implemented for over a decade. If this was for Python 3.0 (or even 3.1) I would definitely vote for fixing it, damn the torpedoes. But given that we're talking about 3.5, I'm more hesitant. It's also not something for which a deprecation warning would work well. (When would it be issued? Whenever bool(<time>) is called? Or only when it's about to return False?) > > Still, my intuition tells me that it's pretty much a nuisance feature and few people have actually relied on it, so I'd be okay with fixing (and breaking!) this in 3.5, perhaps after a thorough search for how much the feature is actually relied upon and how legitimate or important those uses are. A search should *also* try to unearth code that is technically *broken* by the current behavior. I would expect there to be quite a lot of the latter kind, and very few of the former. Hrm, I?m not actually sure how to even achieve such a search. Due to Python?s dynamic nature it?s pretty hard to determine what type is being used in a boolean expression, if that data is even available (It could be coming from a database or some other data source). Do you have any idea how to do such a search? > > Fearing I have said nothing new, > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/8b351013/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/8b351013/attachment.sig> From guido at python.org Sat Mar 8 02:02:15 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 17:02:15 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> Message-ID: <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>wrote: > On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote: > > On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin < > oscar.j.benjamin at gmail.com> > > wrote: > >> > >> The current Decimal constructor comes with a guarantee of exactness. > > > > But Decimal(<float>) is relatively new (it was an error in 2.6). I know > it's > > annoying to change it again, but I also have the feeling that there's not > > much use in producing a Decimal with 53 (or more?) *decimal digits* of > > precision from a float with 53 *bits* -- at least not by default. Maybe > we > > can relent on this guarantee and make the constructor by default use > fewer > > digits (e.g. using the same algorithm used by repr(<float>)) and have an > > optional argument to produce the full precision? Or reserve > > Decimal.from_float() for that? > > I'd rather a TypeError than an inexact conversion. The current > behaviour allows to control how rounding occurs. (I don't use the > decimal module unless this kind of thing is important). > My proposal still gives you control (e.g. through a keyword argument, or by using Decimal.from_float() instead of the constructor). It just changes the default conversion. I have to admit I don't recall the discussion that led to accepting float arguments to Decimal(), so I don't know if the exactness guarantee was ever challenged or even discussed. For all other argument types (str, tuple of digits, Decimal) it makes intuitive sense to guarantee exactness, because the input is already in decimal form, so why would the caller pass in digits they don't care about. But for a float the caller has no such control -- I can call round(math.pi, 2) but the resulting float (taken at face value) is still equal to 3.140000000000000124344978758017532527446746826171875, even though just '3.14' is enough as input to float() to produce that exact value. And unfortunately the default precision for Decimal is much larger than for IEEE double, so using unary + does not get rid of those extraneous digits -- it produces Decimal('3.140000000000000124344978758'). So you really have to work hard to recover produce the intended Decimal('3.14') (or Decimal('3.140000000000000')). >> It accepts int, str and float (and tuple) and creates a Decimal object > >> having the exact value of the int or float or the exact value that > >> results from a decimal interpretation of the str. This exactness is > >> not mandated by any of the standards on which the decimal module is > >> based but I think it is a good thing. I strongly oppose a change that > >> would have it use heuristics for interpreting other numeric types. > > > > Well, given that most floating point values are the result of operations > > that inherently rounded to 53 bits anyway, I'm not sure that this > guarantee > > is useful. Sure, I agree that there should be *a* way to construct the > exact > > Decimal for any IEEE 754 double (since it is always possible, 10 being a > > multiple of 2), but I'm not sure how important it is that this is the > > default constructor. > > It's hard to reason about the accuracy of calculations that use the > decimal module. It's harder than with a fixed-width type because of > the possible presence of higher-than-context precision Decimals. The > invariant that the constructor is exact (or a TypeError) is of > significant benefit when using Decimal for code that accepts inputs of > different numeric types. Can you provide an example of this benefit? I can't seem to guess whether you are talking about passing Decimal instances into code that then just uses +, * etc., or whether you are talking about code that uses Decimal internally and takes arguments of different types (e.g. int, float, string). -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/4f1a5760/attachment-0001.html> From guido at python.org Sat Mar 8 02:03:39 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 17:03:39 -0800 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io> References: <201403051216.12392.shai@platonix.com> <201403051805.15363.shai@platonix.com> <CAPTjJmozAJOwSy-Wswtn5GyzKjqQO-ZOajqAd0rf3=WDp0_ObQ@mail.gmail.com> <201403051838.25342.shai@platonix.com> <CAP7h-xZZ-3UAXGxBS00sPV8XZwzh9bsvZVwODZtr+aH-StpCQw@mail.gmail.com> <5317BF90.9030901@btinternet.com> <CAExdVNn8pED3Woq+to4VVjr1GcgXAxS9HOsWQfSxX35f0UuBQg@mail.gmail.com> <E28EC2EE-4757-4056-AC3E-53857849C826@stufft.io> <CAExdVNnWMRyEZ+tFZC8FhHL5j9k1X-V36J2iegft02CO=BK2tQ@mail.gmail.com> <5318626F.5080203@btinternet.com> <20140306152936.3c990c46@anarchist.wooz.org> <53190A16.4020201@canterbury.ac.nz> <87bnxipslm.fsf@uwakimon.sk.tsukuba.ac.jp> <5319A4A0.10905@btinternet.com> <877g86ovho.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJ+iHh1gB5KK6vT5ofnohbm4M65PmuUM+2KOW0sivYF-yw@mail.gmail.com> <6C5612D4-3811-434E-B61F-42BF8EDF1A0A@stufft.io> Message-ID: <CAP7+vJJgQ0+0_kHNKMvu5ucs95XUEvUra5CFG4zRpziLqR7meQ@mail.gmail.com> On Fri, Mar 7, 2014 at 4:48 PM, Donald Stufft <donald at stufft.io> wrote: > > On Mar 7, 2014, at 12:17 PM, Guido van Rossum <guido at python.org> wrote: > > If I had to do it over again I would *definitely* never make a time value > "falsy". The datetime module was conceived long ago, when the dangers of > falsy objects that weren't clearly trying to be "like numbers" or "like > collections" weren't so clear. Today I would argue that times aren't enough > "like numbers" to qualify. > > The only question left in my mind is how safe it is to change the current > behavior, given that it's been documented and implemented for over a > decade. If this was for Python 3.0 (or even 3.1) I would definitely vote > for fixing it, damn the torpedoes. But given that we're talking about 3.5, > I'm more hesitant. It's also not something for which a deprecation warning > would work well. (When would it be issued? Whenever bool(<time>) is called? > Or only when it's about to return False?) > > Still, my intuition tells me that it's pretty much a nuisance feature and > few people have actually relied on it, so I'd be okay with fixing (and > breaking!) this in 3.5, perhaps after a thorough search for how much the > feature is actually relied upon and how legitimate or important those uses > are. A search should *also* try to unearth code that is technically > *broken* by the current behavior. I would expect there to be quite a lot of > the latter kind, and very few of the former. > > Hrm, I'm not actually sure how to even achieve such a search. Due to > Python's dynamic nature it's pretty hard to determine what type is being > used in a boolean expression, if that data is even available (It could be > coming from a database or some other data source). > > Do you have any idea how to do such a search? > I know someone at Sourcegraph who thinks he knows how to modify Sourcegraph to do it. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/67d53f0a/attachment.html> From steve at pearwood.info Sat Mar 8 02:05:26 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 12:05:26 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> Message-ID: <20140308010526.GV28804@ando> On Fri, Mar 07, 2014 at 11:01:01AM -0800, Mark H. Harris wrote: > Here is the main post for today, from me, having slept on this and > putting some things back into historical timeframe. Sometime before > Py2.7.x it was not possible to promote a binary float to a decimal > float; the system threw an exception (convert to string). > > First, that was the correct way to handle this issue from the > inception of decimal.Decimal. Throw an exception and let everyone know > that there is a problem; then let the user figure out how to solve it. No, it was the *conservative* way to handle the issue. By disallowing the feature, it was possible to add it later without breaking backwards compatibility. If it had been allowed from the beginning, it would have been impossible to remove the feature if it had turned out to be a mistake. > It is not possible to correctly promote a binary float to a decimal > float. The decision (in Py2.7?) was to NOT throw an exception and to > copy the binary float "exactly" into the decimal float. That's incorrect. The way Python converts between the two is the right way to do the conversion. Given a decimal d and a float f constructed from d, f is the closest possible float to d. And the same applies for conversions the other way around. (This may only apply on platforms with IEEE-754 semantics, but I think that's nearly all platforms CPython runs on these days.) > Second, that was a bad decision--really--IMHO. It was the wrong > decision for two reasons 1) it denies what is inherently poor about > binary floating point representation in the first place, What is inherently poor about binary floats is also poor for Decimal floats: limited precision, rounding errors, and violation of fundamental laws of the real number system. These problems, save one, apply equally to Decimal, just in different places with different numbers. The *sole* difference is that with Decimal there is no rounding between what the user writes as a decimal string "1.23456" and what the value generated is. (Decimal also includes a bunch of extra features, like control of the precision, which floats don't support, but they *could* be supported if there was enough interest to justify the work.) This benefit is mostly of use if you are dealing with (1) numerically naive users or (2) using Python as an interactive calculator. In the case of (1), numerically naive users are not the most important subset of users for Python. In fact, I would say that numerically sophisticated users (scientists, mostly) are far more important to the Python ecosystem, and they *do not want your suggested change*. If we forced Decimals on them, they would be around with flaming torches and pitchforks. Seriously, the numeric community would likely abandon Python (with much wailing and gnashing of teeth) if we forced this on them. In the case of (2), anyone who has used a common pocket calculator is used to seeing values calculated with a string of 9s or 0s at the end: 2.01000000000001 2.00999999999999 instead of the expected 2.01. In fact, even on advanced scientific calculators which use decimal floats internally, such as the TI-89, we still *frequently* witness this problem. It seems to me that the average numerically naive user has simply learned that "calculators and computers do approximate calculations" (which is actually correct, when compared to the mathematically exact result), and don't particularly care too much about it. But if *you* care, that's okay. Python is a programming language. You can *program* it to make an interactive calculator with whatever properties you desire, including using Decimal floats exclusively. Python's done 97% of the work for you: the Decimal module, and the cmd module which makes building interactive command-line oriented applications easy. You can even simulate the Python interactive interpreter. In other words, if you don't want to convert from floats to Decimal, there is *absolutely no reason* why you should. > and 2) further compounds the problem by denying what is the benefit of > using Decimal floating point in the first place. I must admit I'm not clear as to what you think is the benefit of Decimal which is being denied. > Not to mention, it gives the wrong answer; from a decimal floating > point perspective it gives a VERY wrong answer. Now that's simply not true. The relative error between the "right" answer: py> from decimal import Decimal as D py> a = D("2.01").sqrt()**2 and the "wrong" answer: py> b = D(2.01).sqrt()**2 is *miniscule*, just one part in ten thousand million million: py> (a-b)/a Decimal('1.060511545905472636815920399E-16') [...] > What needs to happen is that binary floats need to be "correctly" > promoted to decimal floats as appropriate. This must not happen by > simple copying (that does not work). If I have given you the impression that float to Decimal conversion occurs by copying the bits from one into the other, I apologise, that was not my intention. That is not what happens. What actually happens is that given any float, not just floats that are typed in by hand by the user, are converted to the closest possible Decimal. What you appear to want is for Python to inspect the value of a float like 2.0099999999999997868 (the actual float you get from typing 2.01) and intelligently decide that what you *actually* wanted is the decimal 2.01. Even if this intelligence was possible, it has one serious flaw that cripples the whole exercise. It cannot apply only to numbers typed in by the user, but would have to apply across the board. Python cannot distinguish between these two cases: Decimal(1.8703152) # Give me this precise decimal and x = 1519.63802016624 # result of some intermediate calculation y = 812.5037 # result of another intermediate calculation z = x/y # yet another intermediate calculation Decimal(z) # convert from float to nearest decimal But z is the binary float 1.8703152, and Decimal cannot do one thing in the first case and a different thing in the second, because it cannot see where the number came from, only what it is. It does not and can not see the literal string, only the float. In the first case, you want 1.8703152 because that's what the user typed, but in the second case, we *must* return the result: Decimal('1.8703152000000000665380639475188218057155609130859375') because that's the closest conversion, the one with the least error. Rounding it to 7 decimal places may be sufficient for some applications, but that's not Python's decision to make, any more than Python can decide that if you type 1.99999998 that you must have meant exactly 2. > There needs to be some policy in place that will correctly "set" the > decimal float value based on intelligence http://www.catb.org/jargon/html/D/DWIM.html -- Steven From steve at pearwood.info Sat Mar 8 02:36:05 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 12:36:05 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> Message-ID: <20140308013605.GW28804@ando> On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote: > On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin > <oscar.j.benjamin at gmail.com>wrote: > > > On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote: [...] > > > I also have the feeling that there's not > > > much use in producing a Decimal with 53 (or more?) *decimal digits* of > > > precision from a float with 53 *bits* -- at least not by default. Maybe > > > we can relent on this guarantee and make the constructor by default use > > > fewer digits (e.g. using the same algorithm used by repr(<float>)) and > > > have an optional argument to produce the full precision? Or reserve > > > Decimal.from_float() for that? > > > > I'd rather a TypeError than an inexact conversion. The current > > behaviour allows to control how rounding occurs. (I don't use the > > decimal module unless this kind of thing is important). Like Oscar, I would not like to see this change. > My proposal still gives you control (e.g. through a keyword argument, or by > using Decimal.from_float() instead of the constructor). It just changes the > default conversion. Who is this default conversion meant to benefit? I think that anyone passing floats to Decimal ought to know what they are doing, and be quite capable of controlling the rounding via the decimal context, or via an intermediate string: value = Decimal("%.3f" % some_float) I believe that Mark wants to use Python as an interactive calculator, and wants decimal-by-default semantics without having to type quotes around his decimal numbers. I suspect that for the amount of time and effort put into this discussion, he probably could have written an interactive calculator using Decimals with the cmd module in half the time :-) It seems to me that Decimal should not try to guess what precision the user wants for any particular number. Mark is fond of the example of 2.01, and wants to see precisely 2.01, but that assumes that some human being typed in those three digits. If it's the result of some intermediate calculation, there is no reason to believe that a decimal with the value 2.01 exactly is better than one with the value 2.00999999999999978... Note that the issue is more than just Decimal: py> from fractions import Fraction py> Fraction(2.01) Fraction(1131529406376837, 562949953421312) Even more so than for Decimal, I will go to the barricades to defend this exact conversion. Although perhaps Fraction could grow an extra argument to limit the denominator as a short-cut for this: py> Fraction(2.01).limit_denominator(10000) Fraction(201, 100) I think it is important that Decimal and Fraction perform conversions as exactly as possible by default. You can always throw precision away afterwards, but you can't recover what was never saved in the first place. [...] > And unfortunately the default precision for Decimal is much larger than for > IEEE double, so using unary + does not get rid of those extraneous digits > -- it produces Decimal('3.140000000000000124344978758'). So you really have > to work hard to recover produce the intended Decimal('3.14') (or > Decimal('3.140000000000000')). I wouldn't object to Decimal gaining a keyword argument for precision, although it is easy to use an intermediate string. In other words, Decimal(math.pi, prec=2) could be the same as Decimal("%.2f" % math.pi) But what would prec do for non-float arguments? Perhaps we should leave Decimal() alone and put any extra bells and whistles that apply only to floats into the from_float() method. -- Steven From guido at python.org Sat Mar 8 03:02:02 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 18:02:02 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308010526.GV28804@ando> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> Message-ID: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano <steve at pearwood.info> wrote: > [...] The way Python converts between the two is the right > way to do the conversion. It's *exact*. I don't know we all agree it is the *right* way. > Given a decimal d and a float f constructed > from d, f is the closest possible float to d. And the same applies for > conversions the other way around. > It's actually stronger the other way around: when d is constricted from f, d is *equal* to the mathematical value of f. The issue (as I see it) is that there are many different decimals d that all convert to the same float f (because of rounding). The d that is constructed by taking the exact value of f is gross overkill. If d was constructed from f by always rounding to (I think) 17 digits it would still back convert exactly to f; the new float repr() gives us an even "better" (i.e. shorter) string of digits that are still guaranteed to to be converted into exactly f (because there's simply no other float that is closer to d than f). (This may only apply on platforms with IEEE-754 semantics, but I think > that's nearly all platforms CPython runs on these days.) > Sure. > > Second, that was a bad decision--really--IMHO. It was the wrong > > decision for two reasons 1) it denies what is inherently poor about > > binary floating point representation in the first place, > > What is inherently poor about binary floats is also poor for Decimal > floats: limited precision, rounding errors, and violation of fundamental > laws of the real number system. These problems, save one, apply equally > to Decimal, just in different places with different numbers. > > The *sole* difference is that with Decimal there is no rounding between > what the user writes as a decimal string "1.23456" and what the value > generated is. (Decimal also includes a bunch of extra features, like > control of the precision, which floats don't support, but they *could* > be supported if there was enough interest to justify the work.) > > This benefit is mostly of use if you are dealing with (1) numerically > naive users or (2) using Python as an interactive calculator. > > In the case of (1), numerically naive users are not the most important > subset of users for Python. In fact, I would say that numerically > sophisticated users (scientists, mostly) are far more important to the > Python ecosystem, and they *do not want your suggested change*. What is the "suggested change" here? If it's "default float literals to Decimal" I agree. But if the suggestion is my "Decimal(<float>) should be implemented as Decimal(repr(<float>))" I don't think most scientists care (few of them use Decimal, they stay entirely in the realm of binary floating point). > If we > forced Decimals on them, they would be around with flaming torches and > pitchforks. Seriously, the numeric community would likely abandon Python > (with much wailing and gnashing of teeth) if we forced this on them. > Right. In Mark's "post of the day" he already accepts that "decimal by default" is untenable -- and my claim is that even if it was desirable, it would be too big an undertaking. So it's dead. > In the case of (2), anyone who has used a common pocket calculator is > used to seeing values calculated with a string of 9s or 0s at the end: > > 2.01000000000001 > 2.00999999999999 > > instead of the expected 2.01. But the cause is calculations, like 1/3 ==> 0.333333333 and then multiply by 3 ==> 0.999999999. If you enter 2.1, a pocket calculater displays 2.1. > In fact, even on advanced scientific > calculators which use decimal floats internally, such as the TI-89, we > still *frequently* witness this problem. I suspect that if Python exhibited the problem in exactly those cases where a TI-89 exhibits it, few people would complain. The complaints would most likely come from numerically *sophisticated* users who object against using decimal (since binary has much better rounding behavior for arithmetic operations). > It seems to me that the average > numerically naive user has simply learned that "calculators and > computers do approximate calculations" (which is actually correct, when > compared to the mathematically exact result), and don't particularly > care too much about it. > And yet I have seen plenty of Tweets, emails and other public discussions where people gleefully pointed out that "Python has a bug." I think I've also seen tracker items created for such issues. > But if *you* care, that's okay. Python is a programming language. You > can *program* it to make an interactive calculator with whatever > properties you desire, including using Decimal floats exclusively. > That sounds a bit condescending. :-( Python's done 97% of the work for you: the Decimal module, and the cmd > module which makes building interactive command-line oriented > applications easy. You can even simulate the Python interactive > interpreter. > > In other words, if you don't want to convert from floats to Decimal, > there is *absolutely no reason* why you should. > > > > and 2) further compounds the problem by denying what is the benefit of > > using Decimal floating point in the first place. > > I must admit I'm not clear as to what you think is the benefit of > Decimal which is being denied. > If I may venture a guess, I think the assumed benefit of Decimal here is that it doesn't shatter naive users' expectations. > > Not to mention, it gives the wrong answer; from a decimal floating > > point perspective it gives a VERY wrong answer. > > Now that's simply not true. The relative error between the "right" > answer: > > py> from decimal import Decimal as D > py> a = D("2.01").sqrt()**2 > > and the "wrong" answer: > > py> b = D(2.01).sqrt()**2 > > is *miniscule*, just one part in ten thousand million million: > > py> (a-b)/a > Decimal('1.060511545905472636815920399E-16') > This feels like unnecessary pedantry. (Not the first time in this thread. :-( ) > [...] > > What needs to happen is that binary floats need to be "correctly" > > promoted to decimal floats as appropriate. This must not happen by > > simple copying (that does not work). > > If I have given you the impression that float to Decimal conversion > occurs by copying the bits from one into the other, I apologise, that > was not my intention. That is not what happens. > > What actually happens is that given any float, not just floats that are > typed in by hand by the user, are converted to the closest possible > Decimal. Well, actually, a Decimal that is mathematically *equal*. (I understand that this is technically the "closest possible" but the use of the latter phrase suggests that it's not always *equal*, which the actual algorithm used always produces a value that is mathematically equal to the input float.) > What you appear to want is for Python to inspect the value of a > float like 2.0099999999999997868 (the actual float you get from typing > 2.01) and intelligently decide that what you *actually* wanted is the > decimal 2.01. > > Even if this intelligence was possible, it has one serious flaw that > cripples the whole exercise. It cannot apply only to numbers typed in by > the user, but would have to apply across the board. Python cannot > distinguish between these two cases: > > Decimal(1.8703152) # Give me this precise decimal > > and > > x = 1519.63802016624 # result of some intermediate calculation > y = 812.5037 # result of another intermediate calculation > z = x/y # yet another intermediate calculation > Decimal(z) # convert from float to nearest decimal > > But z is the binary float 1.8703152, and Decimal cannot do one thing in > the first case and a different thing in the second, because it cannot > see where the number came from, only what it is. It does not and can > not see the literal string, only the float. > > In the first case, you want 1.8703152 because that's what the user > typed, but in the second case, we *must* return the result: > > Decimal('1.8703152000000000665380639475188218057155609130859375') > > because that's the closest conversion, the one with the least error. > Steven, that's actually not true. print(z) produces 1.8703152, and if I enter float(Decimal('1.8703152')) I get the exact same value of z. Try it. (In Python 2.7 or 3, please.) So I claim that if Decimal(z) produced the same value as Decimal('1.8703152') nothing would be lost. > Rounding it to 7 decimal places may be sufficient for some applications, > but that's not Python's decision to make, any more than Python can > decide that if you type 1.99999998 that you must have meant exactly 2. > The repr() function does not round to a fixed number of decimals. It produces (in theory, dynamically, although I suspect that the current algorithm is better) the shortest decimal string that, when converted back to binary, equals *exactly* the input. > > > There needs to be some policy in place that will correctly "set" the > > decimal float value based on intelligence > > http://www.catb.org/jargon/html/D/DWIM.html Also condescending. :-( -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/b7bfde77/attachment.html> From rosuav at gmail.com Sat Mar 8 03:10:11 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Mar 2014 13:10:11 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> Message-ID: <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote: > The repr() function does not round to a fixed number of decimals. It > produces (in theory, dynamically, although I suspect that the current > algorithm is better) the shortest decimal string that, when converted back > to binary, equals *exactly* the input. If that's the definition of a float's repr, then I'd support using that by default for the construction of a Decimal from a float. You can always use .as_integer_ratio() to get the exact stored representation. ChrisA From guido at python.org Sat Mar 8 03:15:11 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 18:15:11 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308013605.GW28804@ando> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> Message-ID: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> On Fri, Mar 7, 2014 at 5:36 PM, Steven D'Aprano <steve at pearwood.info> wrote: > On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote: > > On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin > > <oscar.j.benjamin at gmail.com>wrote: > > > > > On 8 March 2014 00:10, Guido van Rossum <guido at python.org> wrote: > [...] > > > > > I also have the feeling that there's not > > > > much use in producing a Decimal with 53 (or more?) *decimal digits* > of > > > > precision from a float with 53 *bits* -- at least not by default. > Maybe > > > > we can relent on this guarantee and make the constructor by default > use > > > > fewer digits (e.g. using the same algorithm used by repr(<float>)) > and > > > > have an optional argument to produce the full precision? Or reserve > > > > Decimal.from_float() for that? > > > > > > I'd rather a TypeError than an inexact conversion. The current > > > behaviour allows to control how rounding occurs. (I don't use the > > > decimal module unless this kind of thing is important). > > Like Oscar, I would not like to see this change. > Can you argue your position? > > My proposal still gives you control (e.g. through a keyword argument, > or by > > using Decimal.from_float() instead of the constructor). It just changes > the > > default conversion. > > Who is this default conversion meant to benefit? > > I think that anyone passing floats to Decimal ought to know what they > are doing, I think that is provably not the case. (That they don't know -- we can still argue about whether they ought to know.) > and be quite capable of controlling the rounding via the > decimal context, or via an intermediate string: > > value = Decimal("%.3f" % some_float) > Or, Decimal(repr(some_float)), which DWIM. > I believe that Mark wants to use Python as an interactive calculator, > and wants decimal-by-default semantics without having to type quotes > around his decimal numbers. I suspect that for the amount of time and > effort put into this discussion, he probably could have written an > interactive calculator using Decimals with the cmd module in half the > time :-) > Mark doesn't like it when we try to guess what he means or believes, but my guess is that he's actually trying to argue on behalf of Python users who would benefit from using Decimal but don't want to be bothered with a lot of rules. Telling such people "use the Decimal class" is much easier than telling them "use the Decimal class, and always put quotes around your numbers" (plus the latter advice is still very approximate, so the real version would be even longer and more confusing). > It seems to me that Decimal should not try to guess what precision the > user wants for any particular number. Mark is fond of the example of > 2.01, and wants to see precisely 2.01, but that assumes that some human > being typed in those three digits. If it's the result of some > intermediate calculation, there is no reason to believe that a decimal > with the value 2.01 exactly is better than one with the value > 2.00999999999999978... > Given the effort that went into making repr(<float>) do the right thing I think that you're dismissing a strawman here. The real proposal does not require guessing intent -- it just uses an existing, superior conversion from float to decimal that is already an integral part of the language. And it uses some static information that Decimal currently completely ignores, i.e. all Python floats (being IEEE doubles) are created with 53 bits of precision. > Note that the issue is more than just Decimal: > > py> from fractions import Fraction > py> Fraction(2.01) > Fraction(1131529406376837, 562949953421312) > > Even more so than for Decimal, I will go to the barricades to defend > this exact conversion. Although perhaps Fraction could grow an extra > argument to limit the denominator as a short-cut for this: > > py> Fraction(2.01).limit_denominator(10000) > Fraction(201, 100) > An inferior solution. Anyway, nobody cares about Fraction. > I think it is important that Decimal and Fraction perform conversions as > exactly as possible by default. Again, why produce 56 *digits* of precision by default for something that only intrinsically has 53 *bits*? > You can always throw precision away > afterwards, but you can't recover what was never saved in the first > place. > There should definitely be *some* API to convert a float to the exact mathematical Decimal. But I don't think it necessarily needs to be the default constructor. (It would be different if Decimal was a fixed-point type. But it isn't.) > [...] > > And unfortunately the default precision for Decimal is much larger than > for > > IEEE double, so using unary + does not get rid of those extraneous digits > > -- it produces Decimal('3.140000000000000124344978758'). So you really > have > > to work hard to recover produce the intended Decimal('3.14') (or > > Decimal('3.140000000000000')). > > I wouldn't object to Decimal gaining a keyword argument for precision, > although it is easy to use an intermediate string. In other words, > > Decimal(math.pi, prec=2) > > could be the same as > > Decimal("%.2f" % math.pi) > This is actually another strawman. Nobody in this thread has asked for a way to truncate significant digits like that. It's the *insignificant* digits that bother some. The question is, what should Decimal(math.pi) do. It currently returns Decimal('3.141592653589793115997963468544185161590576171875'), which helps noone except people interested in how floats are represented internally. > But what would prec do for non-float arguments? Perhaps we should leave > Decimal() alone and put any extra bells and whistles that apply only to > floats into the from_float() method. > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/9b127260/attachment.html> From guido at python.org Sat Mar 8 03:19:14 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 18:19:14 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> Message-ID: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote: > > The repr() function does not round to a fixed number of decimals. It > > produces (in theory, dynamically, although I suspect that the current > > algorithm is better) the shortest decimal string that, when converted > back > > to binary, equals *exactly* the input. > > If that's the definition of a float's repr, then I'd support using > that by default for the construction of a Decimal from a float. You > can always use .as_integer_ratio() to get the exact stored > representation. > Sadly I can't point to exactly where this is documented, but since Python 2.7 and some early version of 3.x that is indeed how repr(<float>) works. (And str(<float>) too -- before 2.7, str() would give only 12 digits while repr() gave 17, but the new repr() makes such shenanigans unnecessary. So I could have saved myself a few characters by using str() instead of repr() everywhere in this thread. :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/c2f7661c/attachment.html> From rosuav at gmail.com Sat Mar 8 03:46:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Mar 2014 13:46:14 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> Message-ID: <CAPTjJmobvscTGCyiuK=+eSFB-9VRDCMo03vy3y+evSAEPKLCgQ@mail.gmail.com> On Sat, Mar 8, 2014 at 1:19 PM, Guido van Rossum <guido at python.org> wrote: > On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com> wrote: >> >> On Sat, Mar 8, 2014 at 1:02 PM, Guido van Rossum <guido at python.org> wrote: >> > The repr() function does not round to a fixed number of decimals. It >> > produces (in theory, dynamically, although I suspect that the current >> > algorithm is better) the shortest decimal string that, when converted >> > back >> > to binary, equals *exactly* the input. >> >> If that's the definition of a float's repr, then I'd support using >> that by default for the construction of a Decimal from a float. You >> can always use .as_integer_ratio() to get the exact stored >> representation. > > Sadly I can't point to exactly where this is documented, but since Python > 2.7 and some early version of 3.x that is indeed how repr(<float>) works. I couldn't find it anywhere - help(float.__repr__) just points to it being the implementation of repr(some_float), and poking around on the docs didn't show up any explanation. It'd be something worth mentioning somewhere, as that's a great feature, but I don't really know where, heh. ChrisA From eric at trueblade.com Sat Mar 8 03:50:22 2014 From: eric at trueblade.com (Eric V. Smith) Date: Fri, 07 Mar 2014 21:50:22 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> Message-ID: <531A856E.1020207@trueblade.com> On 3/7/2014 9:19 PM, Guido van Rossum wrote: > On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico <rosuav at gmail.com > <mailto:rosuav at gmail.com>> wrote: > If that's the definition of a float's repr, then I'd support using > that by default for the construction of a Decimal from a float. You > can always use .as_integer_ratio() to get the exact stored > representation. > > > Sadly I can't point to exactly where this is documented, but since > Python 2.7 and some early version of 3.x that is indeed how > repr(<float>) works. It's mentioned at the bottom of this page: http://docs.python.org/2/tutorial/floatingpoint.html But oddly the wording in http://docs.python.org/3/tutorial/floatingpoint.html is not as precise. There are some weasel words in there because we do have a fallback if we can't run Gay's code on some systems (sys.float_repr_style='legacy'). But I think for all practical purposes it's always true. > (And str(<float>) too -- before 2.7, str() would give only 12 digits > while repr() gave 17, but the new repr() makes such shenanigans > unnecessary. So I could have saved myself a few characters by using > str() instead of repr() everywhere in this thread. :-) Indeed! Eric. From nad at acm.org Sat Mar 8 03:51:35 2014 From: nad at acm.org (Ned Deily) Date: Fri, 07 Mar 2014 18:51:35 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <CAPTjJmq+q1pVY-+1Rzm37MbAOpnB5aU40Cc5tnpdiZj_M28prQ@mail.gmail.com> <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw@mail.gmail.com> Message-ID: <nad-0F1CB1.18513507032014@news.gmane.org> In article <CAP7+vJKFYy+RGbryMSWzXaExZajL1zUTmiWPdnPeG5RpkmRHiw at mail.gmail.com>, Guido van Rossum <guido at python.org> wrote: > On Fri, Mar 7, 2014 at 6:10 PM, Chris Angelico > <rosuav at gmail.com> wrote: > > If that's the definition of a float's repr, then I'd support using > > that by default for the construction of a Decimal from a float. You > > can always use .as_integer_ratio() to get the exact stored > > representation. > Sadly I can't point to exactly where this is documented, but since Python > 2.7 and some early version of 3.x that is indeed how repr(<float>) works. If nowhere else: http://docs.python.org/3/whatsnew/3.1.html#other-language-changes http://bugs.python.org/issue1580 http://docs.python.org/2/whatsnew/2.7.html#other-language-changes http://bugs.python.org/issue7117 -- Ned Deily, nad at acm.org From random832 at fastmail.us Sat Mar 8 04:24:04 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 07 Mar 2014 22:24:04 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> Message-ID: <1394249044.23276.91986741.2D83D0A2@webmail.messagingengine.com> On Fri, Mar 7, 2014, at 16:24, Andrew Barnert wrote: > There is no such thing as copying a binary float exactly into a decimal > float. Decimal float values are not a superset of binary float values; in > either direction, you have to round. Which is what Python 2.7+ does. Actually, arbitrary-precision decimal floats are a superset of binary floats, because 2 is a factor of 10. As the binary floats get smaller you need more and more digits to represent the decimal though. >>> decimal.Decimal(1/2**1074) Decimal('4.940656458412465441765687928682213723650598026143247644255856825006755072702087518652998363616359923797965646954457177309266567103559397963987747960107818781263007131903114045278458171678489821036887186360569987307230500063874091535649843873124733972731696151400317153853980741262385655911710266585566867681870395603106249319452715914924553293054565444011274801297099995419319894090804165633245247571478690147267801593552386115501348035264934720193790268107107491703332226844753335720832431936092382893458368060106011506169809753078342277318329247904982524730776375927247874656084778203734469699533647017972677717585125660551199131504891101451037862738167250955837389733598993664809941164205702637090279242767544565229087538682506419718265533447265625E-324') From stephen at xemacs.org Sat Mar 8 04:46:50 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 08 Mar 2014 12:46:50 +0900 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <87r46fpk8w.fsf@uwakimon.sk.tsukuba.ac.jp> <1394225284.11877.91899617.542E02EC@webmail.messagingengine.com> Message-ID: <871tydpd1x.fsf@uwakimon.sk.tsukuba.ac.jp> random832 at fastmail.us writes: > Or, for that matter, you could limit both the numerator and the > denominator to some maximum value - when an intermediate result exceeds > it (won't be more than the square of the maximum for most operations), > you find the closest representable value. Knuth calls that "fixed slash" (actually somewhat more general than I think you mean -- numerator and denominator can have different maximum values). From paul.dubois at gmail.com Sat Mar 8 04:55:50 2014 From: paul.dubois at gmail.com (Paul Du Bois) Date: Fri, 7 Mar 2014 19:55:50 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> Message-ID: <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org> wrote: > Or, Decimal(repr(some_float)), which DWIM. > Because I haven't seen anyone else bring this up, using a non-exact conversion breaks the invariant "Decimal(f)==f". There are so many pairs of numeric types that break that invariant that it might not be a big deal; but in all other cases the invariant is broken because it is theoretically impossible. from decimal import Decimal f = 2.2 print(Decimal(f)==f) print(Decimal(repr(f))==f) p -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/4f52a9e9/attachment.html> From harrismh777 at gmail.com Sat Mar 8 04:59:11 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 19:59:11 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> Message-ID: <6cf3dba9-ec77-431e-ade6-383b2d09ff80@googlegroups.com> On Friday, March 7, 2014 8:15:11 PM UTC-6, Guido van Rossum wrote: > > > . . . but my guess is that he's actually trying to argue on behalf of > Python users who would benefit from using Decimal but don't want to be > bothered with a lot of rules. Telling such people "use the Decimal class" > is much easier than telling them "use the Decimal class, and always put > quotes around your numbers" (plus the latter advice is still very > approximate, so the real version would be even longer and more confusing). > > Yes, that is correct. Although, I would modify "with certain rules". I mean, its ok to expect some rules (I think that's what we mean by semantic). But rules that seem to go against the grain of the normal human experience ... like quoting a numerical input. I mean if you tell me as a programmer that's what I have to do, well that's what I have to do. On the other hand (the hand I'm really concerned about) if you tell a naive user they have to quote their numerical inputs-- that seems like the wrong approach. Yes, its much better to say, "Please use the Decimal module". marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/20a8d222/attachment-0001.html> From rosuav at gmail.com Sat Mar 8 05:00:05 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Mar 2014 15:00:05 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> Message-ID: <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois <paul.dubois at gmail.com> wrote: > On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org> wrote: >> >> Or, Decimal(repr(some_float)), which DWIM. > > > Because I haven't seen anyone else bring this up, using a non-exact > conversion breaks the invariant "Decimal(f)==f". > > There are so many pairs of numeric types that break that invariant that it > might not be a big deal; but in all other cases the invariant is broken > because it is theoretically impossible. > > from decimal import Decimal > f = 2.2 > print(Decimal(f)==f) > print(Decimal(repr(f))==f) How is Decimal==float implemented? Is it by calling Decimal(rhs) and then comparing? If so, changing how Decimal(float) works won't break the invariant, as it'll make the same conversion each time. ChrisA From guido at python.org Sat Mar 8 05:11:26 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 20:11:26 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> Message-ID: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> On Fri, Mar 7, 2014 at 8:00 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois <paul.dubois at gmail.com> > wrote: > > On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum <guido at python.org> > wrote: > >> > >> Or, Decimal(repr(some_float)), which DWIM. > > > Because I haven't seen anyone else bring this up, using a non-exact > > conversion breaks the invariant "Decimal(f)==f". > > > > There are so many pairs of numeric types that break that invariant that > it > > might not be a big deal; but in all other cases the invariant is broken > > because it is theoretically impossible. > > > > from decimal import Decimal > > f = 2.2 > > print(Decimal(f)==f) > > print(Decimal(repr(f))==f) > > How is Decimal==float implemented? Is it by calling Decimal(rhs) and > then comparing? If so, changing how Decimal(float) works won't break > the invariant, as it'll make the same conversion each time. > There's a special function to convert the other operand for the purpose of comparisons, and it currently uses Decimal.from_float(). I am not (yet :-) proposing to change the behavior of that function -- it is and has always been the API function that does exact float-to-Decimal conversion. Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/2fb78557/attachment.html> From rosuav at gmail.com Sat Mar 8 05:16:06 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Mar 2014 15:16:06 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> Message-ID: <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com> On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum <guido at python.org> wrote: >> How is Decimal==float implemented? Is it by calling Decimal(rhs) and >> then comparing? If so, changing how Decimal(float) works won't break >> the invariant, as it'll make the same conversion each time. > > > There's a special function to convert the other operand for the purpose of > comparisons, and it currently uses Decimal.from_float(). I am not (yet :-) > proposing to change the behavior of that function -- it is and has always > been the API function that does exact float-to-Decimal conversion. > Decimal(repr(f)) == f returns False today (for the majority of float values > anyway) and under my proposal Decimal(f) == f would also return False. I > don't (yet :-) think that's a deal breaker. Maybe not a deal breaker, but certainly another panel on the bikeshed. It would make good sense to retain that invariant by making the comparison use repr just like the constructor. ChrisA From guido at python.org Sat Mar 8 05:38:54 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 20:38:54 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com> Message-ID: <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com> On Fri, Mar 7, 2014 at 8:16 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum <guido at python.org> wrote: > >> How is Decimal==float implemented? Is it by calling Decimal(rhs) and > >> then comparing? If so, changing how Decimal(float) works won't break > >> the invariant, as it'll make the same conversion each time. > > > There's a special function to convert the other operand for the purpose > of > > comparisons, and it currently uses Decimal.from_float(). I am not (yet > :-) > > proposing to change the behavior of that function -- it is and has always > > been the API function that does exact float-to-Decimal conversion. > > Decimal(repr(f)) == f returns False today (for the majority of float > values > > anyway) and under my proposal Decimal(f) == f would also return False. I > > don't (yet :-) think that's a deal breaker. > > Maybe not a deal breaker, but certainly another panel on the bikeshed. > It would make good sense to retain that invariant by making the > comparison use repr just like the constructor. > Not sure I agree. I find the invariant Decimal(x) == x rather uninteresting. I find the idea that Decimal.__eq__ compares the exact mathematical values very appealing. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/901ba17b/attachment.html> From tim.peters at gmail.com Sat Mar 8 05:39:59 2014 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 7 Mar 2014 22:39:59 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> Message-ID: <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> [Guido] > Decimal(repr(f)) == f returns False today (for the majority of float values > anyway) and under my proposal Decimal(f) == f would also return False. I > don't (yet :-) think that's a deal breaker. To the contrary, that's "a feature". Current Pythons take equality testing seriously, striving to return True when & only when values are mathematically ("as if using infinite precision") identical, even when that's inconvenient to do: >>> 2**500 == 2.0**500 True >>> 2**500 + 1 == 2.0**500 False If information is lost when converting, it's doing nobody a favor for "==" to pretend it hasn't been lost. So long as Decimal.from_float(f) == f is still True, then equality would be working as expected in all cases. From guido at python.org Sat Mar 8 05:40:49 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 20:40:49 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAPTjJmqN=5Oxb+71=p-wjTGxo5H4tTPuJ1QCE2w49RJxHb3C9Q@mail.gmail.com> <CAP7+vJKKONYsHvo2438CSXxTWo8bPUCVWWJ2S5ce7hW6QxGhqw@mail.gmail.com> Message-ID: <CAP7+vJ+a7xq3-+ZPWRfqQ8TNRR_r8AhaOud3PfMpZYtooLVCrw@mail.gmail.com> Sometime today I think I wrote that the C implementation of Decimal isn't yet in the stdlib. I stand corrected. It is. Yay! -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/3cf9c040/attachment.html> From guido at python.org Sat Mar 8 05:44:18 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 20:44:18 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> Message-ID: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> On Fri, Mar 7, 2014 at 8:39 PM, Tim Peters <tim.peters at gmail.com> wrote: > [Guido] > > Decimal(repr(f)) == f returns False today (for the majority of float > values > > anyway) and under my proposal Decimal(f) == f would also return False. I > > don't (yet :-) think that's a deal breaker. > > To the contrary, that's "a feature". Current Pythons take equality > testing seriously, striving to return True when & only when values are > mathematically ("as if using infinite precision") identical, even when > that's inconvenient to do: > > >>> 2**500 == 2.0**500 > True > >>> 2**500 + 1 == 2.0**500 > False > > If information is lost when converting, it's doing nobody a favor for > "==" to pretend it hasn't been lost. So long as Decimal.from_float(f) > == f is still True, then equality would be working as expected in all > cases. > I agree. But what do you think of my main proposal? I would retract it if you advised me so. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/58ffd2d3/attachment.html> From tim.peters at gmail.com Sat Mar 8 06:01:07 2014 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 7 Mar 2014 23:01:07 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> Message-ID: <CAExdVNkAdkeNdD4JPne12z=DPGmptT6Bqyr5hDeQwo0pqnTR-Q@mail.gmail.com> [Guido] > I agree. But what do you think of my main proposal? I would retract it if > you advised me so. Oh, I stick to tiny detail these days, to avoid getting sucked into endless pointless arguments ;-) That said, I agree that the default Decimal constructor acting like Decimal(x) == Decimal(repr(x)) would be far less surprising to the vast majority of users, and no problem at all for "expert" users (provided Decimal.from_float(x) were still available). Against it, the change would invalidate dozens of stackoverflow answers advising users to try printing Decimal(x) to see where their naive binary floating-point expectations are going wrong. And I have no feel for how much production code may be relying on current Decimal(x) behavior (I have none). But, if we're willing to risk changing the meaning of bool(datetime,time(arguments creating an object in some timezones - but not all - that converts to midnight UTC )), then I suppose the floodgates are waaaaaay open now ;-) From abarnert at yahoo.com Sat Mar 8 06:33:38 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 7 Mar 2014 21:33:38 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> Message-ID: <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com> On Mar 7, 2014, at 20:44, Guido van Rossum <guido at python.org> wrote: > I agree. But what do you think of my main proposal? I would retract it if you advised me so. I think this may be a good time to raise my points from off-list, in hopes that someone can say "those potential problems are not real problems anyone would ever care about". Today, given any real, Decimal(float('real')) always gives you the value of the closest binary float to that real. With this change, it will sometimes give you the value of the second-closest repr-of-a-binary-float to that real. This means the error across any range of reals increases. It's still below the rule-of-thumb cutoff that everyone uses for converting through floats, but it is higher by a nonzero amount that doesn't cancel out. Similarly, today, the distribution of float values across the real number line is... not uniform (because of exponents), and I don't know the right technical term, you know what it looks like. The distribution of repr-of-float variables is not. Do users of Decimal (or, rather, users of Decimal(float) conversion) care about either of those issues? I don't know. I've never written any code that converted floats to Decimals and then used them as anything other than low-precision fixed-point values (like dollars and cents). By the way, to dismiss any potential performance worries: with both 3.3 and trunk, Decimal(repr(f)) is actually faster than Decimal.from_float(f) by a factor of 1.3 to 3.2 in a variety of quick tests. If that weren't true, you might have to define Decimal(f) as if it were building and then parsing a string but them implement more complicated code that feeds digits from one algorithm to the other, but fortunately, it looks like this is a three-line change. From harrismh777 at gmail.com Sat Mar 8 06:40:47 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 21:40:47 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> Message-ID: <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> On Friday, March 7, 2014 3:24:06 PM UTC-6, Andrew Barnert wrote: > The decision was discussed at the time, and all the pros and cons were > hashed out. If you're not willing to read that discussion, your opinion > that the change was a mistake is worth exactly as much as that of any > individual user who asked for the change. > hi Andrew, I have been studying the python-ideas archive & the python-dev archive all night. I have read hundreds of posts. I am finding something very interesting. My proposal has been coming up (time and again) in different flavors for many years; with all the same people participating (with all of the very same discussion almost verbatim). Just for history sake, I thought you might be interested in a blast from the past from Raymond Hettinger in response to Lennart Benschop who made the decimal literal proposal in Oct, 2007: Just for historical context only: On Oct 26, 1:54 am, Lennart Benschop <[hidden email]<http://python.6.x6.nabble.com/user/SendEmail.jtp?type=node&node=1580823&i=0>> wrote: > My proposal: > - Any decimal constant suffixed with the letter "D" or "d" will be > interpreted as a literal of the Decimal type. This also goes for > decimal constants with exponential notation. There's nothing new here that hasn't already been proposed and discussed on python-dev. There were no major objections to the idea; however, it will need to wait until there is a good C implementation of the decimal module (which is in the works but coming along very, very slowly). {from the history department} -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/92054f2d/attachment-0001.html> From steve at pearwood.info Sat Mar 8 07:08:30 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 17:08:30 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> Message-ID: <20140308060830.GX28804@ando> On Fri, Mar 07, 2014 at 09:40:47PM -0800, Mark H. Harris wrote: > hi Andrew, I have been studying the python-ideas archive & the > python-dev archive all night. I have read hundreds of posts. I am > finding something very interesting. My proposal has been coming up > (time and again) in different flavors for many years; with all the > same people participating (with all of the very same discussion almost > verbatim). Which proposal? You have made a few, and I cannot keep track of which ones you still stand by and which ones you have abandoned. - that Python unify all numeric types to one "Python Number" (I think this is abandoned) - that Decimal(float) use AI to determine what number the programmer wanted (you've agreed to stop using the term "AI", but I'm not entirely sure whether you've moved away from this position entirely -- you still describe Python as needing to "intelligently" convert floats) - that allowing Decimal(float) at all is a mistake and should be prohibited - that the default numeric type when the programmer enters a numeral with a decimal point like 2.01 be Decimal and possibly others. Which proposal are you referring to here? > Just for history sake, I thought you might be interested in a blast from > the past from Raymond > Hettinger in response to Lennart Benschop who made the decimal literal > proposal in Oct, 2007: [snip] Would you mind providing a URL for that message so we can see the context please? -- Steven From tjreedy at udel.edu Sat Mar 8 07:14:46 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 08 Mar 2014 01:14:46 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> Message-ID: <lfecgo$609$1@ger.gmane.org> On 3/7/2014 2:20 PM, Guido van Rossum wrote: > However, for the growing contingent of scientists who use Python as a > replacement for Matlab (not Mathematica!), it could be a big nuisance. > They don't care about decimal issues (they actually like binary better) One reason knowledgeable users of floating point numbers as approximations like binary better is that the binary floating point system is much 'smoother' than the decimal floating point system. For example, with 3 decimal digits, consider .999, 1.00, 1.01. The first difference is .001, the second is .01, 10 x larger. This is one of the problems of working with base 10 slide rules. For binary, the largest ratio between differences is 2 rather than 10. > and they write lots of C++ code that interfaces between CPython's > internal API and various C++ libraries for numeric data processing, all > of which use IEEE binary. This too ;-). -- Terry Jan Reedy From abarnert at yahoo.com Sat Mar 8 07:18:53 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 7 Mar 2014 22:18:53 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308060830.GX28804@ando> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> Message-ID: <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> From: Steven D'Aprano <steve at pearwood.info> Sent: Friday, March 7, 2014 10:08 PM > On Fri, Mar 07, 2014 at 09:40:47PM -0800, Mark H. Harris wrote: > >> hi Andrew, I have been studying the python-ideas archive & the >> python-dev archive all night. I have read hundreds of posts. I am >> finding something very interesting. My proposal has been coming up >> (time and again) in different flavors for many years; with all the >> same people participating (with all of the very same discussion almost >> verbatim). > > Which proposal? You have made a few, and I cannot keep track of which > ones you still stand by and which ones you have abandoned. I agree that some clarity would be nice here, but I think from the quote he quoted it's pretty clear that?he's talking about the decimal-literal proposal, not any of the ones that you mentioned. > - that Python unify all numeric types to one "Python? > Number" (I think this is abandoned) > > - that Decimal(float) use AI to determine what number the programmer > wanted (you've agreed to stop using the term "AI", but I'm not > entirely > sure whether you've moved away from this position entirely -- you > still describe Python as needing to "intelligently" convert floats) I'm still not sure exactly what Mark's proposal is here, but Guido has developed this one into something concrete and feasible: change the default conversion from float to Decimal (that is, Decimal(float)) to use the repr. I don't want to put works in Mark's mouth and say that's what he's interested in here, but I can say that it looks like it would solve the problem cases Mark has brought up. > - that allowing Decimal(float) at all is a mistake and should be > prohibited > > - that the default numeric type when the programmer enters a numeral > with a decimal point like 2.01 be Decimal From steve at pearwood.info Sat Mar 8 07:36:04 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 17:36:04 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com> References: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com> Message-ID: <20140308063604.GY28804@ando> On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote: > Today, given any real, Decimal(float('real')) always gives you the > value of the closest binary float to that real. With this change, it > will sometimes give you the value of the second-closest > repr-of-a-binary-float to that real. Please don't talk about "reals". "Real" has a technical meaning in mathematics, and no computer number system is able to represent the reals. In Python, we have floats, which are C doubles, and Decimals. Putting that aside, I'm afraid I don't understand how Guido's suggestion to use repr() in the Decimal constructor will sometimes give the second-closest value. Can you explain in more detail? If you can show an example, that would be great too. > This means the error across any > range of reals increases. It's still below the rule-of-thumb cutoff > that everyone uses for converting through floats, but it is higher by > a nonzero amount that doesn't cancel out. > > Similarly, today, the distribution of float values across the real > number line is... not uniform (because of exponents), and I don't know > the right technical term, you know what it looks like. The > distribution of repr-of-float variables is not. Nope, sorry, I don't get you. I think that what you are trying to say is that for each binary exponent, the floats with that same exponent are distributed uniformly: |...|...|...|...|...|...|...|...| but their reprs are not: |...|..|..|....|...|....|..|....| Is this what you mean? -- Steven From guido at python.org Sat Mar 8 07:42:20 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2014 22:42:20 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <CAP7+vJLX8EJn7o+ivMgF1kwTQhwXw4=dBEji6Aq8fNsXqvC2YQ@mail.gmail.com> <1E332207-7BA6-4739-8B62-A01D1008885B@yahoo.com> <CAP7+vJJ7Qd0LgLmC8RJLdijt=XSpk068eEGd=QkJYwAYa50ZYw@mail.gmail.com> <CAP7+vJKayzZfKBTN1=PQ2+92xTREuErpaMKU7FVUMh47EY=nYA@mail.gmail.com> <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com> Message-ID: <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com> [CC back to the list because you posted the same argument there but without the numerical example, and my working through that might help others understand your point] On Fri, Mar 7, 2014 at 9:18 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > The main point I'm getting at is that by rounding 0.100000000000000012 to > 0.1 instead of 0.10000000000000000555..., You're no longer rounding it to > the nearest binary float, but instead to the second nearest > Decimal(repr(binary float)) (since 0.10000000000000002 is closer than > 0.1). > OK, let me walk through that carefully. Let's name the exact mathematical values and assign them to strings: >>> a = '0.100000000000000012' >>> b = '0.1000000000000000055511151231257827021181583404541015625' >>> c = '0.10000000000000002' Today, Decimal(float(a)) == Decimal(b). Under my proposal, Decimal(float(a)) == Decimal('0.1'). The difference between float('0.1') and float(c) is 1 ulp (2**-56), and a is between those, but closer to c; but it is even closer to b (in the other direction). IOW for the mathematical values, 0.1 < b < a < c, where a is closer to b than to c, So if the choices for rounding a would be b or c, b is preferred. So far so good. (And still good if we replace c with the slightly smaller exact value of float(c).) And your point is that if we change the allowable choices to '0.1' or c, we find that float(b) == float('0.1'), but a is closer to c than to 0.1. This is less than 1 ulp, but more than 0.5 ulp. I find the argument intriguing, but I blame it more on what happens in float(a) than in what Decimal() does to the resulting value. If you actually had the string a, and wanted to convert it to Decimal, you would obviously write Decimal(a), not Decimal(float(a)), so this is really only a problem when someone uses a as a literal in a program that is passed to Decimal, i.e. Decimal(0.100000000000000012). That's slightly unfortunate, but easy to fix by adding quotes. The only place where I think something like this might occur in real life is when someone copies a numerical recipe involving some very precise constants, and mindlessly applies Decimal() without string quotes to the constants. But that's a "recipe" for failure anyway, since if the recipe really uses more precision than IEEE double can handle, *with* the quotes the recipe would be calculated more exactly anyway. Perhaps another scenario would be if the constant was calculated (by the recipe-maker) within 0.5 ulp using IEEE double and rendered with exactly the right number of digits. But these scenarios sound like either they should use the quotes anyway, or the calculation would be better off done in double rather than Decimal. So I think it's still pretty much a phantom problem. > Of course that's not true for all reals (0.1 being the obvious > counterexample), but it's true for some with your proposal, while today > it's true for none. So the mean absolute error in Decimal(repr(f)) across > any range of reals is inherently higher than Decimal.from_float(f). Put > another way, you're adding additional rounding error. That additional > rounding error is still less than the rule-of-thumb cutoff that people use > when talking about going through float, but it's nonzero and not guaranteed > to cancel out. > > On top of that, the distribution of binary floats is uniform (well, more > complicated than uniform because they have an exponent as well as a > mantissa, but you know what I mean); the distribution of closest-repr > values to binary floats is not. > > I have no idea whether either of these are properties that users of > Decimal (or, rather, Decimal and float together) care about. But they are > properties that Decimal(float) has today that would be lost. > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/993daf51/attachment-0001.html> From harrismh777 at gmail.com Sat Mar 8 07:47:59 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 22:47:59 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com> Message-ID: <3eb20953-682b-412b-8ea5-bb56a024ade5@googlegroups.com> On Saturday, March 8, 2014 12:40:36 AM UTC-6, Mark H. Harris wrote: Steve, Andrew, here: http://osdir.com/ml/python.ideas/2008-01/msg00079.html Mr Toronto was crazy like me... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/7e5c37ad/attachment.html> From harrismh777 at gmail.com Sat Mar 8 07:40:36 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 22:40:36 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308060830.GX28804@ando> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> Message-ID: <33fb0113-9577-407c-989f-b4e1f57eeb7b@googlegroups.com> On Saturday, March 8, 2014 12:08:30 AM UTC-6, Steven D'Aprano wrote: > Which proposal are you referring to here? > hi Steven, well, all of them. The archive has discussions pertaining to every one of the discussions we have had on this list. From "python has a bug," to python needs a decimal literal, to "how come this Decimal(.01) doesn't work," to (believe this or not) one person on the ideas archive actually proposed that "all python numbers be unified," I kid you not. Guido put that one to bed easily with one post and there was no further discussion of the topic, but the idea is that I'm not the first one to think about these things. The discussions are pretty much the same as the ones we've had here on this list with people frustrated for pretty much the same reasons. ( some of it ignorance, some naivet?, some over-zealous, one or two crazy like me). Go here: http://python.6.x6.nabble.com/Python-python-dev-f1801473.html and search for D'Aprano decimal or D'Aprano Decimal or search on unify python numbers The discussions are very interesting, very educational, and very entertaining. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/b15a141c/attachment.html> From harrismh777 at gmail.com Sat Mar 8 07:52:32 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 22:52:32 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com> On Saturday, March 8, 2014 12:18:53 AM UTC-6, Andrew Barnert wrote: > but Guido has developed this one into something concrete and feasible: > change the default conversion from float to Decimal (that is, > Decimal(float)) to use the repr. > What you and Guido are discussing is perfect. Thank you. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/53478a2a/attachment.html> From harrismh777 at gmail.com Sat Mar 8 08:06:47 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 23:06:47 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308060830.GX28804@ando> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> Message-ID: <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com> On Saturday, March 8, 2014 12:08:30 AM UTC-6, Steven D'Aprano wrote: > Which proposal? > hi Steve, the reading I would like to find, and can't, is the discussion that led to the change in behavior in Py2.7; where convert from float stopped giving the (convert to string message). I am very interested in reading that discussion. Do you have a link to that? Thanks marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/eb429934/attachment.html> From steve at pearwood.info Sat Mar 8 08:08:40 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 18:08:40 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> Message-ID: <20140308070840.GZ28804@ando> On Fri, Mar 07, 2014 at 06:02:02PM -0800, Guido van Rossum wrote: > On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano <steve at pearwood.info> wrote: > > > [...] The way Python converts between the two is the right > > way to do the conversion. > > > It's *exact*. I don't know we all agree it is the *right* way. Fair point. I can't argue with that :-) > > Given a decimal d and a float f constructed > > from d, f is the closest possible float to d. And the same applies for > > conversions the other way around. > > > > It's actually stronger the other way around: when d is constricted from f, > d is *equal* to the mathematical value of f. Ah, so it is. > The issue (as I see it) is that there are many different decimals d that > all convert to the same float f (because of rounding). The d that is > constructed by taking the exact value of f is gross overkill. Decimal -> float is many-to-one: more than one Decimal will round to a single float. But float -> Decimal is always one-to-one, I think, regardless of whether you use the current exact conversion or repr first. The crux of the matter is whether or not it is overkill for Decimal to use the exact value. Correct me if I'm wrong, but I think we agree that there ought to be a way to convert floats exactly to Decimal, we just disagree on whether that ought to be spelled Decimal(x) or Decimal.from_float(x). Likewise I think we agree that there ought to be some way to convert floats to the nearest "simple" Decimal, with the question being whether that ought to be spelled Decimal(repr(x)) or Decimal(x). If we're all in agreement that this is the only serious area of disagreement for a change which has any hope of appearing in 3.5, then we can put aside the more speculative proposals -- Steven From steve at pearwood.info Sat Mar 8 08:35:11 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Mar 2014 18:35:11 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com> References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <a2004233-0ed3-47a8-8261-b5495922492f@googlegroups.com> Message-ID: <20140308073511.GA28804@ando> On Fri, Mar 07, 2014 at 11:06:47PM -0800, Mark H. Harris wrote: > hi Steve, the reading I would like to find, and can't, is the > discussion that led to the change in behavior in Py2.7; where convert > from float stopped giving the (convert to string message). > > I am very interested in reading that discussion. Do you have a link to > that? http://bugs.python.org/issue8257 https://mail.python.org/pipermail/python-dev/2010-March/098699.html and in particular: https://mail.python.org/pipermail/python-dev/2010-March/098717.html -- Steven From harrismh777 at gmail.com Sat Mar 8 08:50:38 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Fri, 7 Mar 2014 23:50:38 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com> Message-ID: <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com> On Saturday, March 8, 2014 12:52:32 AM UTC-6, Mark H. Harris wrote: > but Guido has developed this one into something concrete and feasible: >> change the default conversion from float to Decimal (that is, >> Decimal(float)) to use the repr. >> > > What you and Guido are discussing is perfect. Thank you. > Guido, Andrew, I have a question about repr() in my current codes for 3.3.4 and below: Would it be reasonable to add a function to pdeclib to be used internally like this: def Dec(numform): return Decimal(repr(numform)) Does this work correctly (for pdeclib internally) ? I think this works better than my first stab at this (just using str() ) because repr() acts like an eval() ?? Do I understand this correctly? It looks like what you are proposing will fix the problem I've been whining about. Thank you. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140307/804b56a7/attachment.html> From abarnert at yahoo.com Sat Mar 8 08:53:40 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 7 Mar 2014 23:53:40 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308063604.GY28804@ando> References: <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> <6DFCFB6D-46F9-47E9-A7D1-203C5022E2B6@yahoo.com> <20140308063604.GY28804@ando> Message-ID: <1394265220.1728.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Steven D'Aprano <steve at pearwood.info> Sent: Friday, March 7, 2014 10:36 PM > On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote: > >> Today, given any real, Decimal(float('real')) always gives you the >> value of the closest binary float to that real. With this change, it >> will sometimes give you the value of the second-closest >> repr-of-a-binary-float to that real. > > Please don't talk about "reals". "Real" has a technical > meaning in > mathematics, and no computer number system is able to represent the > reals. In Python, we have floats, which are C doubles, and Decimals. I am using "real" in the technical sense. That's the whole crux of the problem: you want to use the real[1] number 0.1 in Python, and you can't, because there is no IEEE double that matches that value. If we ignore reals and only talk about floats, then 0.1 and?0.100000000000000012 are the same number, so there is no problem. ? ? [1]: I suppose we could talk about rationals instead of reals here, because (a) you can't enter irrationals as literals, and (b) (b) Guido's proposal obviously doesn't attempt to help with them. I don't think that makes a difference here, but I could be wrong; if I am, please point out below where this leads me astray. Put another way: This cannot be just about converting IEEE double floats to Decimal floats, because Python already does that perfectly. It's about recovering information lost in representing real numbers (or, if you prefer, rational numbers, or Python literals) as IEEE double floats, when converting them to Decimal floats, by taking advantage of the fact that (a)?we know that humans tend to use numbers like 0.1 more often than numbers like?0.100000000000000005551, and (b)?we know the precision of IEEE double and can guarantee that we're not losing any real information. > Putting that aside, I'm afraid I don't understand how Guido's > suggestion > to use repr() in the Decimal constructor will sometimes give the > second-closest value. Can you explain in more detail? If you can show an > example, that would be great too. OK, take the number?0.100000000000000012. The closest IEEE double to this number is?0.1000000000000000055511151231257827021181583404541015625. The next closest is 0.10000000000000001942890293094023945741355419158935546875. Today's design gives you the former when you write Decimal(0.100000000000000012). You get the closest one in this case, and in every case. The closest repr of an IEEE double to this number is?0.10000000000000002. But with Guido's proposal, the one you get is 0.1, which is farther from the number you wanted. So, if you treat evaluate-literal-as-float-then-Decimal.from_float as a (mathematical) function, it has the property that it always gives you the closest value from its range to your input. If you treat evaluate-literal-as-float-then-repr-then-Decimal as a function, it does not have that property. >> This means the error across any? >> range of reals increases. It's still below the rule-of-thumb cutoff >> that everyone uses for converting through floats, but it is higher by >> a nonzero amount that doesn't cancel out. Was this second point clear? In case it wasn't, let me expand on it: In the example above,?Guido's proposal gives you nearly double the error (1.2e-17 vs. 6.4+e-18) of the current design. Of course there are also cases where it gives a lower error, 0.1 being an obvious example. I'm pretty sure (but I don't know how to prove?) that if you integrate the absolute error over any sufficiently large[1] set of reals, it's higher in Guido's proposal than in the current design. ? ? [1]: The "sufficiently large" there is just so you don't choose a set so small that all of its elements map to a single IEEE double, like { 0.1 <= x <?0.100000000000000001 }. >> Similarly, today, the distribution of float values across the real? >> number line is... not uniform (because of exponents), and I don't know >> the right technical term, you know what it looks like. The >> distribution of repr-of-float variables is not. > > Nope, sorry, I don't get you. > > I think that what you are trying to say is that for each binary > exponent, the floats with that same exponent are distributed uniformly: > > |...|...|...|...|...|...|...|...| > > but their reprs are not: > > |...|..|..|....|...|....|..|....| > > Is this what you mean? That's part of it.?But on top of that, I think (but again can't prove) that the reprs are on average skewed further to the left than the right, which means that, e.g., the average of a collection of reprs-of-floats will also be skewed in that direction. So, those are my three points. Remember that I don't know if any of them actually matter to anyone, and in fact am hoping they do not. From abarnert at yahoo.com Sat Mar 8 09:16:26 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 00:16:26 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com> <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com> Message-ID: <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com> From: Mark H. Harris <harrismh777 at gmail.com> Sent: Friday, March 7, 2014 11:50 PM >On Saturday, March 8, 2014 12:52:32 AM UTC-6, Mark H. Harris wrote: >Guido, Andrew, ? I have a question about repr() in my current codes for ?3.3.4 and below: > >Would it be reasonable to add a function to pdeclib to be used internally like this: > > >def Dec(numform): >? ? ? ? return Decimal(repr(numform)) Yes.?If you know that repr(numform) is the string decimal representation that you want for numform, then Decimal(repr(numform)) is guaranteed to be the?decimal?floating-point value that you want. So if numform is a float, in Python 3.3, given your examples, then this is exactly what you want. >Does this work correctly (for pdeclib internally) ? ?I think this works better than my first >stab at this (just using str() ) ?because repr() acts like an eval() ?? I think you're a little confused here. First, in 3.3, for float, repr and str return the same thing. You may be working with other types for which that isn't true?but in that case, you need to figure out which one gives you the decimal representation you want, and use that one, whether it's repr or str (or something else). Second, repr() doesn't act like an eval(). It's sort of an _inverse_ of eval, for some types.[1] For example,?the repr of Decimal('0.1') is the string "Decimal('0.1')", and eval("Decimal('0.1')") will give you Decimal('0.1'). ? ? ? [1] Please don't rely on this. First, it's not true for all types. Second, even if you restrict yourself to types where it is true, repr is not the best way to exchange or persist values, and eval leads to dangerous and fragile code. But this is very useful for things like playing with Python at the interactive interpreter. >It looks like what you are proposing will fix the problem I've been whining about. ?Thank you. I think you may be better off using an explicit repr, even if numform is guaranteed to be a float. It makes your intentions clear: you want the one and only Decimal value that matches the decimal string representation of numform, not just arbitrarily any of the many Decimal values that can round-trip back to the float numform, right? Of course if at all possible, the best thing to do is to avoid creating floats in the first place. Use Decimal('0.1') for constants in your code, use Decimal(user_input) rather than converting user_input with float or eval and then trying to recover the lost information later, etc. But obviously there are cases where you can't do this, because you've, e.g., received the number as a 64-bit IEEE double over the wire from some other program you don't control. That's where Guido's proposal would help you. From abarnert at yahoo.com Sat Mar 8 09:32:33 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 00:32:33 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <CAP7+vJLX8EJn7o+ivMgF1kwTQhwXw4=dBEji6Aq8fNsXqvC2YQ@mail.gmail.com> <1E332207-7BA6-4739-8B62-A01D1008885B@yahoo.com> <CAP7+vJJ7Qd0LgLmC8RJLdijt=XSpk068eEGd=QkJYwAYa50ZYw@mail.gmail.com> <CAP7+vJKayzZfKBTN1=PQ2+92xTREuErpaMKU7FVUMh47EY=nYA@mail.gmail.com> <486FC9F4-FDA9-45B3-A96C-DD496515A470@yahoo.com> <CAP7+vJ+SAu0h7J7AWoDJCw2ARYjg7o1-cv=Hd7=GJfZXqir8+A@mail.gmail.com> Message-ID: <1394267553.65888.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: Guido van Rossum <guido at python.org> Sent: Friday, March 7, 2014 10:42 PM >[CC back to the list because you posted the same argument there but without the numerical example, and my working through that might help others understand your point] Thank you for presenting my point better than I could; it's a lot clearer this way. >On Fri, Mar 7, 2014 at 9:18 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > >The main point I'm getting at is that by rounding?0.100000000000000012 to 0.1 instead of?0.10000000000000000555..., You're no longer rounding it to the nearest binary float, but instead to the second nearest Decimal(repr(binary float)) (since?0.10000000000000002 is closer than 0.1). > >OK, let me walk through that carefully. Let's name the exact mathematical values and assign them to strings: > >>>> a = '0.100000000000000012' >>>> b = '0.1000000000000000055511151231257827021181583404541015625' >>>> c = '0.10000000000000002' > >Today, Decimal(float(a)) == Decimal(b). Under my proposal, Decimal(float(a)) == Decimal('0.1'). The difference between float('0.1') and float(c) is 1 ulp (2**-56), and a is between those, but closer to c; but it is even closer to b (in the other direction). IOW for the mathematical values, 0.1 < b < a < c, where a is closer to b than to c, So if the choices for rounding a would be b or c, b is preferred. So far so good. (And still good if we replace c with the slightly smaller exact value of float(c).) > >And your point is that if we change the allowable choices to '0.1' or c, we find that float(b) == float('0.1'), but a is closer to c than to 0.1. This is less than 1 ulp, but more than 0.5 ulp. Yes. It's the same two problems that inspired Clinger's correct rounding papers[1][2]: it does not have the closest-match property, and it can lose?almost twice as much accuracy. But the context is very different, so I'm not sure Clinger's arguments are relevant here. ? ? [1]:?http://citeseer.ist.psu.edu/william90how.html ? ? [2]:?ftp://ftp.ccs.neu.edu/pub/people/will/retrospective.pdf >I find the argument intriguing, but I blame it more on what happens in float(a) than in what Decimal() does to the resulting value. If you actually had the string a, and wanted to convert it to Decimal, you would obviously write Decimal(a), not Decimal(float(a)), so this is really only a problem when someone uses a as a literal in a program that is passed to Decimal, i.e. Decimal(0.100000000000000012). Agreed on both counts. However, the entire problem you're trying to solve here is caused by what happens in float(a). You're effectively attempting to recover the information lost in float() in Decimal(), in a way that often does what people want, and otherwise never does anything too bad. So long as giving up the correct-rounding property,?doubling the error (but still staying under 1 ulp), and skewing the distribution of Decimals created this way (by less than 0.5 ulp, but possibly in a way that can accumulate) are not "too bad", I believe your proposal succeeds completely. >That's slightly unfortunate, but easy to fix by adding quotes. Yes, but the motivating example to this whole thread, Decimal(1.1), is just as easy to fix by using quotes. I think I can see the distinction: Novices don't know to use quotes; people trying to implement numerical recipes in Python do (or at least really, really should); therefore a change that helps the former but hurts the latter, when they both leave off the quotes, is a net gain. Yes? From dickinsm at gmail.com Sat Mar 8 10:43:58 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 8 Mar 2014 09:43:58 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> Message-ID: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> On Sat, Mar 8, 2014 at 12:10 AM, Guido van Rossum <guido at python.org> wrote: > > But Decimal(<float>) is relatively new (it was an error in 2.6). I know > it's annoying to change it again, but I also have the feeling that there's > not much use in producing a Decimal with 53 (or more?) *decimal digits* of > precision from a float with 53 *bits* -- at least not by default. Maybe we > can relent on this guarantee and make the constructor by default use fewer > digits (e.g. using the same algorithm used by repr(<float>)) and have an > optional argument to produce the full precision? Or reserve > Decimal.from_float() for that? > Using the same algorithm as used by repr(float) feels like a bad idea to me: my gut feeling is that a conversion from float to Decimal is a fundamental operation that should be easily describable in simple terms, and should not involve the level of complication and 'magic' involved in the float repr. (That level of magic seems fine for the repr, because in most cases you don't actually care too much what string you get so long as it evaluates back to the correct float; you're not doing arithmetic with the result.) IEEE 754-2008 requires simply that conversions between numbers in different formats round correctly. It specifies a "convertFormat" operation, described as follows: """ If the conversion is to a format in a different radix or to a narrower precision in the same radix, the result shall be rounded as specified in Clause 4. Conversion to a format with the same radix but wider precision and range is always exact. ... """ ... where clause 4 is the one that outlines the general rounding rules that apply to almost all IEEE 754 operations. I see three sane options for float to Decimal conversion: 1. Raise an exception. 2. Round to the nearest Decimal value using the current context for that round operation. 3. Do what we're currently doing, and do an exact conversion. Option 2 would seem closest to what IEEE 754 specifies. To augment option 2, we could also make it easier to create Decimal contexts corresponding to the IEEE 754 recommended interchange formats (decimal32, decimal64, decimal128). This was already proposed in http://bugs.python.org/issue8786. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/0f2441de/attachment.html> From dickinsm at gmail.com Sat Mar 8 11:01:47 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 8 Mar 2014 10:01:47 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> Message-ID: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > > I see three sane options for float to Decimal conversion: > > 1. Raise an exception. > 2. Round to the nearest Decimal value using the current context for that > round operation. > 3. Do what we're currently doing, and do an exact conversion. > Proposals for change should also take into consideration that Decimal already does *exact* conversions for integers (and I believe has done since it first existed). It would be quite surprising for `Decimal(2**1000)` and `Decimal(2.0**1000)` to be different numbers. If we change the float behaviour, we might also want to change conversion from int to round to the nearest Decimal using the current context. Again, that's closer to what IEEE 754 specifies for the "convertFromInt" operation. On the other hand, we probably shouldn't lend *too* much weight to IEEE 754, especially when talking about choice of precision. IEEE 754 isn't a perfect fit for Decimal: the IEEE standard is mostly concerned with fixed width decimal formats, which is subtly different from Mike Cowlishaw's approach of "extensible precision" where the precision is not so closely tied to the format. Python's decimal module is based on Cowlishaw's standard, not on IEEE 754. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/c3bff20b/attachment.html> From eric at trueblade.com Sat Mar 8 12:20:22 2014 From: eric at trueblade.com (Eric V. Smith) Date: Sat, 08 Mar 2014 06:20:22 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> References: <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAHVvXxRRFBjkK3Bh6bkdseOHSGUaxp_kXLECjUQLtYmW6+J-sw@mail.gmail.com> <CAP7+vJ+epvi2Motdpm0bXBwG9Nf7m6_p8RuAQMb30bQv192_PQ@mail.gmail.com> <20140308013605.GW28804@ando> <CAP7+vJJKGPzqgUtLBzfzR2zZD+4AO44-z789M0AD5UM-CJN=pw@mail.gmail.com> <CAJLbtS-+oasXD=-Ms5gQGD3Lax0-+V2rGJGe3g37=DHmsO8uBQ@mail.gmail.com> <CAPTjJmoqftY49YvZ8-t82CLJj5Fzb7cEO8boXWdLME4M6a12=w@mail.gmail.com> <CAP7+vJLfOHe9U2PP6gS_rh1eDAjwRRf9k7MY65X-DR8K7e8Fqg@mail.gmail.com> <CAExdVNn1WREOJo_FykqAp=wh1-aD97ZsDziNFASOFxrTTxSKrA@mail.gmail.com> <CAP7+vJKRoSjwVsyyM1Uvn_MQgSWeoeVGHPPvCEyPi0keg_S7+A@mail.gmail.com> Message-ID: <531AFCF6.5070901@trueblade.com> On 3/7/2014 11:44 PM, Guido van Rossum wrote: > I agree. But what do you think of my main proposal? I would retract it > if you advised me so. I'd like to hear what Mark Dickinson has to say. I assume like most people he lost interest in this thread. I've sent him a message asking him to take a look. Eric. From solipsis at pitrou.net Sat Mar 8 13:01:47 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 8 Mar 2014 13:01:47 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <lfecgo$609$1@ger.gmane.org> Message-ID: <20140308130147.1dcfbc74@fsol> On Sat, 08 Mar 2014 01:14:46 -0500 Terry Reedy <tjreedy at udel.edu> wrote: > On 3/7/2014 2:20 PM, Guido van Rossum wrote: > > > > However, for the growing contingent of scientists who use Python as a > > replacement for Matlab (not Mathematica!), it could be a big nuisance. > > They don't care about decimal issues (they actually like binary better) > > One reason knowledgeable users of floating point numbers as > approximations like binary better is that the binary floating point > system is much 'smoother' than the decimal floating point system. For > example, with 3 decimal digits, consider .999, 1.00, 1.01. The first > difference is .001, the second is .01, 10 x larger. This is one of the > problems of working with base 10 slide rules. For binary, the largest > ratio between differences is 2 rather than 10. Well, can you explain what difference it does in practice? Regards Antoine. From ron3200 at gmail.com Sat Mar 8 15:05:48 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Mar 2014 08:05:48 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> Message-ID: <lff83f$9ls$1@ger.gmane.org> On 03/07/2014 01:01 PM, Mark H. Harris wrote: > The point here is that binary floats should not be promoted to decimal > floats by using "exact" copy from binary float representation to decimal > float representation. In the case of writing out a number. Decimal(2.1), A decimal literal solves that case since no floats are involved. It would be the same as the exact copy, but that is perfectly expected as well. (and already works when a string is used in the constructor. Regarding not raising an error when a float is used directly in the decimal constructor: As wrong as it may seem at first, also consider python does not raise an error for this. >>> int(.1) 0 The users is expected to know that the result will not always equal to what is given. It's just really obvious in this case because, this is what this function has always done, and what we expect it to do. The decimal case isn't that different except that it isn't so obvious. What that means is the docs needs be adjusted to make it more easily findable... But currently... Help on class Decimal in module decimal: class Decimal(builtins.object) | Decimal(value="0", context=None): Construct a new Decimal object. | value can be an integer, string, tuple, or another Decimal object. | If no value is given, return Decimal('0'). The context does not affect | the conversion and is only passed to determine if the InvalidOperation | trap is active. | Which says nothing about using floats, and should if it is allowed. It does talk about floats in the from_float method including comments on exact representation. Help on built-in function from_float: from_float(...) method of builtins.type instance from_float(f) - Class method that converts a float to a decimal number, exactly. Since 0.1 is not exactly representable in binary floating point, Decimal.from_float(0.1) is not the same as Decimal('0.1'). >>> Decimal.from_float(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') >>> Decimal.from_float(float('nan')) Decimal('NaN') >>> Decimal.from_float(float('inf')) Decimal('Infinity') >>> Decimal.from_float(float('-inf')) Decimal('-Infinity') So I think the first case should say, it uses from_float() when a floating point is used, and explicitly also say to look at the from_float docs. (or better yet include them directly in the class doc string. As far as Ulps go. Unless the average error can be reduced I don't see any reason to change it. If the docs say it's uses from_float and from_float gives the same result. I think it's perfectly reasonable and likely to answer most users questions where they expect to find the answer. Cheers, Ron From dickinsm at gmail.com Sat Mar 8 15:15:59 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 8 Mar 2014 14:15:59 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308130147.1dcfbc74@fsol> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <lfecgo$609$1@ger.gmane.org> <20140308130147.1dcfbc74@fsol> Message-ID: <CAAu3qLUpoGB2SPOqZApvchqyr3jYFETOEtJ9SD+0QZ+qrvFLdw@mail.gmail.com> On Sat, Mar 8, 2014 at 12:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > problems of working with base 10 slide rules. For binary, the largest > > ratio between differences is 2 rather than 10. > > Well, can you explain what difference it does in practice? > Probably not much that the average user would care about, but there are a whole host of 'nice' properties that work in binary floating-point but not in decimal. For example, in binary, assuming IEEE 754, round-to-nearest, etc., you're guaranteed that for any two representable floats x and y, the "average" (x+y)/2 lies in the interval [x, y] (even strictly between x and y provided that x and y are at least 2 ulps apart), so a naively written floating-point bisection search will converge. In decimal that's not true: you can lose a whole digit of precision when adding x and y and end up with a result that's outside [x, y]. >>> from decimal import * >>> getcontext().prec = 3 >>> x = Decimal('0.516') >>> y = Decimal('0.518') >>> (x + y) / 2 Decimal('0.515') # ouch! Then if you're doing numerical analysis and error computations, the "wobble" (the variation in scale of the ratio between the mathematical relative error and the error expressed in ulps) is 2 for binary, 10 for decimal. That makes for weaker error bounds and faster-growing errors for operations done in decimal floating-point rather than binary. (And it's even worse for hexadecimal floating-point, which is why IEEE 754 is a big improvement over IBM's hex float format.) Binary is just better for serious numerical work. :-) Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/1b03b50c/attachment.html> From stefan at bytereef.org Sat Mar 8 15:18:20 2014 From: stefan at bytereef.org (Stefan Krah) Date: Sat, 8 Mar 2014 15:18:20 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> Message-ID: <20140308141820.GA20171@sleipnir.bytereef.org> Mark Dickinson <dickinsm at gmail.com> wrote: > I see three sane options for float to Decimal conversion: > > 1. Raise an exception. > 2. Round to the nearest Decimal value using the current context for that round > operation. > 3. Do what we're currently doing, and do an exact conversion. I agree that these are the main reasonable options. All functions in Cowlishaw's specification use all digits from overlong input operands, so there should be a convenient way of creating such operands exactly. I think we should not make an exeception for floats, unless we move into the direction of IEEE 754-2008 (but that surely is a different topic). Some general comments: The current model works well, since we already have a fallback constructor that rounds: >>> from decimal import * >>> context = getcontext() >>> Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') >>> context.create_decimal(1.1) Decimal('1.10') >>> If you want safety against accidental conversions, set FloatOperation: >>> context.traps[FloatOperation] = True >>> Decimal(1.1) Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.FloatOperation: [<class 'decimal.FloatOperation'>] If you want extra safety against *any* accidental float comparisons, check the flags manually. This is necessary, since equality operations cannot raise due to the fact that they are also used for membership testing: >>> context.clear_flags() >>> Decimal(9) in [9.0] True >>> if context.flags[FloatOperation]: ... raise FloatOperation ... Traceback (most recent call last): File "<stdin>", line 2, in <module> decimal.FloatOperation You see that the float operation is still recorded, even though it isn't raised automatically. Such a check could be done periodically or at the end of a program. To summarize, I think we should leave things as they are or turn on FloatOperation by default. Stefan Krah From harrismh777 at gmail.com Sat Mar 8 17:13:39 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 08:13:39 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <F9F7C761-65FF-470C-80B3-0EA16E22F01E@yahoo.com> <4ba01e98-29fd-4ac9-8ce7-673027cc26c5@googlegroups.com> <20140308060830.GX28804@ando> <1394259533.45359.YahooMailNeo@web181004.mail.ne1.yahoo.com> <3cccaf94-e8f4-45fb-81bb-8fc01b2f91e9@googlegroups.com> <db7ae43e-ffb3-4882-bdbe-d5dabd962a7a@googlegroups.com> <1394266586.80465.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <8541b8b9-95a4-4897-ac24-2ea0e6fa4184@googlegroups.com> On Saturday, March 8, 2014 2:16:26 AM UTC-6, Andrew Barnert wrote: > > I think you're a little confused here. > > repr() doesn't act like an eval(). It's sort of an _inverse_ of eval, for > some types.[1] For example, the repr of Decimal('0.1') is the string > "Decimal('0.1')", and eval("Decimal('0.1')") will give you Decimal('0.1'). > You are correct, I was confused about the inverse relationship, for some types, because the http doc says: For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), But, the system doc help() makes things more clear: repr(...) repr(object) -> string Return the canonical string representation of the object. For most object types, eval(repr(object)) == object. Thank you, your explanation of the caveats for repr() were helpful to me. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/5a735ce8/attachment.html> From guido at python.org Sat Mar 8 17:54:24 2014 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Mar 2014 08:54:24 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> Message-ID: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman. On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > >> >> I see three sane options for float to Decimal conversion: >> >> 1. Raise an exception. >> 2. Round to the nearest Decimal value using the current context for that >> round operation. >> 3. Do what we're currently doing, and do an exact conversion. >> > I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself an expert at all, but I do think I have something to bring to the table -- insight the experience of non-expert users. When a non-expert writes Decimal(1.1), each of the three above outcomes surprises. We know that (1) was unpopular, that's why we changed it. We now know that (3) is unpopular at least in some circles (Mark Harrison can't be the only one who doesn't like it). Changing to (2) wouldn't do much to address this, because the default context has way more precision than float, so it still shows a lot of extraneous digits. Yes, it's trivial to get rid of those extra digits, just use quotes. But if *my* proposal were adopted, it would be trivial for numerical experts to get the extra digits -- just use from_float(). At this point, my claim is that we're talking essentially about what is the better experience for most users, and while I am not much of a user of Decimal myself, I believe that my proposal has benefits more people and situations than it has downsides. > Proposals for change should also take into consideration that Decimal > already does *exact* conversions for integers (and I believe has done since > it first existed). It would be quite surprising for `Decimal(2**1000)` and > `Decimal(2.0**1000)` to be different numbers. > This feels like a strawman, since Decimal(2**1000 + 1) and Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot of digits but is one too small), and the similar examples with a large exponent (10000) differ dramatically (the float version raises an exception). For most purposes it's a fluke that 2.0**1000 can be represented exactly as a float, and the argument doesn't convince me at all. There are just too many odd examples like that, and it always will remain a minefield. > If we change the float behaviour, we might also want to change conversion > from int to round to the nearest Decimal using the current context. Again, > that's closer to what IEEE 754 specifies for the "convertFromInt" operation. > I don't think that's no the table. Python's int doesn't lose precision, and users know this and depend on it. (In fact, I find ignoring the current context in the constructor a totally reasonable approach -- it does this uniformly, regardless of the argument type. My proposal doesn't change this, the context would *still* not be used to decide how a float is converted. Only the fact that it's a float would.) > > On the other hand, we probably shouldn't lend *too* much weight to IEEE > 754, especially when talking about choice of precision. IEEE 754 isn't a > perfect fit for Decimal: the IEEE standard is mostly concerned with fixed > width decimal formats, which is subtly different from Mike Cowlishaw's > approach of "extensible precision" where the precision is not so closely > tied to the format. Python's decimal module is based on Cowlishaw's > standard, not on IEEE 754. > I wonder what Cowlishaw would say about our current discussion. He is also the father of Rexx... -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/a05e7e03/attachment.html> From skreft at gmail.com Sat Mar 8 17:57:45 2014 From: skreft at gmail.com (Sebastian Kreft) Date: Sat, 8 Mar 2014 17:57:45 +0100 Subject: [Python-ideas] Enhance exceptions by attaching some more information to them In-Reply-To: <530210C5.8000406@btinternet.com> References: <CAFuVyPkZ+0hPkTKsh4M7r6veCj5Dq0J7i1LL=V5DdESm1iQx9Q@mail.gmail.com> <530210C5.8000406@btinternet.com> Message-ID: <CAFuVyPkJxUti_HDH4EQsPRFkC_dYL7Q6kFF2ogPeoxCJLucyZg@mail.gmail.com> Thanks for the comments, I will start working on a PEP for Step 1. On Mon, Feb 17, 2014 at 2:38 PM, Rob Cliffe <rob.cliffe at btinternet.com>wrote: > +1 on the idea (I can't comment on the implementation). One of Python's > great strengths is its excellent error messages. But if they can be made > still better, that would be fantastic. > Rob Cliffe > > > On 15/02/2014 15:40, Sebastian Kreft wrote: > > More than once I've been in a situation where I wish that some of the > stdlib exceptions had a better message or some more information to help me > diagnose the problem. > > For example: > a = [1, 2, 3, 4, 5] > a[5] > IndexError: list index out of range > > In this case there's no reference to neither the length of the array nor > to the offending index. > > I'm of the idea that we could extend the exceptions by adding some more > information to them, so 3rd party libraries could use them for > debugging/logging. > > For example, one such use case would be to have a modified test runner, > that in case of exceptions automatically prints some debug information. > Another would be a logger system that in case of an exception also logs > some debug info that could be relevant to understand and solve the issue. > > I propose extending (at least) the following exceptions with the > following attributes: > KeyError: key, object > IndexError: index, object > AttributeError: attribute, object > NameError: name > > Of course that populating these attributes could be controlled by a flag. > > I know that some of this information is already present in some > exceptions, depending on the context. However, I propose adding these > attributes, as in this way a tool could easily and reliably extract the > information and work with it, as opposed to have to parse the huge variety > of different messages there are. > > For the first use case mentioned above I have a working prototype, > although it does not use this proposal, but a series of hacks (I'm > modifying the bytecode to save a reference to the key and object :() and > parsing of the exception messages. But I want to share what the output of > such a tool could be. > > ====================================================================== > ERROR: test_attribute (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 18, in > test_attribute > AttributeError: 'str' object has no attribute 'Lower'. Did you mean > 'islower', 'lower'? > Debug info: > Object: '' > Type: <type 'str'> > > ====================================================================== > ERROR: test_index (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 6, in > test_index > IndexError: list index out of range > Debug info: > Object: [1, 2] > Object len: 2 > Index: 2 > > ====================================================================== > ERROR: test_key (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 10, in > test_key > KeyError_: 'fooo', did you mean 'foo'? > Debug info: > Object: {'foo': 1} > Key: 'fooo' > > ====================================================================== > ERROR: test_name (example.ExampleTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/home/skreft/test/debug_exception/example.py.py", line 14, in > test_name > NameError: global name 'fooo' is not defined. Did you mean 'foo'? > > ---------------------------------------------------------------------- > Ran 4 tests in 0.005s > > -- > Sebastian Kreft > > > _______________________________________________ > Python-ideas mailing listPython-ideas at python.orghttps://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 2012.0.2247 / Virus Database: 3705/6597 - Release Date: 02/16/14 > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Sebastian Kreft -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ab1431dd/attachment-0001.html> From ron3200 at gmail.com Sat Mar 8 18:35:34 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Mar 2014 11:35:34 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308070840.GZ28804@ando> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <20140308070840.GZ28804@ando> Message-ID: <lffkco$7rl$1@ger.gmane.org> On 03/08/2014 01:08 AM, Steven D'Aprano wrote: > On Fri, Mar 07, 2014 at 06:02:02PM -0800, Guido van Rossum wrote: >> >On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano<steve at pearwood.info> wrote: >> >The issue (as I see it) is that there are many different decimals d that >> >all convert to the same float f (because of rounding). The d that is >> >constructed by taking the exact value of f is gross overkill. > Decimal -> float is many-to-one: more than one Decimal will round to a > single float. But float -> Decimal is always one-to-one, I think, > regardless of whether you use the current exact conversion or repr > first. The crux of the matter is whether or not it is overkill for > Decimal to use the exact value. > Correct me if I'm wrong, but I think we agree that there ought to be a > way to convert floats exactly to Decimal, we just disagree on whether > that ought to be spelled Decimal(x) or Decimal.from_float(x). Decimal(x), just like you do... float(x) when x is an int. I think the from_float on the decimal type was an attempt to solve a problem that should have been solved by more explicit docs on the decimal class. A constructor should call the type to_type method when it's higher precision type. float(decimal('2.1')) # float call's decimals to_float() method. int(decima('2.1')) # int call's decimals to_int() mehtod. *1 1.[* or __int__] Going in the other direction is different. They aren't symmetric operations and to think they are is a mistake. float(int('2.1')) # float knows how to convert ints. decimal(float('2.1)) # decimal knows how to convert floats. What this says if an object can supply it's own converter for less accurate types, but should ask the other how to covert for more accurate types. (or ask if it doesn't know how to covert.) So the decimal.from_float method is redundant, as it was combined into the constructor. (which is the more consistent to python interface. I'm not sure if that was the main reason, but it makes sense to me.) > Likewise I think we agree that there ought to be some way to convert > floats to the nearest "simple" Decimal, with the question being whether > that ought to be spelled Decimal(repr(x)) or Decimal(x). It should be this... I'm absolutely certain! :-) Decimal(str(n)) No problems with that as long as it's an explicit choice. And by adding a decimal literal, we avoid the float to decimal conversion completely for setting decimal value constants and entering in raw data by hand. 2.1d The use of str(n) above is more about getting a human readable form. When converting the internal value of a float to a decimal, it should equal the floats exact value. A repr should give the exact value its object if it's suppose to be a machine readable version of it. (As numbers __repr__ should do.) The __str__ method should be the human readable version. Possibly the console can have a setting to use str() in place of repr(). So when you just use the console as a calculator, it will work more like one. In other words, don't fix decimal, because floats repr isn't showing it's exact value. That's really an issue with float, not decimal. And don't use repr() if you want a nice human readable value, use str(). > If we're all in agreement that this is the only serious area of > disagreement for a change which has any hope of appearing in 3.5, then > we can put aside the more speculative proposals The following seem like important relationships to me. For all numbers (n), with exact representation in int. '1' -> int -> float -> decimal (exact) decimal' -> float -> int -> '1' True >>> n = D(float(int(1))) >>> int(float(n)) 1 For all numbers (n), with exact representations in float. n -> float -> decimal (exact) decimal -> float -> n True >>> n = D(float('2.5')) >>> float(n) 2.5 Is this valid in python 3.4? When going from higher precision to lower precision values, (as stored internally), the best answer is to use the best value with the lowest possible error. (Not necessarily the one that looks the nicest.) I say potential, because if you don't know what the error may be, you shouldn't guess. It's possible someone somewhere is studying exact float representations as a set. ;-) Data input inaccuracies are for the user to determine and handle, but python should supply the tools to help them do that. It just shouldn't do it prematurely. Cheers, Ron From rosuav at gmail.com Sat Mar 8 18:43:56 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 04:43:56 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lffkco$7rl$1@ger.gmane.org> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org> Message-ID: <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com> On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam <ron3200 at gmail.com> wrote: > A repr should give the exact value its object if it's suppose to be a > machine readable version of it. (As numbers __repr__ should do.) As I understand it, float.__repr__ does indeed give the exact value, in terms of reconstructing the float. There are infinitely many float literals that will result in the exact same bit pattern, so any of them is valid for repr(n) to return. >>> 1.233999999999999985 == 1.234 True >>> repr(1.233999999999999985) '1.234' ChrisA From ron3200 at gmail.com Sat Mar 8 19:10:45 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Mar 2014 12:10:45 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org> <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com> Message-ID: <lffmen$to0$1@ger.gmane.org> On 03/08/2014 11:43 AM, Chris Angelico wrote: > On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com> wrote: >> >A repr should give the exact value its object if it's suppose to be a >> >machine readable version of it. (As numbers __repr__ should do.) > As I understand it, float.__repr__ does indeed give the exact value, > in terms of reconstructing the float. What I'm thinking about is... If floats repr is changed to disregard exta digits, will this still be true? How is float to know what exta digits should be disregarded? > There are infinitely many float literals that will result in the exact > same bit pattern, so any of them is valid for repr(n) to return. When are float literals actually converted with floats. It seems to me, that the decimal functions can be used to get the closest one. Then they will be consistent with each other. (if that isn't already being done.) Cheers, Ron From oscar.j.benjamin at gmail.com Sat Mar 8 19:11:54 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 8 Mar 2014 18:11:54 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> Message-ID: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote: > I'll try to respond to Mark Dickinson's second message (and nothing else > that happens in the thread since last night), because (a) it concisely > summarizes his position and (b) brings up a new strawman. > > On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >> >> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >>> >>> >>> I see three sane options for float to Decimal conversion: >>> >>> 1. Raise an exception. >>> 2. Round to the nearest Decimal value using the current context for that >>> round operation. >>> 3. Do what we're currently doing, and do an exact conversion. > > I think you're writing this entirely from the POV of an expert in floating > point. And I'm glad we have experts like you around! I don't consider myself > an expert at all, but I do think I have something to bring to the table -- > insight the experience of non-expert users. Standards compliance is important though. Mark is essentially restating something that is repeated hundreds of times throughout various floating point standards documents: a result may be exact or it must be correctly rounded according to the current context (with appropriate flags set and traps fired). It is not mandated that results requiring more precision than the current context be exact. It is mandated that if a result is to be inexact then the implementation must use a very specific type of inexactness. I don't believe that a standards-compliant decimal module has any wiggle room to invent a new kind of rounding (which I think would be required to achieve what you suggest). > When a non-expert writes Decimal(1.1), each of the three above outcomes > surprises. We know that (1) was unpopular, that's why we changed it. We now > know that (3) is unpopular at least in some circles (Mark Harrison can't be > the only one who doesn't like it). Changing to (2) wouldn't do much to > address this, because the default context has way more precision than float, > so it still shows a lot of extraneous digits. > > Yes, it's trivial to get rid of those extra digits, just use quotes. But if > *my* proposal were adopted, it would be trivial for numerical experts to get > the extra digits -- just use from_float(). At this point, my claim is that > we're talking essentially about what is the better experience for most > users, and while I am not much of a user of Decimal myself, I believe that > my proposal has benefits more people and situations than it has downsides. If you write Decimal(1.1) and are surprised by the result then you have misunderstood something. It may be that you have little understanding of the difference between binary and decimal floating point (but then why are you using Decimal?). Perhaps you don't fully understand the literal->float->Decimal pathway that occurs when the expression is evaluated because you're new to Python or just haven't really thought about it before. In any case if the result of Decimal(1.1) surprises you then it's because you're expecting it do something that should be done in a different way. Hiding the extra digits does not help a user to understand how to use Decimal. I actually use this in teaching to demonstrate how binary floating point works. I think it's important when teaching my students for them to understand that the following is a lie: >>> a = 1.1 >>> a 1.1 The helpful digit-hiding repr is lying to you. There is no float with the value 1.1. I can sort-of demonstrate this with some arithmetic: >>> 0.1 + 0.11 - 0.11 - 0.1 1.3877787807814457e-17 But that gives the misleading impression that inexact arithmetic is the source of the error. It's important to understand that the error occurs straight away in the literal "1.1". I demonstrate this by showing that >>> from decimal import Decimal >>> a = 1.1 >>> Decimal(a) Decimal('1.100000000000000088817841970012523233890533447265625') which also shows the true value stored in the float. The fact that I use this in teaching is not supposed to serve as an argument for keeping it (I could just as easily use from_float). My point is that showing the digits helps someone to understand what is going on. If a user is converting float to Decimal without knowing what they're doing then the extra digits are a clue that they don't fully understand what's happening and haven't necessarily used the best approach. There is a good solution to the problem of non-experts wanting to write 1.1 and get the exact value 1.1: decimal literals. With that they can just write 1.1d and not need to learn any more about it. Earlier in this thread you reject that idea saying that you can't teach it to "newbies": ''' Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.) ''' I disagree. If you're at the point of wanting to use the Decimal module then you're at the point where it's reasonable to learn about Decimal literals. Also I've written code using the decimal module for high precision calculation and it has lots of decimal literals. That is I do (and I see other's doing) >>> from decimal import Decimal as D >>> d = D('1e-20') even though this is never used in the decimal documentation. This is the closest you can get to decimal literals right now. Oscar From abarnert at yahoo.com Sat Mar 8 19:43:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 10:43:23 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lffmen$to0$1@ger.gmane.org> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org> <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com> <lffmen$to0$1@ger.gmane.org> Message-ID: <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com> From: Ron Adam <ron3200 at gmail.com> Sent: Saturday, March 8, 2014 10:10 AM > On 03/08/2014 11:43 AM, Chris Angelico wrote: >> On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com>? wrote: > >>> >A repr should give the exact value its object if it's suppose > to be a >>> >machine readable version of it.? (As numbers __repr__ should do.) > > >> As I understand it, float.__repr__ does indeed give the exact value, >> in terms of reconstructing the float. > > What I'm thinking about is... > > If floats repr is changed to disregard exta digits, will this still be > true?? How is float to know what exta digits should be disregarded? No one is suggesting such a change, and it would be shot down if anyone did. The old repr and str for float used to discard (different numbers of) digits. The current version does not. Instead, if picks the shortest string that would evaluate back to the same value if passed to the float constructor (or to eval). So, repr(0.100000000000000006) == '0.1', not because repr is discarding digits, but because?0.100000000000000006 ==?0.1 (because the closest binary IEEE double value to both is?0.1000000000000000055511151231257827021181583404541015625). >> There are infinitely many float literals that will result in the exact >> same bit pattern, so any of them is valid for repr(n) to return. > > When are float literals actually converted with floats.? It seems to me, > that the decimal functions can be used to get the closest one.? Then they > will be consistent with each other. (if that isn't already being done.) Float literals are converted to floats at compile time. When the compiler sees 0.1, or?0.100000000000000006, it works out the nearest IEEE double to that literal and stores that double. So, by the time any Decimal function sees the float, there's no way to tell whether it was constructed from the literal 0.1, the literal?0.100000000000000006, or some long chain of transcendental functions whose result happened to have a result within 1 ulp of 0.1. The current behavior guarantees that, for any float, float(Decimal(f)) == float(repr(Decimal(f))) == f. Guido's proposal would preserve that guarantee. If that's all you care about, nothing would change. Guido is just suggesting that, instead of using the middle Decimal from the infinite set of Decimals that would make that true, we use the shortest one. From dickinsm at gmail.com Sat Mar 8 19:49:02 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 8 Mar 2014 18:49:02 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> Message-ID: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> On Sat, Mar 8, 2014 at 4:54 PM, Guido van Rossum <guido at python.org> wrote: > When a non-expert writes Decimal(1.1), each of the three above outcomes > surprises. We know that (1) was unpopular, that's why we changed it. We now > know that (3) is unpopular at least in some circles (Mark Harrison can't be > the only one who doesn't like it). Changing to (2) wouldn't do much to > address this, because the default context has way more precision than > float, so it still shows a lot of extraneous digits. > That at least could be addressed by making the default context one that corresponds to IEEE's decimal64, which has a precision of 16 decimal digits; there doesn't seem to be any particularly good rationale for the current choice of default context. And that would address your point (which I agree with) that `Decimal(x)` currently shows way more digits than are useful. It wouldn't address Mark Harris's needs, though, and would in some sense make things worse for non-expert users: for *most* floats x, Decimal(x) would then give what the user expects purely because 16 digits is a good match for the binary float precision, but for a small fraction of inputs it would give a surprising result. >>> from decimal import Decimal, getcontext >>> getcontext().prec = 16 >>> +Decimal(1.1) # + to simulate rounding to 16 digits Decimal('1.100000000000000') >>> +Decimal(1.2) Decimal('1.200000000000000') >>> +Decimal(9.7) Decimal('9.699999999999999') This feels like a strawman, since Decimal(2**1000 + 1) and > Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot > of digits but is one too small), and the similar examples with a large > exponent (10000) differ dramatically (the float version raises an > exception). > > For most purposes it's a fluke that 2.0**1000 can be represented exactly > as a float, and the argument doesn't convince me at all. There are just too > many odd examples like that, and it always will remain a minefield. > Accepted, but I believe the proposal would break a number of other expectations too with respect to ints and floats. For one, we currently have the unsurprising, and I believe desirable, property that conversion to Decimal is monotonic: for any finite numbers (int, float or Decimal) x and y, if x <= y then Decimal(x) <= Decimal(y). The proposal would break that property, too: you could find examples of an integer x and float y such that `x < y` and `Decimal(x) > Decimal(y)` would be simultaneously True. >>> x = 49534541648432951 >>> y = x + 2.0 >>> x < y True >>> Decimal(x) > Decimal(repr(y)) # simulating proposed meaning of Decimal(y) True I'm still struggling a bit to express exactly what it is that bothers me so much about the proposal. It's a combination of the above with: - there's an obvious *correct* way to convert any value to Decimal, and that's to round to the context precision using the context rounding mode - that's what's done almost universally for Decimal arithmetic operations; it feels wrong that something as direct as `Decimal(x)` would do anything else (the current exact conversions actually *do* rankle with me a bit, but this is what I'd replace them with) - this proposal would not play well with other binary number formats if they were ever introduced: if we introduced a float128 type (or a float32 type), it would be awkward to reconcile the proposed conversion with a conversion from float128 to Decimal. That's mostly because the semantics of the proposed conversion use information about the *format* of the input type, which is unusual for floating-point and floating-point standards: operations tend to be based purely on the *values* of the inputs, disregarding the formats. - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will Decimal(0.12345123451234512345) - contrarily, the fix goes too far: it feels wrong to be changing the semantics of float to Decimal conversion when the real problem is that the user wants or expects floating-point literals to represent decimal numbers. If the proposal goes forward, I'll live with it, and will simply avoid using the `Decimal` type to convert from floats or ints. But I'd really prefer to keep the short Decimal(x) spelling as something simple and non-magic, and find more complicated ways (quotes!) of spelling the magic stuff. I wonder what Cowlishaw would say about our current discussion. He is also > the father of Rexx... > I wonder that, too. I've said too much already; beyond registering my strong -1 on this proposal, I'm going to keep out of further discussion. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/60e1a2b2/attachment.html> From harrismh777 at gmail.com Sat Mar 8 19:51:59 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 10:51:59 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> Message-ID: <eaeb9a71-318e-4cd2-80bd-5e04b23e165c@googlegroups.com> On Saturday, March 8, 2014 4:01:47 AM UTC-6, Mark Dickinson wrote On the other hand, we probably shouldn't lend *too* much weight to IEEE > 754, especially when talking about choice of precision. IEEE 754 isn't a > perfect fit for Decimal: the IEEE standard is mostly concerned with fixed > width decimal formats, which is subtly different from Mike Cowlishaw's > approach of "extensible precision" where the precision is not so closely > tied to the format. Python's decimal module is based on Cowlishaw's > standard, not on IEEE 754. > hi Mark, Mike's notes include the following: -- the package meets the requirements of IEEE 854-1987 (with minor restrictions discussed below), including support for subnormal numbers, -0, NaNs, infinities, etc. It also conforms to the floating-point arithmetic definition in ANSI X3.274-1996. here: http://grouper.ieee.org/groups/754/email/msg00429.html The extensible precision is what makes the standard interesting. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/1b7c99ee/attachment-0001.html> From solipsis at pitrou.net Sat Mar 8 19:55:37 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 8 Mar 2014 19:55:37 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> Message-ID: <20140308195537.5928d3aa@fsol> On Sat, 8 Mar 2014 18:49:02 +0000 Mark Dickinson <dickinsm at gmail.com> wrote: > > If the proposal goes forward, I'll live with it, and will simply avoid > using the `Decimal` type to convert from floats or ints. But I'd really > prefer to keep the short Decimal(x) spelling as something simple and > non-magic, and find more complicated ways (quotes!) of spelling the magic > stuff. For the record, as a non-float expert, Mark's arguments convince me. Just my 2 cents ;-) Regards Antoine. From guido at python.org Sat Mar 8 19:59:11 2014 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Mar 2014 10:59:11 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> Message-ID: <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> Thanks Oscar, that's a very well-reasoned post. On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com>wrote: > On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote: > > I'll try to respond to Mark Dickinson's second message (and nothing else > > that happens in the thread since last night), because (a) it concisely > > summarizes his position and (b) brings up a new strawman. > > > > On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> > wrote: > >> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> > wrote: > >>> I see three sane options for float to Decimal conversion: > >>> > >>> 1. Raise an exception. > >>> 2. Round to the nearest Decimal value using the current context for > that > >>> round operation. > >>> 3. Do what we're currently doing, and do an exact conversion. > > > > I think you're writing this entirely from the POV of an expert in > floating > > point. And I'm glad we have experts like you around! I don't consider > myself > > an expert at all, but I do think I have something to bring to the table > -- > > insight the experience of non-expert users. > > Standards compliance is important though. Mark is essentially > restating something that is repeated hundreds of times throughout > various floating point standards documents: a result may be exact or > it must be correctly rounded according to the current context (with > appropriate flags set and traps fired). > I have mixed feelings about such standards. I can see its importance. But like the Unicode standard, it seems to want to grab authority over areas that I think belong to the language design. Also at this point claiming "compliance" with some standard is usually a morass of weasel-words rather than clearly implementing a spec. > It is not mandated that results requiring more precision than the > current context be exact. It is mandated that if a result is to be > inexact then the implementation must use a very specific type of > inexactness. I don't believe that a standards-compliant decimal module > has any wiggle room to invent a new kind of rounding (which I think > would be required to achieve what you suggest). > That would be unfortunate. > > When a non-expert writes Decimal(1.1), each of the three above outcomes > > surprises. We know that (1) was unpopular, that's why we changed it. We > now > > know that (3) is unpopular at least in some circles (Mark Harrison can't > be > > the only one who doesn't like it). Changing to (2) wouldn't do much to > > address this, because the default context has way more precision than > float, > > so it still shows a lot of extraneous digits. > > > > Yes, it's trivial to get rid of those extra digits, just use quotes. But > if > > *my* proposal were adopted, it would be trivial for numerical experts to > get > > the extra digits -- just use from_float(). At this point, my claim is > that > > we're talking essentially about what is the better experience for most > > users, and while I am not much of a user of Decimal myself, I believe > that > > my proposal has benefits more people and situations than it has > downsides. > > If you write Decimal(1.1) and are surprised by the result then you > have misunderstood something. It may be that you have little > understanding of the difference between binary and decimal floating > point (but then why are you using Decimal?). Perhaps you don't fully > understand the literal->float->Decimal pathway that occurs when the > expression is evaluated because you're new to Python or just haven't > really thought about it before. > Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the result, but it's still not what I want, and it's a pain to train myself to do the extra thing that gives me what I want. > In any case if the result of Decimal(1.1) surprises you then it's > because you're expecting it do something that should be done in a > different way. Hiding the extra digits does not help a user to > understand how to use Decimal. > But does showing the extra digits do anything to help? It's just as likely to teach them a trick (add quotes or a str() call) without any new understanding. > I actually use this in teaching to demonstrate how binary floating > point works. I think it's important when teaching my students for them > to understand that the following is a lie: > > >>> a = 1.1 > >>> a > 1.1 > > The helpful digit-hiding repr is lying to you. There is no float with > the value 1.1. I can sort-of demonstrate this with some arithmetic: > > >>> 0.1 + 0.11 - 0.11 - 0.1 > 1.3877787807814457e-17 > > But that gives the misleading impression that inexact arithmetic is > the source of the error. It's important to understand that the error > occurs straight away in the literal "1.1". I demonstrate this by > showing that > > >>> from decimal import Decimal > >>> a = 1.1 > >>> Decimal(a) > Decimal('1.100000000000000088817841970012523233890533447265625') > > which also shows the true value stored in the float. > > The fact that I use this in teaching is not supposed to serve as an > argument for keeping it (I could just as easily use from_float). My > point is that showing the digits helps someone to understand what is > going on. If a user is converting float to Decimal without knowing > what they're doing then the extra digits are a clue that they don't > fully understand what's happening and haven't necessarily used the > best approach. > I don't think every user of Decimal necessarily needs to be an expert capable of explaining what's going on. Certainly that's not needed to be an effective user of float -- the anomalies are explained to most people's satisfaction by some hand-waving about imprecise results. > There is a good solution to the problem of non-experts wanting to > write 1.1 and get the exact value 1.1: decimal literals. With that > they can just write 1.1d and not need to learn any more about it. > Earlier in this thread you reject that idea saying that you can't > teach it to "newbies": > ''' > Maybe we can add a new literal notation meaning 'decimal'; that would > be a relatively straightforward change to the parser (once the new > decimal extension module is incorporated), but it would not do much to > avoid surprising newbies (certainly you can't go teaching them to > always write 3.14d instead of 3.14). However, it would probably help > out frequent users of Decimal. (Then again, they might not be using > literals that much -- I imagine the source of most such programs is > user or file input.) > ''' > I disagree. If you're at the point of wanting to use the Decimal > module then you're at the point where it's reasonable to learn about > Decimal literals. > You're right that I dismissed it too quickly. 3.14d is clearly even better than Decimal(3.14) doing the right thing. It is also still a lot more work (touches many parts of the code rather than just the Decimal class). Also I didn't realize that the C-implemented decimal module was already used in CPython (so I thought it would be even more work). > Also I've written code using the decimal module for high precision > calculation and it has lots of decimal literals. That is I do (and I > see other's doing) > > >>> from decimal import Decimal as D > >>> d = D('1e-20') > > even though this is never used in the decimal documentation. This is > the closest you can get to decimal literals right now. Right. But I still have this nagging feeling that the precision Decimal(<float>) currently gives you is, in a sense, *fake*, given that the input has much less precision. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/e16f9768/attachment.html> From ron3200 at gmail.com Sat Mar 8 20:47:43 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 08 Mar 2014 13:47:43 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <20140308070840.GZ28804@ando> <lffkco$7rl$1@ger.gmane.org> <CAPTjJmrgbz=owdBkxcRmmE_TeSRG957wGcPHAL0fTxfx1tv0tw@mail.gmail.com> <lffmen$to0$1@ger.gmane.org> <1394304203.37100.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <lffs4h$opt$1@ger.gmane.org> On 03/08/2014 12:43 PM, Andrew Barnert wrote: > From: Ron Adam <ron3200 at gmail.com> > > Sent: Saturday, March 8, 2014 10:10 AM > >> On 03/08/2014 11:43 AM, Chris Angelico wrote: >>> On Sun, Mar 9, 2014 at 4:35 AM, Ron Adam<ron3200 at gmail.com> wrote: >> >>>> >A repr should give the exact value its object if it's suppose >> to be a >>>> >machine readable version of it. (As numbers __repr__ should do.) >>> As I understand it, float.__repr__ does indeed give the exact value, >>> in terms of reconstructing the float. >> What I'm thinking about is... >> >> If floats repr is changed to disregard exta digits, will this still be >> true? How is float to know what exta digits should be disregarded? > No one is suggesting such a change, and it would be shot down if anyone did. Glad to hear it! :-) > The old repr and str for float used to discard (different numbers of) > digits. The current version does not. Instead, if picks the shortest > string that would evaluate back to the same value if passed to the float > constructor (or to eval). > > So, repr(0.100000000000000006) == '0.1', not because repr is discarding > digits, but because 0.100000000000000006 == 0.1 (because the closest > binary IEEE double value to both is > 0.1000000000000000055511151231257827021181583404541015625). >>> There are infinitely many float literals that will result in the >>> exact >>> >>> same bit pattern, so any of them is valid for repr(n) to return. >> When are float literals actually converted with floats. It seems to >> me, that the decimal functions can be used to get the closest one. >> Then they will be consistent with each other. (if that isn't already >> being done.) > Float literals are converted to floats at compile time. When the > compiler sees 0.1, or 0.100000000000000006, it works out the nearest > IEEE double to that literal and stores that double. > So, by the time any Decimal function sees the float, there's no way to > tell whether it was constructed from the literal 0.1, the literal > 0.100000000000000006, or some long chain of transcendental functions > whose result happened to have a result within 1 ulp of 0.1. > > The current behavior guarantees that, for any float, float(Decimal(f)) > == float(repr(Decimal(f))) == f. Guido's proposal would preserve that > guarantee. If that's all you care about, nothing would change. Guido is > just suggesting that, instead of using the middle Decimal from the > infinite set of Decimals that would make that true, we use the shortest > one. Sounds good to me.. Can that effect rounding where a value may round down instead of up? or vice versa. If so, it should be the shortest string that does not cross the mid point of the two closest floats. (I think I got that right.) Not sure where I read it last night, but there was a mention that only a few languages do the this conversion with less than .5 Ulps error. But it seems to me it might be more important to not to error in the wrong direction if there is a choice. Well I'll leave it up you guys, But it's a very interesting topic for sure. Cheers, Ron From harrismh777 at gmail.com Sat Mar 8 20:13:24 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 11:13:24 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> Message-ID: <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote: - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: > Decimal(1.1 + 2.2) will still surprise, as will {snip} > Correct. That is why a decimal literal notation is also needed. I've emulated it here: >>> from pdeclib import * >>> sqrt(1.1+2.2)**2 Decimal('3.30000000000000026645352591003756970167159') <==== this is a surprise, to a naive user >>> sqrt(d(1.1)+d(2.2))**2 Decimal('3.29999999999999999999999999999999999999999') <=== this if decimal literal, >>> sqrt(1.1d + 2.2d) But, you are correct that what is "really" wanted --someday-- is to have the literal be decimal (rather than float) to begin with. Neither here nor there, step at a time over time is better than simple status quo. Please let me be clear, I think Guido's proposal is a very good first step. It makes sense for the most users (esp naive ones) and does not interfere with advanced users. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/b248c6e4/attachment.html> From rosuav at gmail.com Sat Mar 8 21:12:04 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 07:12:04 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> Message-ID: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> On Sun, Mar 9, 2014 at 6:13 AM, Mark H. Harris <harrismh777 at gmail.com> wrote: > But, you are correct that what is "really" wanted --someday-- is to > have > the literal be decimal (rather than float) to begin with. > > Neither here nor there, step at a time over time is better than simple > status quo. > > Please let me be clear, I think Guido's proposal is a very good first step. > It makes > sense for the most users (esp naive ones) and does not interfere with > advanced users. It's probably time someone [1] wrote up a PEP about all this. The most important points, as I see it, are: 1) Create a new Decimal literal notation - 1.234d seems to be the most popular syntax. This is reasonably uncontroversial, but it has consequences. 2) Create a new float literal notation - 1.234f or 1.234r or any of the other proposals. 3) Possibly change repr(float) to include the tag. 4) Introduce a "from __future__ import decimal_literals" (named to parallel unicode_literals - you can get a u"literal" without that directive, but the default literal type becomes unicode) 5) What about int/int? Should that now be Decimal? Should it be per-module??? 6) Further cans of worms like #5 Introducing 1/2/4 would let you stick the future directive into PYTHONSTARTUP and then run Python as an interactive decimal calculator. ChrisA [1] Not it! From harrismh777 at gmail.com Sat Mar 8 21:16:00 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 12:16:00 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> Message-ID: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote: > - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: > Decimal(1.1 + 2.2) will still surprise, as will {snip} > On the other hand, how is this now possible? >>> ====== RESTART ========================== >>> from pdeclib import * >>> d(1.1+2.2) Decimal('3.3') >>> sqrt(1.1+2.2) Decimal('1.81659021245849499925351968583091621951684') >>> sqrt(1.1+2.2)**2 Decimal('3.29999999999999999999999999999999999999999') >>> def sqrt(x): """ sqrt(x) square root function (x may be string, int, float, or decimal) (returns decimal rounded to context precision) """ y=x.__round__(15) with localcontext(ctx=None) as cmngr: cmngr.prec+=14 sqr=Decimal(repr(y)).sqrt() return +sqr What say you? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/221d8c53/attachment.html> From oscar.j.benjamin at gmail.com Sat Mar 8 21:32:58 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 8 Mar 2014 20:32:58 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> Message-ID: <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> On 8 March 2014 18:59, Guido van Rossum <guido at python.org> wrote: > On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: >> >> On 8 March 2014 16:54, Guido van Rossum <guido at python.org> wrote: >> >> If you write Decimal(1.1) and are surprised by the result then you >> have misunderstood something. > > Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the > result, but it's still not what I want, and it's a pain to train myself to > do the extra thing that gives me what I want. That "you" wasn't directed at *you* personally (and neither are the ones below). If you're using the Decimal module then it is precisely because you do need to care about rounding/accuracy etc. If you want to use it but aren't prepared to take the effort to be careful about how to pass exact input to the constructor then you're basically taking an impossible position that no one else can help you with. >> In any case if the result of Decimal(1.1) surprises you then it's >> because you're expecting it do something that should be done in a >> different way. Hiding the extra digits does not help a user to >> understand how to use Decimal. > > But does showing the extra digits do anything to help? It's just as likely > to teach them a trick (add quotes or a str() call) without any new > understanding. It's enough to make you think "Why did that happen?". It's clear when you see those digits that you have not created an object with the exact value that you wanted. The obvious next question is "How do I make it do what I want?". The docs can lead you very quickly to the correct way of doing it: http://docs.python.org/3.4/library/decimal.html#quick-start-tutorial >> There is a good solution to the problem of non-experts wanting to >> write 1.1 and get the exact value 1.1: decimal literals. With that >> they can just write 1.1d and not need to learn any more about it. > > You're right that I dismissed it too quickly. 3.14d is clearly even better > than Decimal(3.14) doing the right thing. It is also still a lot more work > (touches many parts of the code rather than just the Decimal class). > > Also I didn't realize that the C-implemented decimal module was already used > in CPython (so I thought it would be even more work). As Stefan mentioned earlier there are other issues to resolve around how a decimal literal should work. It's not obvious that there should be a straight-forward equivalence between 3.14d and D('3.14'). Perhaps there should be new thread to consider how to potentially do that (and whether or not it's worth it). > But I still have this nagging feeling that the precision Decimal(<float>) > currently gives you is, in a sense, *fake*, given that the input has much > less precision. That depends what you use it for. My most common reason for converting a float to a Decimal is to test the accuracy of a float-based calculation by comparing it against the corresponding much higher-precision decimal calculation e.g.: with localcontext() as ctx: ctx.prec = 100 error = f(D(x)) - D(f(x)) For this I want the constructor to give me the exact value of the float x. Oscar From harrismh777 at gmail.com Sat Mar 8 21:25:58 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 12:25:58 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com> Message-ID: <6f02d04b-5d9e-44d2-9e0d-4649f033018a@googlegroups.com> On Saturday, March 8, 2014 2:23:23 PM UTC-6, Mark H. Harris wrote: > > > > >> y=x.__round__(15) <====== how to set the round within context so >> that works for float literals & decimals >> with localcontext(ctx=None) as cmngr: >> {snip} >> > > > Is this possible..? > Only do this for float literals. Yes? Set a trap, and choose. I gotta play with this a bit... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/77ab3924/attachment.html> From harrismh777 at gmail.com Sat Mar 8 22:01:32 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 13:01:32 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> Message-ID: <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote: > > > > On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote: > > >> - if we're aiming to eliminate surprises, the 'fix' doesn't go far >> enough: Decimal(1.1 + 2.2) will still surprise, as will {snip} >> > > How about this then: I think I've got it... >>> ===== RESTART ========================= >>> from pdeclib import * >>> d(1.1+2.2) Decimal('3.3') >>> sqrt(1.1+2.2)**2 Decimal('3.29999999999999999999999999999999999999999') >>> x=d(1.1) >>> y=d(2.2) >>> sqrt(x+y)**2 Decimal('3.29999999999999999999999999999999999999999') >>> Code Below-------------------------- #*****************************************************************/ # sqrt(x) return decimal sqrt(x) rounded to context precision #*****************************************************************/ def sqrt(x): """ sqrt(x) square root function (x may be string, int, float, or decimal) (returns decimal rounded to context precision) """ with localcontext(ctx=None) as cmngr: cmngr.prec+=14 if (isinstance(x, float)==True): y=x.__round__(15) sqr=Decimal(repr(y)).sqrt() else: sqr=Decimal(x).sqrt() return +sqr -------------------- what say you? --------------------- marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/611d447d/attachment.html> From p.f.moore at gmail.com Sat Mar 8 22:32:34 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 8 Mar 2014 21:32:34 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140308195537.5928d3aa@fsol> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <20140308195537.5928d3aa@fsol> Message-ID: <CACac1F8SjQJ7iRRdG6j=B5nVJXbEA320ywdJYuH9vV1rsm-TyA@mail.gmail.com> On 8 March 2014 18:55, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sat, 8 Mar 2014 18:49:02 +0000 > Mark Dickinson <dickinsm at gmail.com> wrote: >> >> If the proposal goes forward, I'll live with it, and will simply avoid >> using the `Decimal` type to convert from floats or ints. But I'd really >> prefer to keep the short Decimal(x) spelling as something simple and >> non-magic, and find more complicated ways (quotes!) of spelling the magic >> stuff. > > For the record, as a non-float expert, Mark's arguments convince me. > > Just my 2 cents ;-) I am in the same position. I "instinctively" wanted Decimal(1.1) to return the same value as Decimal('1.1'), but as I read Mark's argument, it became clearer to me that doing so involves a deliberate loss of precision that is not present anywhere else in the chain of conversions. literal 1.1 -> float 1.1 is a precise conversion, in the sense that no other float better represents the (decimal) literal 1.1. The current float 1.1 -> decimal conversion is exact. Converting float 1.1 to Decimal('1.1') deliberately drops precision, even though an exact conversion is possible. I don't think that this is something that should happen implicitly. On the other hand, the *reason* people do Decimal(1.1) is because they want a decimal value of 1.1. (Doh.) The fact that this isn't what they are actually calculating is a mistake they make because Python doesn't make it easy to *get* the decimal value 1.1 (I know the correct approach is Decimal('1.1'), but I don't think that is obvious or intuitive[1]). Decimal literals (1.1d) would resolve this issue without introducing implicit rounding operations. Paul [1] The reason Decimal('1.1') feels non-obvious to me is that it feels like it's getting to a decimal "via" a string. Of course, Decimal(1.1) is also a going "via" a float. The *value* of the current behaviour is that it leads people to an understanding that this is what it is. From mertz at gnosis.cx Sat Mar 8 22:33:09 2014 From: mertz at gnosis.cx (David Mertz) Date: Sat, 8 Mar 2014 13:33:09 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> Message-ID: <CAEbHw4Yj5OA1Z68OwEvsENY_KKyW_Dd+aY0=2eV6cH6Sqq5qbQ@mail.gmail.com> On Sat, Mar 8, 2014 at 10:49 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > Accepted, but I believe the proposal would break a number of other > expectations too with respect to ints and floats. For one, we currently > have the unsurprising, and I believe desirable, property that conversion to > Decimal is monotonic: for any finite numbers (int, float or Decimal) x and > y, if x <= y then Decimal(x) <= Decimal(y). The proposal would break that > property, too: you could find examples of an integer x and float y such > that `x < y` and `Decimal(x) > Decimal(y)` would be simultaneously True. > I think that Mark Dickinson's point about breaking monotonicity is a STRONG argument against the proposal to change the meaning of Decimal(float_num) to produce Decimal(repr(float_num)). That latter spelling is already available if anyone wants it. The focus on float value that have been entered as literals feels like a distraction to me. Floats come from other places as well, and making Decimal(float_num) produce the *exact* value feels far more desirable than producing "something which rounds to the same float. As I think Guido has acknowledged in a recent post, a far better and more intuitive approach is just to make decimal literals easier to write. Teaching users to write '3.14d' doesn't feel that hard to me, and it looks notably prettier than 'decimal.Decimal("3.14")'. A *would* also like an optional explicit binary-float literal though, e.g. '3.14f'. This wouldn't actually mean anything different from '3.14', but then "PI" doesn't mean anything different from u"PI" either. The optional bin-float specifier, in principle, allows for some Python 4000 in which decimal is the default literal (although I don't think I'd ever support that, but no harm in letting that decision be made years from now). - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: > Decimal(1.1 + 2.2) will still surprise, as will > Decimal(0.12345123451234512345) > This is a simple illustration of why the "do what I mean" goal of the proposed change will fall apart rather quickly for anything but the simplest examples. The status quo of Decimal(float_num) producing the *exact* value continues to feel the best to me. I've said too much already; beyond registering my strong -1 on this > proposal, I'm going to keep out of further discussion. > So yeah, me too: -1 on proposal. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/6246cef1/attachment-0001.html> From abarnert at yahoo.com Sat Mar 8 22:35:14 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 13:35:14 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> Message-ID: <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> On Mar 8, 2014, at 13:01, "Mark H. Harris" <harrismh777 at gmail.com> wrote: > > > On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote: >> >> >> >> On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote: >> >>> - if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will {snip} > How about this then: I think I've got it... > > >>> ===== RESTART ========================= > >>> from pdeclib import * > >>> d(1.1+2.2) > Decimal('3.3') > >>> sqrt(1.1+2.2)**2 > Decimal('3.29999999999999999999999999999999999999999') > >>> x=d(1.1) > >>> y=d(2.2) > >>> sqrt(x+y)**2 > Decimal('3.29999999999999999999999999999999999999999') > >>> > > Code Below-------------------------- > > #*****************************************************************/ > # sqrt(x) return decimal sqrt(x) rounded to context precision > #*****************************************************************/ > def sqrt(x): > """ sqrt(x) square root function > > (x may be string, int, float, or decimal) > (returns decimal rounded to context precision) > """ > with localcontext(ctx=None) as cmngr: > cmngr.prec+=14 > if (isinstance(x, float)==True): > y=x.__round__(15) > sqr=Decimal(repr(y)).sqrt() > else: > sqr=Decimal(x).sqrt() > return +sqr > > -------------------- what say you? --------------------- It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ad55f1ce/attachment.html> From stefan at bytereef.org Sat Mar 8 22:43:50 2014 From: stefan at bytereef.org (Stefan Krah) Date: Sat, 8 Mar 2014 22:43:50 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> References: <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> Message-ID: <20140308214349.GA22909@sleipnir.bytereef.org> Guido van Rossum <guido at python.org> wrote: > I wonder what Cowlishaw would say about our current discussion. He is also the > father of Rexx... It's in the second paragraph here: http://speleotrove.com/decimal/daconvs.html My reading is that he prefers exact conversions (if possible). This does not surprise me: Cowlishaw is a proponent of using all information from overlong inputs, which all functions of the specification do (he recently mentioned in a private mail that earlier versions of the specification did not have this feature and it was initially added for Java's BigDecimal -- he called it "an improvement"). Stefan Krah From abarnert at yahoo.com Sat Mar 8 23:15:43 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 14:15:43 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> Message-ID: <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com> On Mar 8, 2014, at 12:12, Chris Angelico <rosuav at gmail.com> wrote: > It's probably time someone [1] wrote up a PEP about all this. > The most > important points, as I see it, are: > > 1) Create a new Decimal literal notation - 1.234d seems to be the most > popular syntax. This is reasonably uncontroversial, but it has > consequences. 1a) Move the Decimal type's implementation out of the module and into Objects, or make it a frozen module that's always loaded at startup, or invent some scheme to load it as needed. 1b) Possibly move the type to builtins? (Certainly not necessary--function is a built-in type, and you can create them without importing anything, but the type lives in types, not builtins.) 1c) Possibly add the implementation to the public C API? (With which methods? FromString, but what about From/AsDouble? Is there any need to From/AsTuple from C? Inspect the object in any other way? Get/set the current context?) > 2) Create a new float literal notation - 1.234f or 1.234r or any of > the other proposals. > 3) Possibly change repr(float) to include the tag. 3a) Change repr(Decimal) to use the new literal. > 4) Introduce a "from __future__ import decimal_literals" (named to > parallel unicode_literals - you can get a u"literal" without that > directive, but the default literal type becomes unicode) > 5) What about int/int? Should that now be Decimal? Should it be per-module??? > 6) Further cans of worms like #5 > > Introducing 1/2/4 would let you stick the future directive into > PYTHONSTARTUP and then run Python as an interactive decimal > calculator. From rosuav at gmail.com Sat Mar 8 23:25:16 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 09:25:16 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> <1FFAF37E-D2C0-4C62-8A74-4AC2B66347A3@yahoo.com> Message-ID: <CAPTjJmohRodSCbLE0evtA9ChC+OdwzYtFu+V98giN93fXXr_RQ@mail.gmail.com> On Sun, Mar 9, 2014 at 9:15 AM, Andrew Barnert <abarnert at yahoo.com> wrote: >> 3) Possibly change repr(float) to include the tag. > > 3a) Change repr(Decimal) to use the new literal. I'd put that in the consequences of part 1 - it's pretty obvious that, with a decimal literal, the repr of a Decimal should be that. It'd affect people's tests but shouldn't break much else. But changing repr(float) means tagging everything, rather than just having a new convenient notation for Decimal. It's the same physical/technical change but it has a lot more implication :) And yeah. You listed several significant quinseconses, and I've no doubt there'll be others. ChrisA From ethan at stoneleaf.us Sat Mar 8 22:59:32 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 08 Mar 2014 13:59:32 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> Message-ID: <531B92C4.3040502@stoneleaf.us> On 03/08/2014 12:32 PM, Oscar Benjamin wrote: > On 8 March 2014 18:59, Guido van Rossum wrote: >> >> But I still have this nagging feeling that the precision Decimal(<float>) >> currently gives you is, in a sense, *fake*, given that the input has much >> less precision. > > That depends what you use it for. My most common reason for converting > a float to a Decimal is to test the accuracy of a float-based > calculation by comparing it against the corresponding much > higher-precision decimal calculation e.g.: > > with localcontext() as ctx: > ctx.prec = 100 > error = f(D(x)) - D(f(x)) > > For this I want the constructor to give me the exact value of the float x. I am not a mathematician, and it's been a long time since I took physics, but I seem to remember that a lot of importance was placed on significant digits. So, how is this justified? Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> from decimal import Decimal as D --> 9017.0109812864350067128347806 9017.010981286436 --> D(9017.0109812864350067128347806) Decimal('9017.01098128643570817075669765472412109375') In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits. Why is the Decimal more "accurate" that the float it came from? -- ~Ethan~ From abarnert at yahoo.com Sat Mar 8 23:27:25 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 8 Mar 2014 14:27:25 -0800 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> Message-ID: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> On Mar 8, 2014, at 13:54, "Mark H. Harris" <harrismh777 at gmail.com> wrote: > On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote: > >> It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal? > > That's fair... if it were me, or you, I wouldn't. In fact, if it were just me, or just you, > I don't think any of us would be talking about this--- I wouldn't even have brought it up. > > I am advocating for people--- I am going to say, " if you want just a tad more than double > just to check things out, please use the decimal module and import pdeclib. These folks > are what Guido called newbies, I'll call them naive users, and these folks are using > a calculator (like the TI89) and they will be prone to keying in sqrt(2.345) / So your goal is that if someone is a TI89 expert but a Python novice, they can easily port a function from their TI89 to Python (or maybe even write a TI89-like desktop calculator program) with your library? That doesn't seem too unreasonable of a goal. The problem is that you're not quite achieving it. If you want a 16-digit-display calculator, using a mix of float64 values and decimals that expand as needed for intermediates is not equivalent to using 20-digit fixed decimals. So they will not get the same results for many functions that they port. And if they really care about having more than a few digits of precision, they either care about these differences, or really really should care. Hiding them in many but not all cases doesn't seem to be doing them a service. A simpler solution might be to just only support a fixed precision of, say, 8 digits. Then you can handle float inputs without running into that one-too-few 3's problem you mentioned. But obviously this isn't great either--you'd just end up with users who say "your library is perfect for me, except that I need 10 digits, not 8"... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/718428b6/attachment.html> From oscar.j.benjamin at gmail.com Sat Mar 8 23:36:49 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 8 Mar 2014 22:36:49 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <39b21e1b-cfc0-4235-bd9e-313f6d4ecb7d@googlegroups.com> <CAPTjJmqNHn7dhbzpvgtPUsNYVu4aeigpSSaGphmOY_aEGANriA@mail.gmail.com> Message-ID: <CAHVvXxTU211VAv5Nf9j2d9wWfPDnkioXs8Ftt1dZ+iDTq0zu6Q@mail.gmail.com> On 8 March 2014 20:12, Chris Angelico <rosuav at gmail.com> wrote: > > It's probably time someone [1] wrote up a PEP about all this. The most > important points, as I see it, are: > > 1) Create a new Decimal literal notation - 1.234d seems to be the most > popular syntax. This is reasonably uncontroversial, but it has > consequences. I feel like I've boxed myself into the corner by arguing that *someone* should do this. :) Unless someone else especially wants the mantle I'll try to write a PEP about decimal literals (I won't have any real time until Monday though). Oscar From tjreedy at udel.edu Sun Mar 9 00:16:52 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 08 Mar 2014 18:16:52 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140307101235.GR28804@ando> References: <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> Message-ID: <lfg8d6$jal$1@ger.gmane.org> On 3/7/2014 5:12 AM, Steven D'Aprano wrote: > For example, between 1 and 100000, about 12% of integer-valued floats > fail the "1/(1/n) == n" test, but over 51% of the integer-valued > Decimals: > > py> from decimal import Decimal as D > py> sum(1/(1/n) != n for n in range(1, 100001)) > 11846 > py> sum(1/(1/D(n)) != n for n in range(1, 100001)) > 51665 > > > Likewise we can test how many fail the "(1/n)*n == 1" test: > > py> sum((1/n)*n != 1 for n in range(1, 100001)) > 13117 > py> sum((1/D(n))*n != 1 for n in range(1, 100001)) > 36806 > > 13% for floats versus 36% for Decimals. > > > One more to prove it isn't a fluke: the "sqrt(n)**2 == n" test: > > py> sum((n**0.5)**2 != n for n in range(1, 100001)) > 49544 > py> sum((n**D("0.5"))**2 != n for n in range(1, 100001)) > 71303 > > > That's three for three in favour of binary floats. Thank you for these concrete examples. Yesterday I posted that binary floats are better because they are smoother, so that the relative approximation is more constant. I would not be surprised if this is behind the numbers above. In any case, the above convince me more strongly that we should stay with binary floats as default. -- Terry Jan Reedy From tim.peters at gmail.com Sun Mar 9 00:20:39 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 8 Mar 2014 17:20:39 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> Message-ID: <CAExdVNmLTi9SurzzHAJZ50x7FTw0guRcskCb3VFpb+uv5Y2XTw@mail.gmail.com> [Mark Dickinson] >> Python's decimal module is based on Cowlishaw's >> standard, not on IEEE 754. [Guido] > I wonder what Cowlishaw would say about our current discussion. He is also > the father of Rexx... I _expect_ Mike likes the status quo. Explaining why by analogy: I'm certain the 754 designers would not like the status quo. To them string<->number conversions were "operations", no different in principle than the operations of, say, addition or root extraction. And _all_ operations in 754 treat the input(s) as infinitely precise, and produce an output correctly rounded according to the current context. Now that's technically not so for float->string operations in 754, but that's an irrelevant distraction (754 allowed for weaker rounding guarantees in that specific context because nobody knows how to achieve correct rounding efficiently in all cases in that context - and the members of the 754 committee later said they regretted allowing this exception). Bottom line here: for all the 754 designers, and regardless of the type of x, Decimal(x) should accept x as exactly correct and produce a decimal object correctly rounded according to the current context (including setting context flags appropriately - e.g., signaling the inexact exception if any information in the input was lost due to rounding). Now the specific instance of this Mike did pronounce on is Decimal(string). It's obvious that the 754 view is that the assumed-to-be infinitely precise string literal be rounded to the current context's precision (etc). But Mike (in email at the time) explicitly wanted to retain all the string digits in the returned decimal object, wholly ignoring the current context. So that's what Python does. Is Decimal(0.1) really different? The 754 view, once you're used to it, is utterly predictable: Whatever the internal representation of 0.1, it's assumed to be infinitely precise, and you round its value to the decimal context's current precision. Mike's view is usually the same, but in one specific case of construction he thought differently. My guess is that he'd choose to be consistent with "the rule" for construction, having made an exception for it once already, than choose to be consistent with the other decimal operations. Or we could hark back to REXX's desire to present an arithmetic that works the way people learned in school. I'd try that, except I'm not sure kids today learn arithmetic in school any more beyond learning how to push buttons ;-) From cfkaran2 at gmail.com Sun Mar 9 00:24:06 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 8 Mar 2014 18:24:06 -0500 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <5317FDB6.5000806@canterbury.ac.nz> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> Message-ID: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> I've written some sample code that can do the type of annotations that I'm suggesting; is there a particular way that code is shared on the list, or should I just put it on the web somewhere? On Mar 5, 2014, at 11:46 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > CFK wrote: >> then it shouldn't be too hard for a project to add its own UUID to the annotation dictionary. > > I can't see what benefit there is in bringing UIIDs into > the picture. > > If you're suggesting that people write the UUIDs directly > into their code as literals, that's *not* going to fly. > It would be appallingly ugly and error-prone. > > The only way to make it usable would be to import the > UUIDs from somewhere under human-readable names. But then > there's no need to use UUIDs, any set of unique sentinel > objects will do. > > -- > Greg There are a number of reasons I'm suggesting UUIDs instead of simple strings: - Since they are truly unique, and since the hex string doesn't form words in any human language, you can enter the string in your favorite search engine to get more information about the string. The probability of false positives is almost 0, which is something that a string like 'doc' simply won't give you. - They don't require any coordination between projects. That means you don't have to worry that the name you choose for your project conflicts with a name that someone else chooses. You also don't have to worry about changing the name of your project; UUIDs don't carry any inherent meaning to people so you can keep them even when your project changes. - They have fixed lengths and formats. Annotations have been around for a while, which means that any automated tooling is going to have to find a way to deal with a new convention. If a parser can convert all the keys into UUID instances, then that gives it a high confidence that the code is using this convention. This is much more difficult to do when the universe of keys is more or less arbitrary. As for being ugly and error prone, I agree that putting it all in by hand would be horrible. The code that I wrote demonstrates a different way, which only requires decorators. It is basically a library that does the following: import annotizer import uuid # Load your uuid in some manner into project_ID. project_annotizer = annotizer.annotizer(ID) # From now on, you can use project_annotizer's decorators everywhere. @project_annotizer.parameter_decorator(a="blah") def func(a): pass The example code I've written doesn't use strings, it uses uuid.UUID instances directly, but that is easily changed. With some work, it can be extended for a large variety of useful annotations, all while hiding the complexities of dealing with UUIDs completely. Thanks, Cem Karan From harrismh777 at gmail.com Sat Mar 8 21:23:23 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 12:23:23 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> Message-ID: <9de4fcb4-9544-422f-abd5-a477aee5a691@googlegroups.com> On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote: > y=x.__round__(15) <====== how to set the round within context so > that works for float literals & decimals > with localcontext(ctx=None) as cmngr: > {snip} > Is this possible..? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/84f6fa98/attachment.html> From alexander.belopolsky at gmail.com Sun Mar 9 00:36:18 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 8 Mar 2014 18:36:18 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> Message-ID: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> On Sat, Mar 8, 2014 at 11:54 AM, Guido van Rossum <guido at python.org> wrote: > > I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman. > > On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >> >> On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >>> >>> >>> I see three sane options for float to Decimal conversion: >>> >>> 1. Raise an exception. >>> 2. Round to the nearest Decimal value using the current context for that round operation. >>> 3. Do what we're currently doing, and do an exact conversion. > > > I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself an expert at all, but I do think I have something to bring to the table -- insight the experience of non-expert users. > This reminded me a discussion we had with Mark Dickinson on the bug tracker: "When a user is entering 0.6112295, she means 0.6112295, not 0x1.38f312b1b36bdp-1 or 0.61122949999999998116351207499974407255649566650390625 which are exact values of the underlying binary representation of 0.6112295." http://bugs.python.org/issue8860#msg108601 The topic was the conversion from binary floats to timedelta which is effectively a fixed point decimal with 6 decimal places after the point. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/535d4dd1/attachment.html> From rosuav at gmail.com Sun Mar 9 00:38:37 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 10:38:37 +1100 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> Message-ID: <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com> On Sun, Mar 9, 2014 at 10:24 AM, Cem Karan <cfkaran2 at gmail.com> wrote: > There are a number of reasons I'm suggesting UUIDs instead of simple strings: But against that is that they're extremely long. Most other places where long hex strings are used, humans won't use the full thing. With git and hg, revisions are identified by hash - but you can use a prefix, and in most repos, 6-8 hex digits will uniquely identify something. When humans look at SSH key fingerprints, how many actually read the whole thing? That's why randart was produced. Tell me, which of these is different? def f(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): pass def asdf(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): pass def hello_world(x:"f0e4c2f76c58916ec258f246851bea891d14d4247a2fc3e18694461b1816e13b"): pass def testing(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): pass def longer_function_name(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): pass As uniqueness guarantors, they're a bit unwieldy, and that makes them pretty unhelpful. ChrisA From harrismh777 at gmail.com Sun Mar 9 00:33:15 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 15:33:15 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> Message-ID: <a86e5574-21f4-48f3-a973-8f39c2371681@googlegroups.com> On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote: > So your goal is that if someone is a TI89 expert but a Python novice, they > can easily port a function from their TI89 to Python (or maybe even write a > TI89-like desktop calculator program) with your library? That doesn't seem > too unreasonable of a goal. > Your points are all mostly valid, taking all assumptions into account. Some of these folks actually have TI89's but don't really know how to use them yet. Some of them have TI84+, or TI83, some TI Nspire... some others. Mostly the comparison between the calc and python isn't the point. The point is using python to do the trig (or whatever) and along the way introducing programming. Its just one use case that is not too different than a thousand other "average" or naive user cases, where the folks are still on the learning curve but need to be introduced to computers and computer science as well as the maths. This will also be good for folks who know what they are doing and will now be able to do it just a little more efficiently. For folks who really know what they are doing spelling issues are not even a problem. So, we can help newbie/naive users without impacting advanced users like yourself. The bottom line is trying to eliminate as few surprises as possible, making the learning curve as easy and fun as possible. The point is not to make a calculator; rather to make a decimal floating point package for python that is flexible. Experienced users can use it for all kinds of purposes, and newbies can use it too. The experts on this list, like you, will have no problem using the package (if you want) and folks that are brand new to python will be able to use it too. At some point the decimal concepts will come up, and the decimal literal will come in very handy then. For folks using 3.3.4 or below obviously the caveats are going to need to be explained in the documentation... and my functions are going to need to take in floats and do something meaningful with them, as you saw in the square root routine. Its a little more hassle for me to code and maintain, but its less "surprise" impact on users will be worth it (esp for young wanna bee naivet? types). marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/bba8d014/attachment-0001.html> From tjreedy at udel.edu Sun Mar 9 00:48:50 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 08 Mar 2014 18:48:50 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> Message-ID: <lfga94$4tq$1@ger.gmane.org> On 3/7/2014 9:02 PM, Guido van Rossum wrote: > On Fri, Mar 7, 2014 at 5:05 PM, Steven D'Aprano > What is the "suggested change" here? If it's "default float literals to > Decimal" I agree. But if the suggestion is my "Decimal(<float>) should > be implemented as Decimal(repr(<float>))" I don't think most scientists > care (few of them use Decimal, they stay entirely in the realm of binary > floating point). As a scientist, I either would not care, or might like it if it reduced the pressure to switch to decimal floats. I would however, consider changing the proposal to producing the best n-digit decimal, where n is the current context precision, or at least the default number (greater than 17?). This might be easily done by making n a parameter instead of a constant in the current internal function used to produce repr(float). > Python's done 97% of the work for you: the Decimal module, and the cmd > module which makes building interactive command-line oriented > applications easy. You can even simulate the Python interactive > interpreter. Idle already does that. > In other words, if you don't want to convert from floats to Decimal, > there is *absolutely no reason* why you should. It might be possible to experiment with this in Idle. It already does some parsing of user input as typed in order to color code it, though that may not recognize float literals. If not, let float_lit = <re that recognizes float literals>. Then use re.sub to replace 'xx.yy', etc, with 'D("xx.yy")' before sending user input to the subprocess. When auto-decimal mode is set, display and send 'from decimal import Decimal as D' to the user-process and change the prompt to, for instance dec> ... -- Terry Jan Reedy From harrismh777 at gmail.com Sun Mar 9 00:44:25 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 15:44:25 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> Message-ID: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> On Saturday, March 8, 2014 5:36:18 PM UTC-6, Alexander Belopolsky wrote: > This reminded me a discussion we had with Mark Dickinson on the bug > tracker: "When a user is entering 0.6112295, she means 0.6112295, not > 0x1.38f312b1b36bdp-1 or > 0.61122949999999998116351207499974407255649566650390625 which are exact > values of the underlying binary representation of 0.6112295." > > http://bugs.python.org/issue8860#msg108601 > Yes, and I certainly don't want to start up the whole underlying discussion again, but very simply put, that's it ! There are two user issues: 1) maths should not give surprises 2) inputs should have a reasonable "expected" interpretation marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/d18d28dc/attachment.html> From greg.ewing at canterbury.ac.nz Sun Mar 9 01:05:53 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 09 Mar 2014 13:05:53 +1300 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> Message-ID: <531BB061.2060009@canterbury.ac.nz> Cem Karan wrote: > There are a number of reasons I'm suggesting UUIDs instead of simple strings: I'm not talking about strings, I'm talking about objects created and exported by the module defining the annotations, and compared by identity. The Python module namespace then ensures they have unique names within any given program. That's all you need, because there's no requirement to persist them from one program execution to another. -- Greg From alexander.belopolsky at gmail.com Sun Mar 9 01:06:39 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 8 Mar 2014 19:06:39 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> Message-ID: <CAP7h-xbh3FukXCBTPBX_-9AM_-1UUum0z4KXp8i7fwP2i2cEnw@mail.gmail.com> On Sat, Mar 8, 2014 at 6:44 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > There are two user issues: > > 1) maths should not give surprises > > 2) inputs should have a reasonable "expected" interpretation > I am with Mark H. on this. While I appreciate the theoretical underpinnings of the status quo, I am yet to see data stored in binary that did not originate from decimal at some point. I think 99.999% of all instances of 0x1.199999999999ap+0 in numerical data come from a conversion of 1.1 from decimal to float rather than an a result of a computation that is correct to 16+ decimal places. In a well-designed system, simple things should be simple and difficult things should be possible. Decimal(x) = Decimal(str(x)) is simple and what most users expect and ultimately need. Expert users will always have Decimal.from_float, contexts, round and other power tools at their disposal. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/3daf6b7b/attachment.html> From rosuav at gmail.com Sun Mar 9 01:07:21 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 11:07:21 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfga94$4tq$1@ger.gmane.org> References: <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <lfga94$4tq$1@ger.gmane.org> Message-ID: <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com> On Sun, Mar 9, 2014 at 10:48 AM, Terry Reedy <tjreedy at udel.edu> wrote: > It might be possible to experiment with this in Idle. It already does some > parsing of user input as typed in order to color code it, though that may > not recognize float literals. If not, let float_lit = <re that recognizes > float literals>. Then use re.sub to replace 'xx.yy', etc, with 'D("xx.yy")' > before sending user input to the subprocess. When auto-decimal mode is set, > display and send 'from decimal import Decimal as D' to the user-process and > change the prompt to, for instance I played with the possibility of an AST transform, but by the time it's compiled into an AST, it's too late - the string has already been floated. A regex would be a fairly hefty hammer, I think. ChrisA From harrismh777 at gmail.com Sat Mar 8 22:54:11 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 13:54:11 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> Message-ID: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote: It looks like you're trying to emulate a pocket calculator here. The > question is, why are you accepting floats in the first place if that's your > goal? > That's fair... if it were me, or you, I wouldn't. In fact, if it were just me, or just you, I don't think any of us would be talking about this--- I wouldn't even have brought it up. I am advocating for people--- I am going to say, " if you want just a tad more than double just to check things out, please use the decimal module and import pdeclib. These folks are what Guido called newbies, I'll call them naive users, and these folks are using a calculator (like the TI89) and they will be prone to keying in sqrt(2.345) / Its just easier than explaining to them (at this point) all the stuff they need to understand to use the functions by making sure they feed decimals to the function; either by quoting their numeric inputs, or even using a decimal literal, &c. They know what the functions do, and will want to use them to get more digits, but may not remember the rules to get the "correct" digits. So I'm advocating for them, to make it easier. If it were just me, I'd just make sure the functions got decimals and be done with it. If you look at my square root function, its really not right; but its closer. For instance if you give it 1/3 it still is only slightly better than before. I don't have all the junk after the 16 th digit, but, I only have .33333333 to 15 places. So, believe me, I know there is no one truly good answer here. Just trying to make it easier for the most people. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/8b8a914b/attachment.html> From harrismh777 at gmail.com Sun Mar 9 01:21:59 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 16:21:59 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> Message-ID: <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com> On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote: > > {snip} Many folks "play" on the interactive python terminal; and some python experienced users and developers forget this. For instance the QPython people did not provide a terminal or interactive python with their first release because they were focused on their product being a script reader and forgot that normal people experiment on the interactive console, for calculation, and other things. We had a discussion the other night about the decimal distribution (numerical digit distribution) of the digits 0-9 in the number PI. The conjecture is that as the number of digits increased the distribution evens out, and PI is for all intents and purposes a random number generator (for large numbers of digits). But what does it look like to users who have never seen it, and how hard is it to code up? Well, its two lines of python code, and inexperienced users can not believe that: below: >>> ================================ RESTART ================================ >>> from pdeclib import * >>> dscale(1010) 42 >>> pi=get_PI() >>> sPI=repr(pi)[:1002] >>> >>> for n in range(10): print(n, sPI.count(str(n))) 0 92 1 114 2 102 3 103 4 92 5 97 6 93 7 95 8 100 9 104 Well, there it is; the distribution of the digits of PI in the first 1002, precisely; all from the console. marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/92d2c7ee/attachment.html> From cfkaran2 at gmail.com Sun Mar 9 01:34:32 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 8 Mar 2014 19:34:32 -0500 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> <CAPTjJmpYEKxxyRzvB+=-TVgcmpCHuubY2kN_vkXxvC_yHFaCPg@mail.gmail.com> Message-ID: <970F7B20-6FA8-4B43-91F1-292440CF8FCB@gmail.com> On Mar 8, 2014, at 6:38 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Sun, Mar 9, 2014 at 10:24 AM, Cem Karan <cfkaran2 at gmail.com> wrote: >> There are a number of reasons I'm suggesting UUIDs instead of simple strings: > > But against that is that they're extremely long. Most other places > where long hex strings are used, humans won't use the full thing. With > git and hg, revisions are identified by hash - but you can use a > prefix, and in most repos, 6-8 hex digits will uniquely identify > something. When humans look at SSH key fingerprints, how many actually > read the whole thing? That's why randart was produced. Tell me, which > of these is different? > > def f(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): > pass > > def asdf(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): > pass > > def hello_world(x:"f0e4c2f76c58916ec258f246851bea891d14d4247a2fc3e18694461b1816e13b"): > pass > > def testing(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): > pass > > def longer_function_name(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"): > pass > > As uniqueness guarantors, they're a bit unwieldy, and that makes them > pretty unhelpful. Agreed... for human beings. However, the purpose isn't just for people, its for automated systems that use annotations in possibly incompatible ways. My original example was the problem of docstrings between projects like PLY (http://www.dabeaz.com/ply/) and Sphinx (http://sphinx-doc.org/). For this pair of projects, you must choose to either document your code, or use the parser; not both. My proposal circumvents the problem. Nick Coghlan suggested using decorators instead, which I implemented. In my test code (posted below), you can stack decorators, which completely hide the UUID from human eyes. If an automated tool requires the annotations, it can look for its own UUID. BTW, my apologies if the test code isn't 100% perfect; I threw it together to make sure that the decorator idea would work well. Also, I'm using actual UUID instances instead of strings; it was easier to see what I was doing that way, but would need to be changed in real code. Thanks, Cem Karan #!/usr/bin/env python # -*- coding: utf-8 -*- __docformat__ = "restructuredtext en" import inspect import uuid import pprint ############################################################################## ############################################################################## ### Helper classes ############################################################################## ############################################################################## class _parameter_decorator(object): """ This class adds a documentation string to the annotations of a function or class instance's parameter. You can add multiple documentation strings, or you can add one at a time. Note that it is intended to only be used by the annotizer class. """ def __init__(self, ID, *args, **kwargs): self._ID = ID self._args = args self._kwargs = kwargs # End of __init__() def __call__(self, f): sig = inspect.signature(f) func_args = {x for x in sig.parameters} decorator_args = {x for x in self._kwargs} keys = func_args.intersection(decorator_args) for key in keys: if key not in f.__annotations__: f.__annotations__[key] = dict() if self._ID not in f.__annotations__[key]: f.__annotations__[key][self._ID] = dict() f.__annotations__[key][self._ID]["doc"] = self._kwargs[key] return f # End of __call__() # End of class _parameter_decorator class _return_decorator(object): """ This class adds a documentation string to the annotations of a function or class instance's return value. """ def __init__(self, ID, *args, **kwargs): self._ID = ID self._args = args self._kwargs = kwargs # End of __init__() def __call__(self, f): sig = inspect.signature(f) key = 'return' if sig.return_annotation == inspect.Signature.empty: f.__annotations__[key] = dict() if self._ID not in f.__annotations__[key]: f.__annotations__[key][self._ID] = dict() f.__annotations__[key][self._ID]["doc"] = self._args[0] return f # End of __call__() # End of class _return_decorator ############################################################################## ############################################################################## ### annotizer ############################################################################## ############################################################################## class annotizer(object): """ This assists in making annotations by providing decorators that are aware of your project's internal `UUID <http://en.wikipedia.org/wiki/UUID>`_. Using this class will ensure that your annotations are separate from the annotations that others use, even if the keys they use are the same as your keys. Thus, projects **A** and **B** can both use the key ''doc'' while annotating the same function, without accidentally overwriting the other project's use. """ def __init__(self, ID, parameter_decorator_class=_parameter_decorator, return_decorator_class=_return_decorator): """ :param ID: This is your project's UUID. The easiest way to generate this is to use the `uuid <http://docs.python.org/3/library/uuid.html`_ module, and then store the ID somewhere convenient. :type ID: ``str`` that can be used to initialize a `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_ instance, or a `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_ instance """ if isinstance(ID, uuid.UUID): self._ID = ID else: self._ID = uuid.UUID(ID) self.parameter_decorator_class = parameter_decorator_class self.return_decorator_class = return_decorator_class # End of __init__() def ID(): doc = ("This is the ID of your project. It is a " + "`UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_" + "instance.") def fget(self): return self._ID return locals() ID = property(**ID()) def parameter_decorator_class(): doc = ("Instances of this class may be used to decorate") def fget(self): return self._parameter_decorator_class def fset(self, value): self._parameter_decorator_class = value def fdel(self): self._parameter_decorator_class = _parameter_decorator return locals() parameter_decorator_class = property(**parameter_decorator_class()) def return_decorator_class(): doc = "The return_decorator_class property." def fget(self): return self._return_decorator_class def fset(self, value): self._return_decorator_class = value def fdel(self): self._return_decorator_class = _return_decorator return locals() return_decorator_class = property(**return_decorator_class()) def parameter_decorator(self, *args, **kwargs): decorator = self.parameter_decorator_class(self.ID, *args, **kwargs) return decorator # End of parameter_decorator() def return_decorator(self, *args, **kwargs): decorator = self.return_decorator_class(self.ID, *args, **kwargs) return decorator # End of return_decorator() # End of class annotizer ############################################################################## ############################################################################## ### Main ############################################################################## ############################################################################## if __name__ == "__main__": ID1 = uuid.uuid1() an1 = annotizer(ID1) ID2 = uuid.uuid1() an2 = annotizer(ID2) @an1.parameter_decorator(a="a", b="b", c="c") @an2.parameter_decorator(a="A", b="B", c="C") @an1.return_decorator("Doesn't return anything of value") @an2.return_decorator("Does not return a value") def func(a,b,c): print("a = {0!s}, b = {1!s}, c = {2!s}".format(a,b,c)) pprint.pprint(func.__annotations__) The output is: $ ./annotizer.py {'a': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'a'}, UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'A'}}, 'b': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'b'}, UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'B'}}, 'c': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'c'}, UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'C'}}, 'return': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': "Doesn't return anything of value"}, UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'Does not return a value'}}} From harrismh777 at gmail.com Sun Mar 9 01:35:57 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 16:35:57 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <8D833E5B-E46E-4BBE-91BB-57FC0152040C@yahoo.com> <03ce9ea1-b4cd-41d7-8eca-43d4624b7926@googlegroups.com> Message-ID: <392731c6-4dfa-4cce-8289-bd9a61701ddd@googlegroups.com> On Saturday, March 8, 2014 6:21:59 PM UTC-6, Mark H. Harris wrote: Actually this is it, because I keep forgetting that repr(D) includes "Decimal " in the string, must use str() >>> from pdeclib import * >>> dscale(1010) 42 >>> pi=get_PI() >>> sPI=str(pi)[:1002] >>> for n in range(10): print(n, sPI.count(str(n))) 0 93 1 116 2 103 3 103 4 93 5 97 6 94 7 95 8 101 9 106 >>> Anyway, its just fun.... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/0da5ab2a/attachment.html> From steve at pearwood.info Sun Mar 9 01:39:25 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Mar 2014 11:39:25 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> Message-ID: <20140309003925.GC28804@ando> On Sat, Mar 08, 2014 at 03:44:25PM -0800, Mark H. Harris wrote: > There are two user issues: > > 1) maths should not give surprises > 2) inputs should have a reasonable "expected" interpretation Computer maths cannot fail to give surprises. py> x = 1/Decimal(3) py> sum([x, x, x]) == 1 False "No surprises" is simply an impossible request. -- Steven From steve at pearwood.info Sun Mar 9 01:44:36 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Mar 2014 11:44:36 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com> References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <lfga94$4tq$1@ger.gmane.org> <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com> Message-ID: <20140309004436.GD28804@ando> On Sun, Mar 09, 2014 at 11:07:21AM +1100, Chris Angelico wrote: > On Sun, Mar 9, 2014 at 10:48 AM, Terry Reedy <tjreedy at udel.edu> wrote: > > It might be possible to experiment with this in Idle. It already does some > > parsing of user input as typed in order to color code it, though that may > > not recognize float literals. If not, let float_lit = <re that recognizes > > float literals>. Then use re.sub to replace 'xx.yy', etc, with 'D("xx.yy")' > > before sending user input to the subprocess. When auto-decimal mode is set, > > display and send 'from decimal import Decimal as D' to the user-process and > > change the prompt to, for instance > > I played with the possibility of an AST transform, but by the time > it's compiled into an AST, it's too late - the string has already been > floated. A regex would be a fairly hefty hammer, I think. Great minds think alike :-) An AST transformation is possible: import ast class MyTransformer(ast.NodeTransformer): def visit_Num(self, node): # Transform float literals to Decimal("literal"). if isinstance(node.n, float): quoted_literal = ast.Str(repr(node.n)) new_node =ast.Call( func=ast.Name(id='Decimal', ctx=ast.Load()), args=[quoted_literal], ) return ast.copy_location(new_node, node) return node node = ast.parse('2.5 + 5**0.5', mode='eval') print("Before:") ast.dump(node) print("After:") ast.dump(MyTransformer().generic_visit(node)) I tried to find some way to hook that transformation into the code module, so as to give an intereactive interpreter with decimal floats, but couldn't work out how to do so. -- Steven From rosuav at gmail.com Sun Mar 9 02:46:53 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 12:46:53 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309004436.GD28804@ando> References: <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <20140307101235.GR28804@ando> <95e9f9da-876a-474a-8c80-433227e34d3b@googlegroups.com> <20140308010526.GV28804@ando> <CAP7+vJ+7ipzS2Zo2g-GESbwA5M3mzd3OiYmVwKyjZCx47gbapg@mail.gmail.com> <lfga94$4tq$1@ger.gmane.org> <CAPTjJmryfnC0Dyn2mpReJxioB4F99NuYbFq6K4+f4XyeOyH9jw@mail.gmail.com> <20140309004436.GD28804@ando> Message-ID: <CAPTjJmqHHfuvf0zExHOKQ3ODs9zMhKGMwJ9H8-tJQi=SkOq=rg@mail.gmail.com> On Sun, Mar 9, 2014 at 11:44 AM, Steven D'Aprano <steve at pearwood.info> wrote: > Great minds think alike :-) > > An AST transformation is possible: > > > import ast > > class MyTransformer(ast.NodeTransformer): > def visit_Num(self, node): > # Transform float literals to Decimal("literal"). > if isinstance(node.n, float): > quoted_literal = ast.Str(repr(node.n)) Somewhat possible, but it's kinda too late at that point; you've already gone via float. It's of some use, perhaps, but it doesn't actually let you "do more". I was hoping to get the source string before it got floated, and to make a decimal out of it instead. But by that time, it's already actually become a float, and the AST node.n is, as you demonstrate here, an actual float. I tried substituting in a Decimal instead of building up a call to Decimal("..."), but compile() won't touch it: >>> ast.dump(node) "Expression(body=BinOp(left=Num(n=Decimal('2.5')), op=Add(), right=BinOp(left=Num(n=5), op=Pow(), right=Num(n=Decimal('0.5')))))" >>> compile(node,"<...>","eval") Traceback (most recent call last): File "<pyshell#15>", line 1, in <module> compile(node,"<...>","eval") TypeError: non-numeric type in Num Awwwww :( ChrisA From cfkaran2 at gmail.com Sun Mar 9 02:54:07 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 8 Mar 2014 20:54:07 -0500 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <531BB061.2060009@canterbury.ac.nz> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> <531BB061.2060009@canterbury.ac.nz> Message-ID: <33D3ACF7-35BB-4AC7-9843-F680A089A547@gmail.com> On Mar 8, 2014, at 7:05 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Cem Karan wrote: >> There are a number of reasons I'm suggesting UUIDs instead of simple strings: > > I'm not talking about strings, I'm talking about > objects created and exported by the module defining > the annotations, and compared by identity. > > The Python module namespace then ensures they have > unique names within any given program. That's all > you need, because there's no requirement to persist > them from one program execution to another. > > -- > Greg I see your point, and I think you're right. I can see a lot of really good additional tricks we can play (adding in metadata to the sentinel, etc.). I've spent the time since I saw this message trying to come up with good counterarguments and I honestly can't come up with any good ones. I can only think of two counterarguments, both of which are really weak. First, if you want to mock the module, you don't need the module's sentinel; you just need the UUID, and then you never need to load the module at all. Second, there are cases where you want to warn people that although the signature of a function has remained the same, the semantics have shifted. In this case, a module user might want to key off of the UUID. If the ID changes, then the meaning has shifted. Both of those counterarguments are really weak though, so unless anyone can think of a reason NOT to use sentinels as Greg suggests, I'd like to shift my proposal to using sentinels instead of UUIDs. Thanks, Cem Karan From harrismh777 at gmail.com Sun Mar 9 02:58:18 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 17:58:18 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309003925.GC28804@ando> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> Message-ID: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> On Saturday, March 8, 2014 6:39:25 PM UTC-6, Steven D'Aprano wrote: > py> x = 1/Decimal(3) > py> sum([x, x, x]) == 1 > False > > "No surprises" is simply an impossible request. > hi Steven, that's a straw man. Your argument presumes that False is a surprise. I am not in the least surprised. As, I am also not surprised by this: >>> dscale(32) 10100 >>> d(1)/3 Decimal('0.33333333333333333333333333333333') <=== not a surprise, expected >>> d(1)/3 + d(1)/3 +d(1)/3 Decimal('0.99999999999999999999999999999999') <=== not a surprise either, quite expected >>> And following: >>> dscale(42) 32 >>> dSum= d(1)/3 + d(1)/3 +d(1)/3 >>> dscale(38) 42 >>> +dSum <===== at this point I expect to see the following line, just as I see it./ Decimal('1.0000000000000000000000000000000000000') >>> Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/81b05f1b/attachment-0001.html> From alexander.belopolsky at gmail.com Sun Mar 9 03:12:53 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 8 Mar 2014 21:12:53 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309003925.GC28804@ando> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> Message-ID: <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> On Sat, Mar 8, 2014 at 7:39 PM, Steven D'Aprano <steve at pearwood.info> wrote: > Computer maths cannot fail to give surprises. > > py> x = 1/Decimal(3) > py> sum([x, x, x]) == 1 > False > > "No surprises" is simply an impossible request. > Maybe, but there is nothing impossible in eliminating this particular surprise: q)x:1 % 3f q)x 0.3333333 q)1 = x + x + x 1b http://en.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems) I am not suggesting that Python should behave like Q in this respect, just showing that this particular wart is not present in all languages. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/ce74a2d7/attachment.html> From tim.peters at gmail.com Sun Mar 9 03:19:54 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 8 Mar 2014 20:19:54 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> Message-ID: <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com> [Oscar Benjamin <oscar.j.benjamin at gmail.com>] > ... > I don't believe that a standards-compliant decimal module > has any wiggle room to invent a new kind of rounding (which I think > would be required to achieve what you suggest). That would be true if the standard explicitly forbade extensions (any optional operation or behavior not specified by the standard). But there are no standards like that ;-) To comply with a standard, there just needs to be _some_ way to "spell" all the behaviors the standard mandates. For example, in a programming language no relevant 754-ish standard says that the infix binary operator "*" has to implement 754-ish multiplication. So far as standard compliance goes, "*" could mean "ignore the second argument and add 1.7 to the first argument, rounding toward 7" - just so long as there's *some* way to spell "754-ish multiplication". Python (or any other implementation) is quite free to invent all the rounding modes (or any other kinds of operations, or modifications to standard behaviors) it likes. That kind of experimentation is where ideas for the _next_ iteration of the standard comes from. From harrismh777 at gmail.com Sun Mar 9 03:26:51 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 18:26:51 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com> Message-ID: <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com> On Saturday, March 8, 2014 8:19:54 PM UTC-6, Tim Peters wrote That would be true if the standard explicitly forbade extensions (any > optional operation or behavior not specified by the standard). But > there are no standards like that ;-) To comply with a standard, there > just needs to be _some_ way to "spell" all the behaviors the standard > mandates. The IEEE 854 1987 is just such a standard (Mike Cowlishaw's extension). The generalized IEEE 754 2008 standard might not have as much wiggly room. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/7c2ca224/attachment.html> From rosuav at gmail.com Sun Mar 9 03:46:21 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Mar 2014 13:46:21 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> Message-ID: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> On Sun, Mar 9, 2014 at 12:58 PM, Mark H. Harris <harrismh777 at gmail.com> wrote: > hi Steven, that's a straw man. Your argument presumes that False is a > surprise. I am > not in the least surprised. As, I am also not surprised by this: > >>>> dscale(32) > 10100 >>>> d(1)/3 > Decimal('0.33333333333333333333333333333333') <=== not a surprise, > expected >>>> d(1)/3 + d(1)/3 +d(1)/3 > Decimal('0.99999999999999999999999999999999') <=== not a surprise either, > quite expected >>>> I was trying to construct a proof from memory that would support this case, but I got tangled, so I cheated and went to Wikipedia for something similar. Let's start with two variables, a and b, which we shall suppose are equal. # Postulation? Or is this called an axiom? I don't remember. a = b # Wave hands vigorously and conclude that: a = 0 Are you surprised? Probably. Normally, two variables being equal doesn't prove the value of either. Without knowing what actual handwaving went on in there, that is and should be a surprising result. Now, here's the handwaving, spelled out in full: # As above, start here a = b # Multiply both sides by a aa = ba # Subtract bb (aka b squared, ASCII-friendly) from both sides aa - bb = ba - bb # Factor (both sides separately) (a + b) (a - b) = b (a - b) # Cancel out the multiplication by (a - b) a + b = b # Subtract b from both sides a = 0 And there you are. To be unsurprised by the result, you have to know exactly what happened in between, and how it's different from the conventional rules of mathematics. Someone who already understands all that may well not be surprised, but someone who's expecting sane real-number arithmetic is in for a shock. ChrisA From tim.peters at gmail.com Sun Mar 9 03:48:18 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 8 Mar 2014 20:48:18 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAExdVNmBjYFYjrS1p8mUjAZWaYGEGXe-Ker8sfnZnjCtU_dd4Q@mail.gmail.com> <5b588633-2df6-4795-a82b-145ef5c0268d@googlegroups.com> Message-ID: <CAExdVNmw2zDZ3ghzSXAgNSST66xLF-j57j+4sYR2zJfPRyn6Qw@mail.gmail.com> [Tim] >> That would be true if the standard explicitly forbade extensions (any >> optional operation or behavior not specified by the standard). But >> there are no standards like that ;-) To comply with a standard, there >> just needs to be _some_ way to "spell" all the behaviors the standard >> mandates. [Mark H. Harris] > The IEEE 854 1987 is just such a standard (Mike Cowlishaw's extension). > > The generalized IEEE 754 2008 standard might not have as much wiggly > room. I have the 2008 version of the standard; it's thoroughly ordinary in these respects. As usual, it's not even "a language" that's normally said to be compliant with the standard. Instead: Conformance to this standard is a property of a specific implementation of a specific programming environment, rather than of a language specification. Although: However a language standard could also be said to conform to this standard if it were constructed so that every conforming implementation of that language also conformed automatically to this standard. So far as languages go, there's very little a standard of this nature _can_ say! Languages vary enormously in syntax and conventions. Indeed, for example, the standard "spells" multiplication as multiplication(x, y) That doesn't mean a conforming implementation has to supply a 2-argument function named "multiplication". It just means a conforming implementation has to supply _some_ way to spell what the standard defines via "multiplication(x, y)". Etc. From harrismh777 at gmail.com Sun Mar 9 04:05:04 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 19:05:04 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> Message-ID: <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com> On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote: > # As above, start here > a = b > # Multiply both sides by a > aa = ba > # Subtract bb (aka b squared, ASCII-friendly) from both sides > aa - bb = ba - bb > # Factor (both sides separately) > (a + b) (a - b) = b (a - b) > # Cancel out the multiplication by (a - b) <======= if you divide by > zero, the universe will explode/ > a + b = b > # Subtract b from both sides > a = 0 > I found a proof once that proved that 1==2, but I can't find it... let you know. Anyway (laughs) thanks for the grins marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/e3bd52f5/attachment.html> From steve at pearwood.info Sun Mar 9 04:15:16 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Mar 2014 14:15:16 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> Message-ID: <20140309031516.GF28804@ando> On Sat, Mar 08, 2014 at 09:12:53PM -0500, Alexander Belopolsky wrote: > On Sat, Mar 8, 2014 at 7:39 PM, Steven D'Aprano <steve at pearwood.info> wrote: > > > Computer maths cannot fail to give surprises. > > > > py> x = 1/Decimal(3) > > py> sum([x, x, x]) == 1 > > False > > > > "No surprises" is simply an impossible request. > > > > Maybe, but there is nothing impossible in eliminating this particular > surprise: Of course not. Numeric surprises are like bubbles under wallpaper: you can push them around, move them from place to place, but you cannot eliminate them entirely. (The analogy is not perfect: with wallpaper, you can push the bubble out to the edge, and at least sometimes, eliminate it.) By using Decimal, you eliminate one class of surprises ("why does my number not equal the decimal digits I typed in?") but at the cost of creating new surprises ("why is the average of x and y not always between x and y?"). > q)x:1 % 3f > q)x > 0.3333333 > q)1 = x + x + x > 1b If I were to write: y:0.3333333 I'd expect y to equal x, since that's what x looks like. If it didn't, that would be a surprise. If it did, that would be a surprise too, because when I went to school, adding y three times ought to get 0.9999999 and not 1. Since Q is based on APL, I'm guessing that the surprise here is actually in the = equals operator. Is it doing a fuzzy comparison? If so, that's also surprising, since that means you can have triplets of numbers such that a = b and b = c but a != c. > http://en.wikipedia.org/wiki/Q_(programming_language_from_Kx_Systems) > > I am not suggesting that Python should behave like Q in this respect, just > showing that this particular wart is not present in all languages. -- Steven From tim.peters at gmail.com Sun Mar 9 04:21:15 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sat, 8 Mar 2014 21:21:15 -0600 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com> Message-ID: <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com> [Mark H. Harris] > I found a proof once that proved that 1==2, but I can't find it... let you > know. Dead easy: >>> 1e100 + 1 == 1e100 + 2 True Now just subtract 1e100 from both sides ;-) From alexander.belopolsky at gmail.com Sun Mar 9 04:41:14 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 8 Mar 2014 22:41:14 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309031516.GF28804@ando> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> <20140309031516.GF28804@ando> Message-ID: <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com> On Sat, Mar 8, 2014 at 10:15 PM, Steven D'Aprano <steve at pearwood.info>wrote: > Of course not. Numeric surprises are like bubbles under wallpaper: you > can push them around, move them from place to place, but you cannot > eliminate them entirely. (The analogy is not perfect: with wallpaper, > you can push the bubble out to the edge, and at least sometimes, > eliminate it.) > Great analogy! I'll start using it with wallpaper covering floor, ceiling as well as all four walls. > > Since Q is based on APL, I'm guessing that the surprise here is > actually in the = equals operator. Is it doing a fuzzy comparison? > Exactly. > If so, that's also surprising, since that means you can have triplets of > numbers such that a = b and b = c but a != c. > Yes, but such triplets are not as easy to stumble upon as 1/3 + 1/3 + 1/3 and lack of transitivity is not something that novices are likely to complain about. You are absolutely right about not being able to eliminate all bubbles, but we can move them away from the center of the wall to the corners and under the wall-pictures. In my view >>> Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') is a big bubble right in the middle of the wall. The most obvious conversion from float to Decimal should implement the most commonly needed operation. I cannot imagine anyone needing to convert floats to 53-digit decimals in a real life application. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/30480e3a/attachment.html> From harrismh777 at gmail.com Sun Mar 9 04:33:17 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 19:33:17 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <a2a43b56-d9f3-402b-a25f-5070300a8ae7@googlegroups.com> <CAExdVNnzjvarFG-YvPgsWbQ+Kof_eSGYh9SQqca0yxi8_PveAw@mail.gmail.com> Message-ID: <b3963d58-6321-4b9c-bbcc-0a8637901ef9@googlegroups.com> On Saturday, March 8, 2014 9:21:15 PM UTC-6, Tim Peters wrote: > >>> 1e100 + 1 == 1e100 + 2 > True > > Now just subtract 1e100 from both sides ;-) ~nice. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/fd7e546c/attachment-0001.html> From harrismh777 at gmail.com Sun Mar 9 04:48:47 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 19:48:47 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> <20140309031516.GF28804@ando> <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com> Message-ID: <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com> On Saturday, March 8, 2014 9:41:14 PM UTC-6, Alexander Belopolsky wrote: > > In my view > > >>> Decimal(1.1) > Decimal('1.100000000000000088817841970012523233890533447265625') > > is a big bubble right in the middle of the wall. > > The mos > hi Alexander, um, no... that's a hole in the plaster, with a mouse chewing his way through the wallpaper... ;-) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/2c4ef05e/attachment.html> From harrismh777 at gmail.com Sun Mar 9 04:27:02 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 19:27:02 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> Message-ID: <b0f98886-5a26-4355-8f9d-9046fca309c2@googlegroups.com> On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote: > > # As above, start here > a = b > # Multiply both sides by a > aa = ba > # Subtract bb (aka b squared, ASCII-friendly) from both sides > aa - bb = ba - bb > # Factor (both sides separately) > (a + b) (a - b) = b (a - b) <========of course all you really > proved is that zero == zero > # Cancel out the multiplication by (a - b) > a + b = b > # Subtract b from both sides > a = 0 But then again, it really is a straw man (on steroids) because nobody is really surprised by the absurdity postulate: a=b {magic} conclusion: a=0 At least no one should be surprised by an absurdity. On the other hand once they see the proof run by them, um, quickly--- slight of hand, as it were, they might be surprised if they don't catch it right away. {still laughing} love it -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/75f0434a/attachment.html> From alexander.belopolsky at gmail.com Sun Mar 9 05:06:04 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 8 Mar 2014 23:06:04 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> <20140309031516.GF28804@ando> <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com> <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com> Message-ID: <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com> On Sat, Mar 8, 2014 at 10:48 PM, Mark H. Harris <harrismh777 at gmail.com>wrote: > > > hi Alexander, um, no... that's a hole in the plaster, with a mouse > chewing > his way through the wallpaper... > > > .. and what would you call this? >>> int(6.02e23) 601999999999999995805696 http://en.wikipedia.org/wiki/Avogadro_constant -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/672994fe/attachment.html> From steve at pearwood.info Sun Mar 9 06:02:07 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Mar 2014 16:02:07 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> Message-ID: <20140309050207.GG28804@ando> On Sat, Mar 08, 2014 at 01:54:11PM -0800, Mark H. Harris wrote: > I am advocating for people--- I am going to say, " if you want just a > tad more than double just to check things out, please use the decimal > module and import pdeclib. These folks are what Guido called newbies, > I'll call them naive users, and these folks are using a calculator > (like the TI89) and they will be prone to keying in sqrt(2.345) / When we get newbies asking why Python does not behave exactly like Java, we explain to them that Python is not Java and describe the differences. We do not try to make Python into a clone of Java. The same applies to newies who complain that Python does not behave exactly like Ruby, or PHP, or Perl, or Lisp. I don't think TI-89 should be the exception. I don't think it is a good idea to try to make a general purpose programming language like Python into a (mathematical) clone of the TI-89 calculator just to satisfy the tiny minority of users who are numerically naive TI-89 users. > Its just easier than explaining to them (at this point) all the stuff > they need to understand to use the functions When you put it like that, it sounds like you are arguing, not that it is better for the users to avoid learning about floating point maths, but that it's easier for *you* if you don't need to explain floating point maths. That's okay. Point them to the python-list or tutor mailing lists, and the regulars there will be more than happy to teach them, to the best of our abilities. > by making sure they feed decimals to the function; either by quoting > their numeric inputs, or even using a decimal literal, &c. They know > what the functions do, and will want to use them to get more digits, > but may not remember the rules to get the "correct" digits. So I'm > advocating for them, to make it easier. [...] > Just trying to make it easier for the most people. Who are these people you are trying to help? The user-base for Python is developers, and people trying to learn to be developers. Python is not an end-user application. Python is a programming language, and the people using it should be treated as if they were programmers. I think that there are very few non-programmer, numerically-naive, TI-89 users working with Python, and I think that shaping Python specifically for them is a mistake. It is completely appropriate to build an interactive calculator using Decimals aimed at the TI-89-using audience, and I applaud that. But that's an application written in Python, not Python itself. > If it were just me, I'd just make sure the functions got decimals and be > done with it. You would do that even though the majority of numeric users of Python would be opposed to such a change? -- Steven From harrismh777 at gmail.com Sun Mar 9 07:09:40 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 22:09:40 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309050207.GG28804@ando> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> Message-ID: <5b6ee23e-974c-4eec-aeb1-5dfaaf79130d@googlegroups.com> On Saturday, March 8, 2014 11:02:07 PM UTC-6, Steven D'Aprano wrote: I don't think TI-89 should be the exception. I don't think it is a good > idea to try to make a general purpose programming language like Python > into a (mathematical) clone of the TI-89 calculator just to satisfy the > tiny minority of users who are numerically naive TI-89 users. > hi Steven, yeah, in some ways I wish I'd never mentioned the TI89, because I've been misinterpreted (for which I am sorry). I mentioned the TI89 because it uses decimal floating points by default , I think (as I believe python should also). I mentioned the TI89 mainly as a "calculator" and that is the one that popped into my head... the reason for mentioning it is that folks who poke a number into a calculator don't quote the number, they don't add a "d" to the number, and they expect the calculator to interpret their number correctly. ( it was a usability reference ) > > > > Its just easier than explaining to them (at this point) all the stuff > > they need to understand to use the functions > > When you put it like that, it sounds like you are arguing, not that it > is better for the users to avoid learning about floating point maths, > but that it's easier for *you* if you don't need to explain floating > point maths. That's okay. Point them to the python-list or tutor mailing > lists, and the regulars there will be more than happy to teach them, to > the best of our abilities. > Well, its that too for sure, but its certainly more. Its a way for maths and other explanations to be brought on-line in stages (learning mods) that are progressive, and yet have the students be able to get the right results from the beginning and not to be frustrated. Again, a usability thing. > Who are these people you are trying to help? The user-base for Python is > developers, and people trying to learn to be developers. Python is not > an end-user application. Python is a programming language, and the > people using it should be treated as if they were programmers. > Unfortunately, I think that is the saddest thing I have heard you say in almost seven years of knowing you. The earliest beginnings of Python (ABC at CWI) was to "Stamp out BASIC" . BASIC was designed at Dartmouth in 1964 to help average users and other scientists to be able to use a computer WITHOUT having to be a developer (computer scientist, nor programmer). Python has the opportunity to be the BASIC of the 21st century (and beyond). Part of the reason Python has NOT had a greater general acceptance is that it is becoming a programming language for experts. This should not generally true, fortunately. I mean that Python is actually a VERY good language for newbies to start with, and its a VERY good language for average people and other scientists to interact with a computer in sophisticated ways WITHOUT having to be a computer scientist nor programmer. > > I think that there are very few non-programmer, numerically-naive, TI-89 > users working with Python, and I think that shaping Python specifically > for them is a mistake. It is completely appropriate to build an > interactive calculator using Decimals aimed at the TI-89-using audience, > and I applaud that. But that's an application written in Python, not > Python itself. > Yes, and no. Again, the TI89 has really nothing to do with it... I am advocating for general average usability of the language by the masses; whether they are software engineers or not, whether they are computer scientists or not, whether they are have a PhD in mathematics or not. Python should be a both | and; new folks should feel welcome and encouraged, and experienced folks should feel the power in their hands. Those ruby slippers should be the magic in the castle, and the way home for Dorothy. There are actually a large number of people trying out Python for the first time, and many professional people and academic (like Matlab or Mathematica users) who are switching to Python because of its power and flexibility (not to mention freedom). Both sets of "people" need to be welcomed and cared for. I honestly think that is one of the richest personal traits of the BDfL, and in my observation (of course he should speak for himself) he seems to advocate for a both |and (at least that is how he makes me feel about it, when I have heard him speak publicly). > If it were just me, I'd just make sure the functions got decimals and be > > done with it. > > You would do that even though the majority of numeric users of Python > would be opposed to such a change? > I am not sure I understand this question. All I'm saying is that as a programmer, I am willing to take on constraints (spelling, rules, syntax, heaps of knowledge) in stride (as a software engineer that's my job) which I do not feel average people and other scientists should have to worry about. Just like at Dartmouth in 1964... a physicist should be able to sit down to a python terminal and in very short order (with some small doc, tutorial) be able to be talking to Python rapidly. The experience should not be frustrating, the experience should in fact be as intuitive as possible, and it might also be fun... very seriously I believe this. Python is a computer language. But, it should not be C++ (we already have one of those), and it should not be Java (for heaven's sake, certainly not that) and it should not be GNU C nor Assembler, nor should it require a PhD in computer science to use it ! Python programming can and should be a rich and rewarding "liberal art" which may be learned by anyone (even children) to accomplish real world tasks with a rapid development cycle and a minimal of head banging (whatever I mean by that, its all debatable). I ran into this at IBM all the time too... sooner or later developers become arrogant and self-absorbed. IBM engineers are sometimes their own best customers, and their own worst enemies. I am hoping that does not happen in the Python community. Python community has a very rich and talented group of engineers, mathematicians, coders, coordinators, doc writers, pep writers, testers, and a super BDfL. It would be sad IMHO for the Python community to implode (that is, build a superstructure or play-box for developers, forgetting what they are developing, and for whom). This is my philosophy, and I don't expect that everyone will share it (that's just not realistic). I do hope that Python receives wider adoption in the future, and I do hope that more people (esp women, kids, young people) will choose computer science (through Python education) and that more people will actually reach out and and learn programming as a liberal art. I am sick frankly, of our point-and-grunt culture (here in the States, and elsewhere). Python is a way out of point-and-grunt, and a way forward into thinking with a computer as a liberal art--- and as fine art. It is the power of thinking in the 21st century, and it is the power of communication, collaboration, and cooperation between people and nations in the 21st century. I mean it. marcus > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/f5b8b8f3/attachment-0001.html> From python at 2sn.net Sun Mar 9 07:21:07 2014 From: python at 2sn.net (Alexander Heger) Date: Sun, 9 Mar 2014 17:21:07 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309050207.GG28804@ando> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> Message-ID: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> > I think that there are very few non-programmer, numerically-naive, TI-89 > users working with Python, students. For numerical applications there is Numpy. > and I think that shaping Python specifically > for them is a mistake. It is completely appropriate to build an > interactive calculator using Decimals aimed at the TI-89-using audience, > and I applaud that. But that's an application written in Python, not > Python itself. The decimal module really should only be used for financial applications; there should be a separate high-precision float type. Another note on the 'd' notation - this might be confused with the FORTRAN 'd' for double precision number. Would you write, 1.2e3d or 1.2d3? -Alexander From harrismh777 at gmail.com Sun Mar 9 07:57:27 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 22:57:27 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <CAP7h-xaQM8OLyEmc28EatXhQQgu-i6PFX-Dz49XPwG+MvgnnfQ@mail.gmail.com> <20140309031516.GF28804@ando> <CAP7h-xZd4ERZTEK2m776kjZ6dcvXxirVyxaQCV2FJViPCL2BcA@mail.gmail.com> <cc5a5633-2a6b-440b-92f4-25533c86d135@googlegroups.com> <CAP7h-xYA4ejDNHagUxPWKR3s0M+w6cpW4k5eZtv+_m5HadU7ig@mail.gmail.com> Message-ID: <d769697b-a5fe-418a-a985-ba89f663da60@googlegroups.com> On Saturday, March 8, 2014 10:06:04 PM UTC-6, Alexander Belopolsky wrote .. and what would you call this? > > >>> int(6.02e23) > 601999999999999995805696 <===== a big fat surprise, and WAY not > enough atoms / mole :) > Well, the number of atoms|molecules in one mole of gas at standard temperature and pressure is actually more precisely 6.02214129x10^23 (depending on who you talk to) However, back in the day--- Chem 201 UMKC 1975, I set the cursor of my dual base log log slide rule (Pickett) just to the right of 6 and then kept track of the decimal place myself in my notebook... ;-) ... but now I have python: whoohoo! >>> d(6.02214129e23) Decimal('6.02214129E+23') >>> av=d(6.02214129e23) >>> int(av) 602214129000000000000000 <===== yousah like this better? >>> Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/45ca7942/attachment.html> From harrismh777 at gmail.com Sun Mar 9 08:10:41 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 23:10:41 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> Message-ID: <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com> On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote: > > {snip} hi Alexander, Steve, this is one of Mike Cowlishaw's papers regarding decimal floating point, hardware decimal floating point, and the IEEE 854 1987 standard, and other topics. He mentions the TI89, and others, that use decimal floating point arithmetic, with citations. http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf Its a good read. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/c7cdbd3d/attachment.html> From harrismh777 at gmail.com Sun Mar 9 07:35:00 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sat, 8 Mar 2014 22:35:00 -0800 (PST) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> Message-ID: <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com> On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote: > The decimal module really should only be used for financial > applications; there should be a separate high-precision float type. > No, no, no.... It may and should be used for financial calculations, so true, but it may ALSO be used for a general purpose high precision (and) arbitrary precision decimal floating point. Otherwise it might as well have been patterned after the fixed width standard. It was patterned after Cowlishaw's extension. > > Why do you say only financial? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140308/16b5b976/attachment.html> From steve at pearwood.info Sun Mar 9 08:32:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 9 Mar 2014 18:32:29 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <531B92C4.3040502@stoneleaf.us> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> Message-ID: <20140309073229.GJ28804@ando> On Sat, Mar 08, 2014 at 01:59:32PM -0800, Ethan Furman wrote: > Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) > [GCC 4.7.3] on linux > Type "help", "copyright", "credits" or "license" for more information. > --> from decimal import Decimal as D > --> 9017.0109812864350067128347806 > 9017.010981286436 > --> D(9017.0109812864350067128347806) > Decimal('9017.01098128643570817075669765472412109375') > > In case my question isn't obvious, the direct float got me 16 digits, while > the Decimal float got me 42 digits. Why is the Decimal more "accurate" > that the float it came from? That's an interesting question. It does appear to be a lot more digits than needed. Here's that float exactly, in hex: py> 9017.010981286436.hex() '0x1.19c8167d5b50ep+13' Let's convert it to decimal: py> digits = list('19c8167d5b50e') py> for i, c in enumerate(digits): ... digits[i] = '0123456789abcdef'.index(c) ... py> d = Decimal(0) py> for n in digits[::-1]: ... d += n ... d /= 16 ... py> d += 1 py> d *= 2**13 py> d Decimal('9017.010981286435708170756694') Note that this round-trips back to float correctly: py> assert float(d) == 9017.010981286436 py> So I must admit, I'm not sure where the extra digits come from: Decimal('9017.01098128643570817075669765472412109375') Decimal('9017.010981286435708170756694') but perhaps my calculation was rather naive. -- Steven From python at 2sn.net Sun Mar 9 10:37:32 2014 From: python at 2sn.net (Alexander Heger) Date: Sun, 9 Mar 2014 20:37:32 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> <ef58402f-599e-4fe3-baae-1a2a5e9ac006@googlegroups.com> Message-ID: <CAN3CYHz-L5fKy2D7pZ4BAJw=h_89EW+KC9TF_jmCaC6OBkW6kA@mail.gmail.com> >> Why do you say only financial? Arbitrary precision is good - but why should it be done in a way that is not natural to how machines work? But I admit, I have not looked at the code how it is implemented. Maybe it is done well (only saving integer part and an exponent). I only recall the BCD standard used in Turbo Pascal - which was not so great - and really mostly for financial use. But there is no point in having fractions in decimal for computation, unless the result has to have a good representation in a specific base. Something you can send to a tax office. The decimal package solves problems specific to financial. And, yes, it can be used for other applications lacking other arbitrary precision classes in Python standard class library - but it would not be my first choice for other applications. Hence I propose to have a different arbitrary precision package for applications that do not require 'decimal' precision. As for class hierarchy it likely should be a base class for the decimal package. (Then you could even add other bases as well.) From p.f.moore at gmail.com Sun Mar 9 13:24:41 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 9 Mar 2014 12:24:41 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <20140305135646.GB28804@ando> <d3cccc11-a712-4861-92d7-7db95f7b8abb@googlegroups.com> <41b59536-c503-4105-a89b-61cbaf493dc1@googlegroups.com> <CAP7+vJJHkGGmeTtFrYJuQkOTTFw7PPTp5m2Gy-DrJRT69mfkOw@mail.gmail.com> <3452d513-0f5a-4365-96c3-c593263daa7d@googlegroups.com> <CAP7+vJ+OFp7XBJN4C4DW3CopTaGRy2_+reJ57wm4OokgJNApHA@mail.gmail.com> <e388a12a-79f4-4735-9a0b-bd45c6485f64@googlegroups.com> <CAP7+vJKE4KnHYiXc+FQmdcu4yO1N=PAkDRUWYi3ZRTshCi9evQ@mail.gmail.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> Message-ID: <CACac1F_BaUaX+w3xkRo0kJ9TjzS9EisKhnAjE3AiTEVwgRfMMA@mail.gmail.com> On 8 March 2014 23:36, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > This reminded me a discussion we had with Mark Dickinson on the bug tracker: > "When a user is entering 0.6112295, she means 0.6112295, not > 0x1.38f312b1b36bdp-1 or > 0.61122949999999998116351207499974407255649566650390625 which are exact > values of the underlying binary representation of 0.6112295." However, when a user enters 0.6112295, she presumably *also* means she wants a float value (assuming she understands how literals in Python work). What the user *doesn't* necessarily realise is that those two desires are mutually contradictory (there *is* no float value for the exact value 0.6112295). So the question becomes, what should we do when the user asks for two mutually-incompatible things. One option is to raise an exception. I presume everyone agrees that this is not a sensible option :-) [0.1 is a syntax error, but 0.5 is OK...] Another option is to fail to satisfy the user's desire for exactly 0.6112295. But do the best we can given the limitations of floats. That's current behaviour. A third option would be to fail to satisfy the user's implied desire for a float: - either by always giving a Decimal (which is a variation on the original "decimal as the default non-integer type" proposal) - or by giving an exactly equivalent float if possible but a Decimal otherwise (which means seemingly-similar literals have different types) The only one of these that feels palatable to me is the current behaviour. The problem of course is that what the user *actually* wants is context-sensitive behaviour - they don't *see* a float being passed to a constructor, but rather a Decimal (or timedelta, or whatever) literal being written using a slightly-clumsy functional notation. That's why explicit decimal literals are a good fit for the actual user issue here. (It also hints at a benefit to more general user-defined literals, but let's not go there :-)) Paul From rob.cliffe at btinternet.com Sun Mar 9 13:31:28 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Sun, 09 Mar 2014 12:31:28 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <9ebb139a-8b51-451e-83a1-a4011b0cd4c2@googlegroups.com> <CAP7+vJLv_nkLd1VR8ziTDPwK8g=JLTAtN7YT48N6n=_w498QOQ@mail.gmail.com> <8eb523ca-9945-498b-99a6-ed2c9ed25082@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> Message-ID: <531C5F20.4090000@btinternet.com> On 08/03/2014 18:49, Mark Dickinson wrote: > > > >>> x = 49534541648432951 > > >>> y = x + 2.0 > > >>> x < y > > True > > >>> Decimal(x) > Decimal(repr(y)) # simulating proposed meaning of > Decimal(y) > > True > > > You don't need to add 2.0 to x to see this effect. You can write y = float(x) or even y = x - 3.0 and you get the same value for y. So in fact you can see >>> x < x - 3.0 True which might be surprising at first sight. [Just for amusement :-) ] Rob Cliffe From stefan at bytereef.org Sun Mar 9 13:36:41 2014 From: stefan at bytereef.org (Stefan Krah) Date: Sun, 9 Mar 2014 13:36:41 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309073229.GJ28804@ando> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando> Message-ID: <20140309123641.GA30184@sleipnir.bytereef.org> Steven D'Aprano <steve at pearwood.info> wrote: > Note that this round-trips back to float correctly: > > py> assert float(d) == 9017.010981286436 Many inexact values map to the same float. > So I must admit, I'm not sure where the extra digits come from: > > Decimal('9017.01098128643570817075669765472412109375') > Decimal('9017.010981286435708170756694') Without looking at your calculation, I guess it is rounding: >>> 9017.010981286436.as_integer_ratio() (2478577105427079, 274877906944) >>> Decimal(2478577105427079) / 274877906944 Decimal('9017.010981286435708170756698') >>> getcontext().prec = 100 >>> Decimal(2478577105427079) / 274877906944 Decimal('9017.01098128643570817075669765472412109375') Stefan Krah From ron3200 at gmail.com Sun Mar 9 15:28:53 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 09 Mar 2014 09:28:53 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309123641.GA30184@sleipnir.bytereef.org> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando> <20140309123641.GA30184@sleipnir.bytereef.org> Message-ID: <lfhtqn$5kq$1@ger.gmane.org> On 03/09/2014 07:36 AM, Stefan Krah wrote: > Steven D'Aprano<steve at pearwood.info> wrote: >> >Note that this round-trips back to float correctly: >> > >> >py> assert float(d) == 9017.010981286436 > Many inexact values map to the same float. To get the same result as Stevens calculation, the context needs to be set to 16. >>> from decimal import * >>> setcontext(Context(16)) >>> Decimal(2478577105427079) / 274877906944 Decimal('9017.010981286436') When converting a float to Decimal... shouldn't the smallest (closest to zero) value that round trips be correct? It seems like anything else is an implicit rounding up. We don't add .5 when we convert an int to a float. The rounding may be due to the financial need to average out small errors rather than have them accumulate. ie.. ROUND_HALF_EVEN, >>> from decimal import * >>> getcontext() Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, capitals=1, clamp=0, flags=[], traps=[InvalidOperation, DivisionByZero, Overflow]) Which is why it's suggested decimal should only be used for financial calculations. But it's only the context that differs I think. Decimals default context does seem to be for financial calculations. Maybe there should be some easy to set contexts could be added. setcontext(MATH) setcontext(FINANCE) setcontext(SCIENCE) Or even... setcontext(CALCULATOR) >> > So I must admit, I'm not sure where the extra digits come from: >> > >> > Decimal('9017.01098128643570817075669765472412109375') >> > Decimal('9017.010981286435708170756694') > Without looking at your calculation, I guess it is rounding: >>>> >>> 9017.010981286436.as_integer_ratio() > (2478577105427079, 274877906944) >>>> >>> Decimal(2478577105427079) / 274877906944 > Decimal('9017.010981286435708170756698') >>>> >>> getcontext().prec = 100 >>>> >>> Decimal(2478577105427079) / 274877906944 > Decimal('9017.01098128643570817075669765472412109375') Something that bothers me about the decimal module is how the rest of the numbers are converted. (Not the ones entered in by hand.) If we should enter x = Decimal('some number') to get the exact value, how does Decimal get y in xy = x * y if y is a float? Would it be the same as Decimal(y)? (Without quotes) Cheers, Ron From solipsis at pitrou.net Sun Mar 9 15:59:56 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 9 Mar 2014 15:59:56 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> Message-ID: <20140309155956.75ad1ca5@fsol> On Sat, 08 Mar 2014 13:59:32 -0800 Ethan Furman <ethan at stoneleaf.us> wrote: > So, how is this justified? > > Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) > [GCC 4.7.3] on linux > Type "help", "copyright", "credits" or "license" for more information. > --> from decimal import Decimal as D > --> 9017.0109812864350067128347806 > 9017.010981286436 > --> D(9017.0109812864350067128347806) > Decimal('9017.01098128643570817075669765472412109375') > > In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits. Why is > the Decimal more "accurate" that the float it came from? Both representations need to uphold the round-tripping property: float(str(some_float)) == some_float Decimal(str(some_decimal)) == some_decimal However, since Decimal has arbitrary precision (as opposed to a fixed number of bits), the number of digits needed in the repr() can be much higher in order to uphold the round-tripping property: >>> float('1.1') == 1.1 True >>> decimal.Decimal('1.1') == decimal.Decimal(1.1) False Moreover, the Decimal constructor strives to uphold the property that Decimal(some_number) == some_number (using the required internal precision). Therefore, decimal.Decimal(1.1) is the exact decimal value of the binary floating-point number *best approaching* the decimal literal '1.1': >>> decimal.Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1 True Indeed, the difference is less than a binary floating-point mantissa's resolution: >>> decimal.Decimal(1.1) - decimal.Decimal('1.1') Decimal('8.881784197001252323389053345E-17') >>> 2**(-sys.float_info.mant_dig) 1.1102230246251565e-16 However, since Decimal has such a high potential resolution, changing even one faraway digit will break the equality: >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265626') == 1.1 False (I changed the trailing '5' to a '6') Which is why Decimal(1.1) has to be so precise, and so has its repr() too. Of course, if you convert those literals to floats, they turn out to convert to the same binary floating-point number: >>> 1.1 == 1.100000000000000088817841970012523233890533447265625 True float() can therefore afford to have a "user-friendly" repr() in some cases where Decimal can't. Regards Antoine. From solipsis at pitrou.net Sun Mar 9 16:05:14 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 9 Mar 2014 16:05:14 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> Message-ID: <20140309160514.3d86ae24@fsol> On Sun, 9 Mar 2014 16:02:07 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > > I don't think TI-89 should be the exception. I don't think it is a good > idea to try to make a general purpose programming language like Python > into a (mathematical) clone of the TI-89 calculator just to satisfy the > tiny minority of users who are numerically naive TI-89 users. I'm surprised: are physical calculators (TI-89) still in use today? Regards Antoine. From ron3200 at gmail.com Sun Mar 9 16:19:00 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sun, 09 Mar 2014 10:19:00 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309155956.75ad1ca5@fsol> References: <46c0dc14-6eda-45d5-8c3d-5e73b0d13146@googlegroups.com> <1394160815.26581.YahooMailNeo@web181002.mail.ne1.yahoo.com> <260d6bdc-6030-49cb-aa07-94adde2473b0@googlegroups.com> <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> <20140309155956.75ad1ca5@fsol> Message-ID: <lfi0om$34u$1@ger.gmane.org> On 03/09/2014 09:59 AM, Antoine Pitrou wrote: > Therefore, decimal.Decimal(1.1) is the exact decimal value of the > binary floating-point number*best approaching* the decimal literal '1.1': > > >>> decimal.Decimal(1.1) > Decimal('1.100000000000000088817841970012523233890533447265625') > >>> decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1 > True > > Indeed, the difference is less than a binary floating-point mantissa's > resolution: Yes, I realized that using the number closest to zero, is an implicit rounding down, after I posted. The increases resolution accounted for the slightly less results in the examples. But because it was using the value closest to the decimal representation (the quoted value) instead of the exact float value. The higher resolution might be slightly higher or lower in that case and have extra, or even possibly (but more rarely) fewer digits. (is that right?) The decimal is more accurate in that manner. But more picky about quality too. Looks like there is a way to set that in the context.. So that (value +- some error amount) is used in the equality tests. Anyway need to go... Check back later this evening. Cheers, Ron From tim.peters at gmail.com Sun Mar 9 16:52:30 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 9 Mar 2014 10:52:30 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309073229.GJ28804@ando> References: <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAHVvXxSm_-+AiP95sEbmevdFzSzshr6aoVOcCnp=2eMdTTdXgw@mail.gmail.com> <CAP7+vJ+Nqjus+S4+x=yPRbjWNv+SM7CS3gTuVMr8sj6-TWzzyg@mail.gmail.com> <CAHVvXxSfSjVX=eRF4k+1NaTsDihoKfEt+BD2KsjOUmE+1u2YNQ@mail.gmail.com> <531B92C4.3040502@stoneleaf.us> <20140309073229.GJ28804@ando> Message-ID: <CAExdVNkDm6x65O=YYJ48=GUPiVMP3eZutVo9j-T++L8Z7JOOLw@mail.gmail.com> [Ethan Furman] >> Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) >> [GCC 4.7.3] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> --> from decimal import Decimal as D >> --> 9017.0109812864350067128347806 >> 9017.010981286436 >> --> D(9017.0109812864350067128347806) >> Decimal('9017.01098128643570817075669765472412109375') >> >> In case my question isn't obvious, the direct float got me 16 digits, while >> the Decimal float got me 42 digits. Why is the Decimal more "accurate" >> that the float it came from? [Steven D'Aprano] > That's an interesting question. It does appear to be a lot more digits > than needed. Here's that float exactly, in hex: > > py> 9017.010981286436.hex() > '0x1.19c8167d5b50ep+13' > > Let's convert it to decimal: > > > py> digits = list('19c8167d5b50e') > py> for i, c in enumerate(digits): > ... digits[i] = '0123456789abcdef'.index(c) > ... > py> d = Decimal(0) > py> for n in digits[::-1]: > ... d += n > ... d /= 16 > ... > py> d += 1 > py> d *= 2**13 > py> d > Decimal('9017.010981286435708170756694') > > ,,, > So I must admit, I'm not sure where the extra digits come from: > > Decimal('9017.01098128643570817075669765472412109375') > Decimal('9017.010981286435708170756694') > > but perhaps my calculation was rather naive. The calculation is fine, but recall that decimal calculations are limited to 28 significant digits by default. It's no coincidence that len('9017.010981286435708170756694') == 29 (28 digits + a decimal point). The answer has "suffered" multiple rounding errors to make it fit in 28 digits. Boost the context precision before you start, and you'll get more digits. Keep boosting it, and you'll eventually get 9017.01098128643570817075669765472412109375 followed by (zero or more) trailing zeroes. A more direct way is to view this as an integer problem: mathematically, 0x1.19c8167d5b50ep+13 is 0x119c8167d5b50e / 2**(4*13 - 13) = 0x119c8167d5b50e / 2**39 = (multiplying top and bottom by 5**39) 0x119c8167d5b50e * 5**39 / 10**39 and now it's in the form of an integer numerator divided by a power of 10. The numerator is: >>> 0x119c8167d5b50e * 5**39 9017010981286435708170756697654724121093750 From harrismh777 at gmail.com Sun Mar 9 19:53:34 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sun, 9 Mar 2014 11:53:34 -0700 (PDT) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309160514.3d86ae24@fsol> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol> Message-ID: <49e2c298-5bee-4797-8dd9-3d8fc7ea0a0b@googlegroups.com> On Sunday, March 9, 2014 10:05:14 AM UTC-5, Antoine Pitrou wrote: > > > tiny minority of users who are numerically naive TI-89 users. > > I'm surprised: are physical calculators (TI-89) still in use today? > Oh my, yes. They are the premier programmable graphing calculator today (not the only one, or course) that supports advanced computation (calc, linear algebra, &c) and with not only polar and rect coord graphing but also 3D graphing. Its native language is BASIC, in the calculator supports a plethora of applications for science, business, academia, engineering, and also a host of organizer apps like calendar, notepad, spread sheet (yes, really) and more. Its a phenomenal device, really. The TI84+ is also still marketed heavily; and it has been updated lately with a color screen. It is not sufficient for college engineering work, but its fine for high school and general programming/graphing. Its language is also BASIC. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/31c880f4/attachment.html> From harrismh777 at gmail.com Sun Mar 9 20:00:14 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sun, 9 Mar 2014 12:00:14 -0700 (PDT) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com> References: <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <CAN3CYHzOjiqZ=zcO_f4b=qa4Hzi0xP82OjFna==n1tUsKXe1sA@mail.gmail.com> <fd2f2b50-5ef6-4b11-824c-f70549f80e34@googlegroups.com> Message-ID: <db797bdc-d6a5-4bd7-8fc7-a5c0d5c18ad0@googlegroups.com> On Sunday, March 9, 2014 1:10:41 AM UTC-6, Mark H. Harris wrote: > this is one of Mike Cowlishaw's papers regarding decimal > floating point, hardware decimal floating point, and the IEEE 854 1987 > standard, > and other topics. He mentions the TI89, and others, that use decimal > floating > point arithmetic, with citations. > > http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf > In the course of this thread I have mentioned Michael Cowlishaw's Rexx, and my work at IBM; for one thing adding a library of decimal floating point routines for use with VM370 Rexx which provides the scientific and transcendental functions for the now arcane 370 system. I found one of the scripts in my archive and uploaded it to code.google.com for historical purposes and perspective. If anyone is interested it can be found here: https://code.google.com/p/rexxdecimallibrary/ Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/59f4d99a/attachment-0001.html> From steve at pearwood.info Sun Mar 9 20:01:46 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 10 Mar 2014 06:01:46 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309160514.3d86ae24@fsol> References: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol> Message-ID: <20140309190145.GA12595@ando> On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote: > On Sun, 9 Mar 2014 16:02:07 +1100 > Steven D'Aprano <steve at pearwood.info> wrote: > > > > I don't think TI-89 should be the exception. I don't think it is a good > > idea to try to make a general purpose programming language like Python > > into a (mathematical) clone of the TI-89 calculator just to satisfy the > > tiny minority of users who are numerically naive TI-89 users. > > I'm surprised: are physical calculators (TI-89) still in use today? I don't know about other places, but in Australia graphing calculators, including the TI-89 series, are required for all high school school students doing maths. -- Steven From harrismh777 at gmail.com Sun Mar 9 20:32:03 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Sun, 9 Mar 2014 12:32:03 -0700 (PDT) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140309190145.GA12595@ando> References: <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAAu3qLWJwQnnk_WYoLVtJ0z1zmEkewTQocdbv0P3x=abx-18qA@mail.gmail.com> <616abc5f-5bf3-4d7b-8a1d-7c946c4da8e3@googlegroups.com> <2ee188ac-cf2d-4db1-85a2-5f3bfa218888@googlegroups.com> <63E3717D-F5C9-48E8-9E2F-0B04A9161DAD@yahoo.com> <b54d82d8-a331-4584-b2e2-e526362fd42a@googlegroups.com> <20140309050207.GG28804@ando> <20140309160514.3d86ae24@fsol> <20140309190145.GA12595@ando> Message-ID: <0b19b5d8-e776-4820-9922-7058e7bd3b10@googlegroups.com> On Sunday, March 9, 2014 2:01:46 PM UTC-5, Steven D'Aprano wrote: > > On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote: > > I'm surprised: are physical calculators (TI-89) still in use today? > > I don't know about other places, but in Australia graphing calculators, > including the TI-89 series, are required for all high school school > students doing maths. The same is true in the States too; well at least in Southeast Minnesota where I reside. Also, I have a son who is planning on engineering school at Iowa State next fall; they require the TI89 {preferably} or equiv for their course work. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/bb98c393/attachment.html> From guido at python.org Sun Mar 9 20:34:12 2014 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Mar 2014 12:34:12 -0700 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> Message-ID: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> I would dearly like to put this thread to rest, as it has strayed mightily from the topic of improvements to Python, and all points of view have been amply defended. I'm hoping to hear from Cowlishaw, but I expect he'll side with one of Mark Dickinson's proposals. I hope that somebody will take up the task to write a PEP about introducing a decimal literal; that sounds like an obtainable goal, less controversial than changing Decimal(<float>). I did do some more thinking about how the magic repr() affects the distribution of values, and came up with an example of sorts that might show what it does. We've mostly focused on simple like 1.1, but to understand the distribution issue it's better to look at a very large value. I took 2**49 as an example and added a random fraction. When printed this always gives a single digit past the decimal point, e.g. 562949953421312.5. Then I measured the distribution of the last digit. What I found matched my prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 occurred with roughly equal probability (1/8th). So 3 and 7 are completely missing. The explanation is simple enough: using the (current) Decimal class it's easy to see that there are only 8 possible actual values, whose fractional part is a multiple of 1/8. IOW the exact values end in .000, .125, .250, .375, .500, .625, .750, .875. (*) The conclusion is that there are only 3 bits represented after the binary point, and repr() produces a single digit here, because that's all that's needed to correctly round back to the 8 possible values. So it picks the digit closest to each of the possible values, and when there are two possibilities it picks one. I don't know how it picks, but it is reproducible -- in this example it always chooses .2 to represent .250, and .8 to represent .750. The exact same thing happens later in the decimal expansion for smaller numbers. I think that the main objection to this distribution is that, while the exact distribution is evenly spaced (the possibilities are 1/8th away from each other), the rounded distribution has some gaps of 1/10th and some of 1/5th. I am not enough of a statistician to know whether this matters (the distribution of *digits* in the exact values is arguably less randomized :-) but at least this example clarified my understanding of the phenomenon we're talking about when we discuss distribution of values. ________ (*) I was momentarily startled to find that the set of Decimals produced contained 9 items, until I realized that some random() call must have produced a value close enough to 1 to be rounded up. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/fb4ac37c/attachment.html> From oscar.j.benjamin at gmail.com Sun Mar 9 21:07:39 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sun, 9 Mar 2014 20:07:39 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> Message-ID: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> On 9 March 2014 19:34, Guido van Rossum <guido at python.org> wrote: > > I hope that somebody will take up the task to write a PEP about introducing > a decimal literal; that sounds like an obtainable goal, less controversial > than changing Decimal(<float>). I said I'd have a go at this. It's not as simple as it might seem though. The simplest proposal would be to say that 3.14d is equivalent to Decimal('3.14'). The fact the the constructor is independent of the context is good here. It means that the value of the literal can be determined statically which is good for trying to understand what code does and also for constant folding etc. The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators. This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'): >>> from decimal import Decimal, getcontext >>> getcontext().prec = 3 >>> Decimal('+1234') # Exact Decimal('1234') >>> +Decimal('1234') # Rounded Decimal('1.23E+3') >>> Decimal('-1234') # Exact Decimal('-1234') >>> -Decimal('1234') # Rounded Decimal('-1.23E+3') I think that this behaviour would be surprising from a literal. I would prefer to be able to say that -1234d was equivalent to Decimal('-1234d') and would be guaranteed not to round. I just don't really know if that's trivially easy or really difficult given the current grammar specification and parse/compiler infrastructure. Oscar From guido at python.org Sun Mar 9 21:39:01 2014 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Mar 2014 13:39:01 -0700 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> Message-ID: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> Any solutions you are going to come up with would cause other anomalies such as -(5d) not being equal to -5d. My vote goes to treating + and - as the operators they are and telling people that if they want a negative constant that exceeds the context's precision they're going to have to use Decimal('-N'), rather than adding a terrible hack to the parser (and hence to the language spec). BTW Do you why the default precision is 28? On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>wrote: > On 9 March 2014 19:34, Guido van Rossum <guido at python.org> wrote: > > > > I hope that somebody will take up the task to write a PEP about > introducing > > a decimal literal; that sounds like an obtainable goal, less > controversial > > than changing Decimal(<float>). > > I said I'd have a go at this. It's not as simple as it might seem though. > > The simplest proposal would be to say that 3.14d is equivalent to > Decimal('3.14'). The fact the the constructor is independent of the > context is good here. It means that the value of the literal can be > determined statically which is good for trying to understand what code > does and also for constant folding etc. > > The problem though is with things like +3.14d or -3.14d. Python the > language treats the + and - signs as not being part of the literal but > as separate unary operators. This is unimportant for int, float and > imaginary literals because there unary + is a no-op and unary - is > exact. For decimal this is not the same as +Decimal('3.14') is not the > same as Decimal('+3.14'): > > >>> from decimal import Decimal, getcontext > >>> getcontext().prec = 3 > >>> Decimal('+1234') # Exact > Decimal('1234') > >>> +Decimal('1234') # Rounded > Decimal('1.23E+3') > >>> Decimal('-1234') # Exact > Decimal('-1234') > >>> -Decimal('1234') # Rounded > Decimal('-1.23E+3') > > I think that this behaviour would be surprising from a literal. I > would prefer to be able to say that -1234d was equivalent to > Decimal('-1234d') and would be guaranteed not to round. I just don't > really know if that's trivially easy or really difficult given the > current grammar specification and parse/compiler infrastructure. > > > Oscar > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140309/23c9e87b/attachment-0001.html> From tim.peters at gmail.com Sun Mar 9 22:33:41 2014 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 9 Mar 2014 16:33:41 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> Message-ID: <CAExdVN=XyYZhQ4R9rm21N7CJxAEmtLriU7YUktFmR1vHgXoNvA@mail.gmail.com> [Guido] > ... > I did do some more thinking about how the magic repr() affects the > distribution of values, and came up with an example of sorts that might show > what it does. We've mostly focused on simple like 1.1, but to understand the > distribution issue it's better to look at a very large value. > > I took 2**49 as an example and added a random fraction. When printed this > always gives a single digit past the decimal point, e.g. 562949953421312.5. > Then I measured the distribution of the last digit. What I found matched my > prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 occurred with roughly equal > probability (1/8th). So 3 and 7 are completely missing. > > The explanation is simple enough: using the (current) Decimal class it's > easy to see that there are only 8 possible actual values, whose fractional > part is a multiple of 1/8. IOW the exact values end in .000, .125, .250, > .375, .500, .625, .750, .875. (*) The conclusion is that there are only 3 > bits represented after the binary point, and repr() produces a single digit > here, because that's all that's needed to correctly round back to the 8 > possible values. So it picks the digit closest to each of the possible > values, and when there are two possibilities it picks one. I don't know how > it picks, but it is reproducible -- in this example it always chooses .2 to > represent .250, and .8 to represent .750. Just the usual round-to-nearest/even. 0.25 and 0.75 are exactly halfway, so round to the closest even retained digit (0.2 and 0.8). > ... > (*) I was momentarily startled to find that the set of Decimals produced > contained 9 items, until I realized that some random() call must have > produced a value close enough to 1 to be rounded up. Not rare! About half the trailing zeroes are accounted for by rounding up fractions >= 0.9375 (1-1/16), and the other half from rounding down fractions <= 0.0625 (1/16). For example, here's the full distribution across 10,000 tries: 562949953421312.0 631 562949953421312.1 1230 562949953421312.2 1224 562949953421312.4 1270 562949953421312.5 1245 562949953421312.6 1287 562949953421312.8 1271 562949953421312.9 1206 562949953421313.0 636 Except that .3 and .7 are missing (and 2 values must be missing regardless of rounding rules), there's no bias toward higher or lower numbers. There is a bias toward trailing even digits, but that's an intended consequence of the "even" in "round-to-nearest/even". BTW, I still like changing Decimal(a_float) to work with repr(a_float). It's more-than-less unprincipled, but it would reduce surprises for non-experts. Ya, I know they may eventually bump into a non-monotonic case (etc), but by the time they can _spell_ "monotonic" they won't be newbies anymore ;-) For a while I hung out on stackoverflow answering "Python fp is broken!" complaints. One thing I learned is that while the questioners always had a shallow (to be charitable) understanding of fp issues, exceedingly few had the slightest interest in gaining deeper knowledge. They just wanted their immediate problem to go away, happy to accept any quick hack. Will they get burned again later? Sure. But you can't teach the unteachable. What you can do is delay the day they realize they're in trouble again ;-) From stefan at bytereef.org Sun Mar 9 22:57:15 2014 From: stefan at bytereef.org (Stefan Krah) Date: Sun, 9 Mar 2014 22:57:15 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> References: <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> Message-ID: <20140309215714.GA3282@sleipnir.bytereef.org> Guido van Rossum <guido at python.org> wrote: > I would dearly like to put this thread to rest, as it has strayed mightily from > the topic of improvements to Python, and all points of view have been amply > defended. I'm hoping to hear from Cowlishaw, but I expect he'll side with one > of Mark Dickinson's proposals. I hope that somebody will take up the task to > write a PEP about introducing a decimal literal; that sounds like an obtainable > goal, less controversial than changing Decimal(<float>). One more thing that hasn't been mentioned yet: Java's BigDecimal constructor appears to perform exact double -> Decimal conversions, unless a context is passed: http://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html """ BigDecimal(double val) Translates a double into a BigDecimal which is the exact decimal representation of the double's binary floating-point value. BigDecimal(double val, MathContext mc) Translates a double into a BigDecimal, with rounding according to the context settings. """ I strongly suspect that this matches Mike Cowlishaw's preference. Stefan Krah From saghul at gmail.com Sun Mar 9 23:02:16 2014 From: saghul at gmail.com (=?UTF-8?B?U2HDumwgSWJhcnJhIENvcnJldGfDqQ==?=) Date: Sun, 09 Mar 2014 23:02:16 +0100 Subject: [Python-ideas] Extend socket.accept to put accepted socket in non-blocking mode In-Reply-To: <20140307114039.720efff9@fsol> References: <53199376.40007@gmail.com> <20140307114039.720efff9@fsol> Message-ID: <531CE4E8.704@gmail.com> On 03/07/2014 11:40 AM, Antoine Pitrou wrote: > On Fri, 07 Mar 2014 10:37:58 +0100 > Sa?l Ibarra Corretg? <saghul-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> > wrote: >> >> accept4 allows us to do this with a single syscall, and it made it into >> FreeBSD 10, so that's another system that could benefit from this >> optimization. > > If it doesn't, then perhaps the configure script needs to be fixed. > I checked and the condigure script seems to do the right thing, so FreeBSD 10 should just use accpet4 without nay change. -- Sa?l Ibarra Corretg? bettercallsaghul.com From tjreedy at udel.edu Sun Mar 9 23:22:29 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 09 Mar 2014 18:22:29 -0400 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> Message-ID: <lfipj6$3iv$1@ger.gmane.org> On 3/9/2014 3:34 PM, Guido van Rossum wrote: > I would dearly like to put this thread to rest, as it has strayed > mightily from the topic of improvements to Python, and all points of > view have been amply defended. I'm hoping to hear from Cowlishaw, but I > expect he'll side with one of Mark Dickinson's proposals. I hope that > somebody will take up the task to write a PEP about introducing a > decimal literal; that sounds like an obtainable goal, less controversial > than changing Decimal(<float>). > > I did do some more thinking about how the magic repr() affects the > distribution of values, and came up with an example of sorts that might > show what it does. We've mostly focused on simple like 1.1, but to > understand the distribution issue it's better to look at a very large value. > > I took 2**49 as an example and added a random fraction. When printed > this always gives a single digit past the decimal point, e.g. > 562949953421312.5. Then I measured the distribution of the last digit. > What I found matched my prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 > occurred with roughly equal probability (1/8th). So 3 and 7 are > completely missing. > > The explanation is simple enough: using the (current) Decimal class it's > easy to see that there are only 8 possible actual values, whose > fractional part is a multiple of 1/8. IOW the exact values end in .000, > .125, .250, .375, .500, .625, .750, .875. (*) The conclusion is that > there are only 3 bits represented after the binary point, and repr() > produces a single digit here, because that's all that's needed to > correctly round back to the 8 possible values. So it picks the digit > closest to each of the possible values, and when there are two > possibilities it picks one. I don't know how it picks, but it is > reproducible -- in this example it always chooses .2 to represent .250, > and .8 to represent .750. The exact same thing happens later in the > decimal expansion for smaller numbers. It is doing 'round to even'. > I think that the main objection to this distribution is that, while the > exact distribution is evenly spaced (the possibilities are 1/8th away > from each other), the rounded distribution has some gaps of 1/10th and > some of 1/5th. I am not enough of a statistician to know whether this > matters (the distribution of *digits* in the exact values is arguably > less randomized :-) but at least this example clarified my understanding > of the phenomenon we're talking about when we discuss distribution of > values. The fact that the rounding rule produces one round down an one up and a symmetric distribution about 5 of non-0 final digits ameliorates any statistical problems (this is the reason for 'round to even'). If we add 0 as a final digit, we must remember (as you noted in your footnote) that this is sometimes a result of rounding down to 0 and sometimes a result of rounding up to 10 (with the 1 added into the preceding digit). I think the statistical issue you framed is this. We have numbers between 0 and 1 with a distribution that has a mean and variance. The uniform [0,1] distribution (mean .5, variance 1/12) is the simplest, though it may or may not be the most realistic in any particular situation. We have two procedures. 1. Round each number to the nearest n/10, from 0/10 to 10/10. 2. Round each number to the nearest m/8 (0 <= m <= 8) and thence to some n/10 as described above. If we compare the mean and variance of the rounded numbers, how much bias is introduced by the rounding process and is the bias introduced by the second enough more than by the first to worry about. For a uniform distribution, there is no bias in the mean of rounded numbers. As I write this, I suspect there will be a small bias in the variance, and slightly more in the second, but not enough that I would worry about it. Let's see. >>> from statistics import pvariance as pvar >>> p1 = (0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10) >>> p2 = (0, 1, 1, 2, 2, 4, 4, 5, 5, 6, 6, 8, 8, 9, 9, 10) >>> print(1/12, pvar(p1)/100, pvar(p2)/100) 0.08333333333333333 0.085 0.09625 Hmmm. The removal of the .3s and .7s biases the digits away from .5, making the variance increase more than I expected. However, if I were looking as numbers from 0.0 to 10.0 (instead of 1.0), about half the .x double rounded digits would be moved toward 5.0 rather than away. Ditto for rounding 0 to 1.0 to 2 digits instead of 1. Either way, I would expect the double rounding bias to be quite reduced. >>> def end37(n): return (n % 10) in (3, 7) >>> p3 = list(range(100)) + list(range(1, 101)) >>> p4 = list(i for i in range(100) if not end37(i)) + list(i for i in range(1, 101) if not end37(i)) >>> print(1/12, pvar(p3)/10000, pvar(p4)/10000) 0.08333333333333333 0.08335 0.0834625 >>> p5 = list(range(1000)) + list(range(1, 1001)) >>> p6 = list(i for i in range(1000) if not end37(i)) + list(i for i in range(1, 1001) if not end37(i)) >>> print(1/12, pvar(p5)/1000000, pvar(p6)/1000000) 0.08333333333333333 0.0833335 0.083334625 For any realistic number of significant digits, variance bias for uniformly distributed numbers is not an issue. -- Terry Jan Reedy From dickinsm at gmail.com Mon Mar 10 09:32:16 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Mar 2014 08:32:16 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> Message-ID: <CAAu3qLUJDMrEk7RmrLrpdbJrhPZvzpaV6n8EcbRYqHr8+-BCrQ@mail.gmail.com> On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com>wrote: > The problem though is with things like +3.14d or -3.14d. Python the > language treats the + and - signs as not being part of the literal but > as separate unary operators. This is unimportant for int, float and > imaginary literals because there unary + is a no-op and unary - is > exact. For decimal this is not the same as +Decimal('3.14') is not the > same as Decimal('+3.14'): > Another option would be to change the behaviour of + and - on Decimal objects by binding `__pos__` and `__neg__` to the specification's 'copy' and 'copy-negate' operations instead of (as currently) to the 'plus' and 'minus' operations. Those behaviours are more in keeping with the way that + and - operate on other objects. -- Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/565967ec/attachment-0001.html> From oscar.j.benjamin at gmail.com Mon Mar 10 11:07:11 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 10:07:11 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> References: <6F35C6C7-1809-4A3F-A7DF-E9EF8A628AE2@yahoo.com> <523d520b-54a6-4782-970d-c34a6ff50cfd@googlegroups.com> <CAP7+vJK6Bquc1VQfdO87JV6HhfhXs3LA6aMJfDoZArNCkKH9aA@mail.gmail.com> <CAHVvXxTgvXRWdX96mtQ4nuninNYVmaiDgENhKKWra9OO6B20rg@mail.gmail.com> <CAP7+vJJCxfdqoBYTE7x7vaC=r4TKpaeGVOoaKHwfB2o5n67fjA@mail.gmail.com> <CAAu3qLWvUpC8OWw3KnGmjOhiLEVyapi+73g6-C8=_-e5tdtCQQ@mail.gmail.com> <CAAu3qLVQd_N34D0-5qdi6iFo4tcjkc4eC8Pb=9JcUaCLNnQ5tA@mail.gmail.com> <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> Message-ID: <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote: > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: >> >> The problem though is with things like +3.14d or -3.14d. Python the >> language treats the + and - signs as not being part of the literal but >> as separate unary operators. This is unimportant for int, float and >> imaginary literals because there unary + is a no-op and unary - is >> exact. For decimal this is not the same as +Decimal('3.14') is not the >> same as Decimal('+3.14'): > Any solutions you are going to come up with would cause other anomalies such > as -(5d) not being equal to -5d. My vote goes to treating + and - as the > operators they are and telling people that if they want a negative constant > that exceeds the context's precision they're going to have to use > Decimal('-N'), rather than adding a terrible hack to the parser (and hence > to the language spec). I think it's reasonable that -(5d) not be equal to -5d. Expert users would exploit this behaviour the same way that they currently exploit unary + for rounding to context. I don't think that non-expert users would write that by mistake. I'm also not that bothered about +5d rounding to context since the + is optional and current users of the decimal module would probably expect that to round. What I dislike about this is that it would mean that negative decimal literals were essentially unsafe. It seems okay if you assume that the precision is 28 or higher but users can set it to a lower value and decimal contexts are unscoped so they have action at a distance on other code. So if I have a module libmod.py with: # libmod.py def libfunc(): a = -1.23465789d return f(a) Then I have a script: # script.py from decimal import localcontext import libmod with localcontext() as ctx: ctx.prec = 5 b = libmod.libfunc() I think we could easily get into a situation where the author of libmod just assumes that the decimal context has enough precision and the author of script.py doesn't realise the effect that their change of context has on libmod. These kinds of things are generally problematic when using the decimal module and the solution is typically that libfunc needs to take control of the context. I think that it would be particularly surprising with decimal literals though. The change in the precision of "a" above might be a harmless reduction in the precision of the result or it could lead to an infinite loop or basically anything else depending on what libmod uses the value for. > BTW Do you why the default precision is 28? I have no idea. The standards I've read describe a few particular 9 digit contexts that should be provided by a conforming implementation and decimal provides them but neither is the default: >>> import decimal >>> decimal.BasicContext.prec 9 >>> decimal.ExtendedContext.prec 9 AFAIK there's no standard that suggests 28 digits. There are other common contexts such as decimal32 (7 digits), decimal64 (16 digits), and decimal128 (34 digits) and these are the contexts that Java's BigDecimal provides (as well as an "unlimited" context). Oscar From oscar.j.benjamin at gmail.com Mon Mar 10 11:16:00 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 10:16:00 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] Message-ID: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote: > On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> > wrote: >> >> The problem though is with things like +3.14d or -3.14d. Python the >> language treats the + and - signs as not being part of the literal but >> as separate unary operators. This is unimportant for int, float and >> imaginary literals because there unary + is a no-op and unary - is >> exact. For decimal this is not the same as +Decimal('3.14') is not the >> same as Decimal('+3.14'): > > Another option would be to change the behaviour of + and - on Decimal > objects by binding `__pos__` and `__neg__` to the specification's 'copy' and > 'copy-negate' operations instead of (as currently) to the 'plus' and 'minus' > operations. Those behaviours are more in keeping with the way that + and - > operate on other objects. Hi Mark and thanks for responding, Do you think it's really feasible to make such a change? The use of unary + for rounding to context has been a feature of the decimal module for as long as I've been using it. It's shown in the docs both in the Recipes section and the FAQ. I just advised Mark Harris to do this and it know appears ~20 times in his pdeclib module: https://code.google.com/p/pythondecimallibrary/source/browse/pdeclib.py#207 I would expect that a change such as this would break quite a lot of code (some of mine would break) so there would have to be some kind of compatibility mechanism or deprecation process or something but I can't think of a good way of doing it. Oscar From oscar.j.benjamin at gmail.com Mon Mar 10 11:54:54 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 10:54:54 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> Message-ID: <CAHVvXxSMxPPYvMMHNQr1aRZKytd+GyE9qv9LndyF2M4bcYogAQ@mail.gmail.com> On 10 March 2014 10:16, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote: >> On Sun, Mar 9, 2014 at 8:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> >> wrote: >>> >>> The problem though is with things like +3.14d or -3.14d. Python the >>> language treats the + and - signs as not being part of the literal but >>> as separate unary operators. This is unimportant for int, float and >>> imaginary literals because there unary + is a no-op and unary - is >>> exact. For decimal this is not the same as +Decimal('3.14') is not the >>> same as Decimal('+3.14'): >> >> Another option would be to change the behaviour of + and - on Decimal >> objects by binding `__pos__` and `__neg__` to the specification's 'copy' and >> 'copy-negate' operations instead of (as currently) to the 'plus' and 'minus' >> operations. Those behaviours are more in keeping with the way that + and - >> operate on other objects. > > Hi Mark and thanks for responding, > > Do you think it's really feasible to make such a change? The use of > unary + for rounding to context has been a feature of the decimal > module for as long as I've been using it. It's shown in the docs both > in the Recipes section and the FAQ. On the other hand the issue is primarily with __neg__ rather than __pos__as I see it because a leading + is unnecessary and I really just want a safe way of spelling a negative value. I can't see any documented reference to the behaviour of Decimal.__neg__ and it's probably used less often with the same intent. So perhaps only __neg__ could be changed. This would still be a backward-incompatible change but I think it would have significantly less breakage. Oscar From steve at pearwood.info Mon Mar 10 11:59:43 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 10 Mar 2014 21:59:43 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> Message-ID: <20140310105942.GB12595@ando> On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote: > On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote: > > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > >> > >> The problem though is with things like +3.14d or -3.14d. Python the > >> language treats the + and - signs as not being part of the literal but > >> as separate unary operators. Is that still true? Possibly the peephole optimizer has changed the situation? Using Python 1.5: >>> from dis import dis >>> dis(compile("-5", "", "single")) 0 SET_LINENO 0 3 SET_LINENO 1 6 LOAD_CONST 0 (5) 9 UNARY_NEGATIVE 10 PRINT_EXPR 11 LOAD_CONST 1 (None) 14 RETURN_VALUE but using Python 2.4: py> dis(compile("-5", "", "single")) 1 0 LOAD_CONST 0 (-5) 3 PRINT_EXPR 4 LOAD_CONST 1 (None) 7 RETURN_VALUE (versions more recent than 2.4 also use a constant). > >> This is unimportant for int, float and > >> imaginary literals because there unary + is a no-op and unary - is > >> exact. For decimal this is not the same as +Decimal('3.14') is not the > >> same as Decimal('+3.14'): > > > Any solutions you are going to come up with would cause other anomalies such > > as -(5d) not being equal to -5d. My vote goes to treating + and - as the > > operators they are and telling people that if they want a negative constant > > that exceeds the context's precision they're going to have to use > > Decimal('-N'), rather than adding a terrible hack to the parser (and hence > > to the language spec). > > I think it's reasonable that -(5d) not be equal to -5d. Did you leave the word "not" out of that sentence? :-) I don't think that it is reasonable for -(5d) to not equal -5d. I think that's an awful interface, and terribly surprising. I don't think that there are any situations were an otherwise redundant pair of parens changes behavious. E.g. x*(y) is the same as x*y. (Parens are sometimes used to disambiguate syntax or change precedence. That's different, since the parens aren't redundant.) Considering that the motivation for this change is to make it easier for newbies and numerically naive users, I really do not relish having to explain to newbies why -5d and -(5d) are different. -- Steven From oscar.j.benjamin at gmail.com Mon Mar 10 13:12:56 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 12:12:56 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310105942.GB12595@ando> References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> Message-ID: <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com> On 10 March 2014 10:59, Steven D'Aprano <steve at pearwood.info> wrote: > On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote: >> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote: >> > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: >> >> >> >> The problem though is with things like +3.14d or -3.14d. Python the >> >> language treats the + and - signs as not being part of the literal but >> >> as separate unary operators. > > Is that still true? Possibly the peephole optimizer has changed the > situation? Yes it does. It also does the same for "complex literals" even though the language formally only defines imaginary literals. The question is how to define exactly the grammar. Then once defined if the result is independent of any context the optimiser can safely constant fold decimal literals in the same way. The grammar is here: http://docs.python.org/3.4/reference/grammar.html The relevant part is here: factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False') So if the specification is that decimal literals go into the NUMBER part just like int, float and imaginary literals do then the + or - are at a separate place. The expression -5 ** 2 is evaluated as -(5 ** 2) which is -25 rather than (-5) ** 2 which is +25. It may be possible but it's certainly not straight-forward to rewrite the grammar so that -25d becomes D('-25') without affecting the normal precedence rules. >> I think it's reasonable that -(5d) not be equal to -5d. > > Did you leave the word "not" out of that sentence? :-) It's reasonable if you're an experienced user of the decimal module but yeah okay it's not really reasonable in a general sense... > I don't think that it is reasonable for -(5d) to not equal -5d. I think > that's an awful interface, and terribly surprising. I don't think that > there are any situations were an otherwise redundant pair of parens > changes behavious. E.g. x*(y) is the same as x*y. > > (Parens are sometimes used to disambiguate syntax or change precedence. > That's different, since the parens aren't redundant.) > > Considering that the motivation for this change is to make it easier for > newbies and numerically naive users, I really do not relish having to > explain to newbies why -5d and -(5d) are different. Yes but newbies won't write -(5d). Experts might if it had the semantics they want. The current behaviour of the Decimal type is already surprising in this regard so unless __pos__ and __neg__ are changed there will always be cases that surprise people. Oscar From ncoghlan at gmail.com Mon Mar 10 13:57:56 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Mar 2014 22:57:56 +1000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310105942.GB12595@ando> References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> Message-ID: <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> On 10 Mar 2014 21:06, "Steven D'Aprano" <steve at pearwood.info> wrote: > > On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote: > > On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote: > > > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin < oscar.j.benjamin at gmail.com> wrote: > > >> > > >> The problem though is with things like +3.14d or -3.14d. Python the > > >> language treats the + and - signs as not being part of the literal but > > >> as separate unary operators. > > Is that still true? Possibly the peephole optimizer has changed the > situation? > > Using Python 1.5: > > >>> from dis import dis > >>> dis(compile("-5", "", "single")) > 0 SET_LINENO 0 > > 3 SET_LINENO 1 > 6 LOAD_CONST 0 (5) > 9 UNARY_NEGATIVE > 10 PRINT_EXPR > 11 LOAD_CONST 1 (None) > 14 RETURN_VALUE > > > but using Python 2.4: > > py> dis(compile("-5", "", "single")) > 1 0 LOAD_CONST 0 (-5) > 3 PRINT_EXPR > 4 LOAD_CONST 1 (None) > 7 RETURN_VALUE > > (versions more recent than 2.4 also use a constant). It actually makes the discrepancy worse, rather than better - the optimiser is just doing constant folding at compile time, so it would just be rounding using the context that applied at compile time. That context dependence creates problems for the entire *notion* of constant folding decimal literals - you can't constant fold them at compile time at all, because the context may be wrong. I think users of decimal literals will just need to deal with the risk of unexpected rounding, as the alternatives are even more problematic. Cheers, Nick. > > > > >> This is unimportant for int, float and > > >> imaginary literals because there unary + is a no-op and unary - is > > >> exact. For decimal this is not the same as +Decimal('3.14') is not the > > >> same as Decimal('+3.14'): > > > > > Any solutions you are going to come up with would cause other anomalies such > > > as -(5d) not being equal to -5d. My vote goes to treating + and - as the > > > operators they are and telling people that if they want a negative constant > > > that exceeds the context's precision they're going to have to use > > > Decimal('-N'), rather than adding a terrible hack to the parser (and hence > > > to the language spec). > > > > I think it's reasonable that -(5d) not be equal to -5d. > > Did you leave the word "not" out of that sentence? :-) > > I don't think that it is reasonable for -(5d) to not equal -5d. I think > that's an awful interface, and terribly surprising. I don't think that > there are any situations were an otherwise redundant pair of parens > changes behavious. E.g. x*(y) is the same as x*y. > > (Parens are sometimes used to disambiguate syntax or change precedence. > That's different, since the parens aren't redundant.) > > Considering that the motivation for this change is to make it easier for > newbies and numerically naive users, I really do not relish having to > explain to newbies why -5d and -(5d) are different. > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/64852efb/attachment.html> From dickinsm at gmail.com Mon Mar 10 14:30:47 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Mar 2014 13:30:47 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> Message-ID: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> On Mon, Mar 10, 2014 at 10:16 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com > wrote: > On 10 March 2014 08:32, Mark Dickinson <dickinsm at gmail.com> wrote: > > > > Another option would be to change the behaviour of + and - on Decimal > > objects by binding `__pos__` and `__neg__` to the specification's 'copy' > and > > 'copy-negate' operations instead of (as currently) to the 'plus' and > 'minus' > > operations. Those behaviours are more in keeping with the way that + > and - > > operate on other objects. > > Hi Mark and thanks for responding, > > Do you think it's really feasible to make such a change? Probably not. :-( As you point out, there are significant backward compatibility issues. It's the solution I'd adopt if those concerns didn't exists, and the one I'd still look for a way to get to even *with* those concerns, but the level of breakage may make this out of the question. I doubt we're going to get to Decimal literals without breaking at least *some* code, but this may be too much breakage. I guess I have ulterior motives; (ab)using '+' to mean 'round to the current context' has always felt way too implicit for my tastes, and I'd jump at the chance to see it go away. The unconventional behaviour with respect to signed zeros bugs me, too. >>> -0.0 -0.0 >>> -Decimal(0.0) # negative sign mysteriously disappears. Decimal('0') It's perfectly legitimate, and part of the standard, but ... ugh. I argued with Mike Cowlishaw about it at one point. I lost. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/a32fd8db/attachment-0001.html> From dickinsm at gmail.com Mon Mar 10 14:38:49 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Mar 2014 13:38:49 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com> References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CAHVvXxS2sv+ADKTfW98uDgReQ9XDPfXMp6yMnQRpF0HGtCAyWg@mail.gmail.com> Message-ID: <CAAu3qLWd6fNYW-DhehAOJKaXjk9kg3Oa1cHHOX-+nPsK4RJv5A@mail.gmail.com> On Mon, Mar 10, 2014 at 12:12 PM, Oscar Benjamin <oscar.j.benjamin at gmail.com > wrote: > On 10 March 2014 10:59, Steven D'Aprano <steve at pearwood.info> wrote: > > On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote: > >> On 9 March 2014 20:39, Guido van Rossum <guido at python.org> wrote: > >> > On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin < > oscar.j.benjamin at gmail.com> wrote: > >> >> > >> >> The problem though is with things like +3.14d or -3.14d. Python the > >> >> language treats the + and - signs as not being part of the literal > but > >> >> as separate unary operators. > > > > Is that still true? Possibly the peephole optimizer has changed the > > situation? > > Yes it does. It also does the same for "complex literals" even though > the language formally only defines imaginary literals. ... and don't forget the Python 2.x-only hack for negation of integers: >>> type(-9223372036854775808) <type 'int'> >>> type(-(9223372036854775808)) <type 'long'> which means that it's not true that Python 2.x behaves "as if" there were no negative literals. Python 3 is cleaner in this respect. I guess this shows that we *could* in theory reintroduce such a hack for negation of decimal literals, but I agree with everyone else so far that that would be a bad idea. -- Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/22c26861/attachment.html> From random832 at fastmail.us Mon Mar 10 14:49:48 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 10 Mar 2014 09:49:48 -0400 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310105942.GB12595@ando> References: <CAP7+vJKvhZLh7mqAO-dm6RyhWZffF0sD=3etdDK01Pdt-YZ00Q@mail.gmail.com> <CAP7h-xZStW7ZJ81j45OHa7f8ASbm-43Lvi_Bd8L5KHGncRrRaQ@mail.gmail.com> <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> Message-ID: <1394459388.28493.92665413.408BEE02@webmail.messagingengine.com> On Mon, Mar 10, 2014, at 6:59, Steven D'Aprano wrote: > Is that still true? Possibly the peephole optimizer has changed the > situation? The optimizer does not dictate the specification; rather, the optimizer can do this because this operation is defined to have this result. If +3.14d has a different value than 3.14d, then the optimizer should load a different constant for these expressions. From stefan at bytereef.org Mon Mar 10 14:53:42 2014 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 10 Mar 2014 14:53:42 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> Message-ID: <20140310135342.GA10968@sleipnir.bytereef.org> [My apologies for being terse, I don't have much time to follow this discussion right now.] Nick Coghlan <ncoghlan at gmail.com> wrote: > I think users of decimal literals will just need to deal with the risk of > unexpected rounding, as the alternatives are even more problematic. That is why I think we should seriously consider moving to IEEE semantics for a decimal literal. Among other things: - Always round the literal inputs. - Supply IEEE contexts. - Make Decimal64 the default. - Add the missing rounding modes to sqrt() and exp(). - Keep the ability to create exact Decimals through the constructor when no context is passed. - Make the Decimal constructor use the context for rounding if it is passed. - ... The existing specification is largely compatible with IEEE 754-2008: http://speleotrove.com/decimal/dascope.html We can still support setting irregular (non IEEE) contexts. Stefan Krah From random832 at fastmail.us Mon Mar 10 14:55:09 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 10 Mar 2014 09:55:09 -0400 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> Message-ID: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote: > I guess I have ulterior motives; (ab)using '+' to mean 'round to the > current context' has always felt way too implicit for my tastes, and I'd > jump at the chance to see it go away. It occurs to me that there is an unprecedented opportunity to make warts "go away" - making Decimal a "first-class citizen" suggests moving it from the decimal module to a built-in type. Any backwards-incompatible change you want to make, you can give programs importing decimal the old behavior, and reserve the new behavior for the new type in builtins. Then remove the module from Python 4000. From stefan at bytereef.org Mon Mar 10 14:58:24 2014 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 10 Mar 2014 14:58:24 +0100 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> Message-ID: <20140310135824.GB10968@sleipnir.bytereef.org> Mark Dickinson <dickinsm at gmail.com> wrote: > >>> -Decimal(0.0) ?# negative sign mysteriously disappears. > > Decimal('0') Unless the rounding mode is ROUND_FLOOR. :( >>> getcontext().rounding = ROUND_FLOOR >>> -Decimal(0.0) Decimal('-0') >>> Stefan Krah From oscar.j.benjamin at gmail.com Mon Mar 10 15:05:22 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 14:05:22 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> Message-ID: <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com> On 10 March 2014 13:53, Stefan Krah <stefan at bytereef.org> wrote: > [My apologies for being terse, I don't have much time to follow this > discussion right now.] > > > Nick Coghlan <ncoghlan at gmail.com> wrote: >> I think users of decimal literals will just need to deal with the risk of >> unexpected rounding, as the alternatives are even more problematic. > > That is why I think we should seriously consider moving to IEEE semantics > for a decimal literal. Among other things: > > - Always round the literal inputs. > > - Supply IEEE contexts. > > - Make Decimal64 the default. > > - Add the missing rounding modes to sqrt() and exp(). > > - Keep the ability to create exact Decimals through the constructor when > no context is passed. > > - Make the Decimal constructor use the context for rounding if it is passed. > > - ... > > > The existing specification is largely compatible with IEEE 754-2008: > > http://speleotrove.com/decimal/dascope.html > > > We can still support setting irregular (non IEEE) contexts. This is what I've been thinking about. Most non-expert users will be very happy with Decimal64 and a single fixed context. There could be a separate type called decimal64 in builtins that always used the same standard context. Literals could create decimal64 instances. The semantics of code that uses this type and decimal literals would be independent of any arithmetic context which is good not just for constant folding but for understanding. There would be no need to explain what an arithmetic context is to new users. You can just say: "here's a type that represents decimal values with 16 digits. It sometimes needs to round if the result of a calculation exceeds 16 digits so it uses the standard decimal rounding mode XXX." Where this would get complicated is for people who also use the Decimal type. They'd need to keep track of which objects were of which type and so decimal literals might seem more annoying than useful. Oscar From oscar.j.benjamin at gmail.com Mon Mar 10 15:21:22 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 14:21:22 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> Message-ID: <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com> On 10 March 2014 13:55, <random832 at fastmail.us> wrote: > On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote: >> >> I guess I have ulterior motives; (ab)using '+' to mean 'round to the >> current context' has always felt way too implicit for my tastes, and I'd >> jump at the chance to see it go away. I tend to agree. How would you prefer to spell it? I guess something like: return d.contextround() I've sometimes used it in a polymorphic kind of way so that a function does the right thing with Decimal but also works with float/int etc. (with the + just being a no-op). That's not really an argument against your point though. > It occurs to me that there is an unprecedented opportunity to make warts > "go away" - making Decimal a "first-class citizen" suggests moving it > from the decimal module to a built-in type. Any backwards-incompatible > change you want to make, you can give programs importing decimal the old > behavior, and reserve the new behavior for the new type in builtins. That's also possible. I wonder what a good name for a new Decimal type would be. It's unfortunate that calling it 'decimal' would sort of conflict with people doing 'import decimal'. Oscar From dickinsm at gmail.com Mon Mar 10 15:22:19 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Mar 2014 14:22:19 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> Message-ID: <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote: > That is why I think we should seriously consider moving to IEEE semantics > for a decimal literal. I think that would make a *lot* of sense. -- Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/299bac63/attachment.html> From random832 at fastmail.us Mon Mar 10 15:42:26 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Mon, 10 Mar 2014 10:42:26 -0400 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> <CAHVvXxSs29JHJ+6Z+Q+_kPqtTDxAf1wqgcEVv=R5gZ_-3XsM=w@mail.gmail.com> Message-ID: <1394462546.13875.92685197.059D8531@webmail.messagingengine.com> On Mon, Mar 10, 2014, at 10:21, Oscar Benjamin wrote: > That's also possible. I wonder what a good name for a new Decimal type > would be. It's unfortunate that calling it 'decimal' would sort of > conflict with people doing 'import decimal'. I don't think the conflict is a big deal. You can shadow builtins with globals, and on the off chance that you want to use both, you can import it as a different name. Any module that imports decimal is going to be one that doesn't know or care about the built-in type. To me, the more dicey aspect is what happens if you mix the types - if you pick a type for mixed operations to convert to, then passing in the 'wrong' type will slowly 'infect' that module's data with a type which has behavior subtly different from what it expects. You could disallow mixed operations like it does now for floats, I suppose. From guido at python.org Mon Mar 10 16:01:03 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2014 08:01:03 -0700 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> Message-ID: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote: > > That is why I think we should seriously consider moving to IEEE semantics >> for a decimal literal. > > > I think that would make a *lot* of sense. > What's the proposal? Just using decimal64 for decimal literals, or introducing decimal64 as a new builtin type? (I could get behind either one.) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/fcafca28/attachment.html> From steve at pearwood.info Mon Mar 10 16:19:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 11 Mar 2014 02:19:23 +1100 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> Message-ID: <20140310151923.GD12595@ando> On Mon, Mar 10, 2014 at 09:55:09AM -0400, random832 at fastmail.us wrote: > On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote: > > I guess I have ulterior motives; (ab)using '+' to mean 'round to the > > current context' has always felt way too implicit for my tastes, and I'd > > jump at the chance to see it go away. > > It occurs to me that there is an unprecedented opportunity to make warts > "go away" - making Decimal a "first-class citizen" suggests moving it > from the decimal module to a built-in type. Any backwards-incompatible > change you want to make, you can give programs importing decimal the old > behavior, and reserve the new behavior for the new type in builtins. Historically, I don't think Python has changed the API for types when they've moved from a module to a built-in. I can't think of any example except set, but apart from a change in name from set.Set to set, I don't think there were any changes. Any other precedents? On the other hand, the built-in doesn't need to be a replacement for the decimal module, but rather an alternative solution. If decimal64 was a built-in, presumably it wouldn't need contexts and would be a lot simpler. The decimal module would still be available for those with advanced needs. -- Steven From steve at pearwood.info Mon Mar 10 16:21:49 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 11 Mar 2014 02:21:49 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310135342.GA10968@sleipnir.bytereef.org> References: <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> Message-ID: <20140310152149.GE12595@ando> On Mon, Mar 10, 2014 at 02:53:42PM +0100, Stefan Krah wrote: > That is why I think we should seriously consider moving to IEEE semantics > for a decimal literal. Among other things: Okay, I'm confused. I thought the decimal module had IEEE-754 semantics, and so I assumed that so would any decimal literal. What have I missed? -- Steven From steve at pearwood.info Mon Mar 10 16:31:27 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 11 Mar 2014 02:31:27 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com> References: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com> Message-ID: <20140310153127.GF12595@ando> On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote: > This is what I've been thinking about. Most non-expert users will be > very happy with Decimal64 and a single fixed context. There could be a > separate type called decimal64 in builtins that always used the same > standard context. Literals could create decimal64 instances. > > The semantics of code that uses this type and decimal literals would > be independent of any arithmetic context which is good not just for > constant folding but for understanding. There would be no need to > explain what an arithmetic context is to new users. You can just say: > "here's a type that represents decimal values with 16 digits. It > sometimes needs to round if the result of a calculation exceeds 16 > digits so it uses the standard decimal rounding mode XXX." All this sounds quite good to me. What's the catch? :-) > Where this would get complicated is for people who also use the > Decimal type. They'd need to keep track of which objects were of which > type and so decimal literals might seem more annoying than useful. Hmmm. I don't think this should necessarily be complicated, or at least no more complicated than dealing with any other mixed numeric types. If you don't want to mix them, don't mix them :-) (Perhaps there could be a Decimal context flag to allow/disallow mixed Decimal/decimal64 operations?) I would expect that decimal64 and decimal.Decimal would be separate types. They might share parts of the implementation under the hood, but I don't think it would be necessary or useful to have decimal64 inherit from Decimal, or visa versa. -- Steven From oscar.j.benjamin at gmail.com Mon Mar 10 16:54:16 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 15:54:16 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310152149.GE12595@ando> References: <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <20140310152149.GE12595@ando> Message-ID: <CAHVvXxRA9vjMe77tY7dp8vwENEHC-+XNo34uAKErZuyyyzMn-g@mail.gmail.com> On 10 March 2014 15:21, Steven D'Aprano <steve at pearwood.info> wrote: > On Mon, Mar 10, 2014 at 02:53:42PM +0100, Stefan Krah wrote: > >> That is why I think we should seriously consider moving to IEEE semantics >> for a decimal literal. Among other things: > > Okay, I'm confused. I thought the decimal module had IEEE-754 semantics, > and so I assumed that so would any decimal literal. What have I missed? IEEE 754 (2008) defines fixed width decimal types decimal64 and decimal128. It also refers to "extended and extendable" with "extendable" meaning something like the current Decimal type. I think that Stefan is proposing that decimal literals should be parsed as if the result needed to be stored in a fixed width decimal64 type. The standard defines all the "Details of conversion between floating-point data and external character sequences" which would regulate how this could be done. Just parsing the number as decimal64 and using a decimal64 type context by default is not enough to prevent the IMO confusing action at a distance effect of Decimal contexts though. It also wouldn't solve the issue around negative literals if a context with less precision or exponent range than decimal64 was used. If there were a separate type whose context always corresponded to decimal64 then unary + and - on a literal would have the meaning that people would expect and it would be possible to determine exactly what the code would do without needing to place caveats on what the decimal context is. I think that this would be the best approach if we're aiming at non-expert users. Oscar From oscar.j.benjamin at gmail.com Mon Mar 10 17:00:38 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 16:00:38 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310153127.GF12595@ando> References: <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com> <20140310153127.GF12595@ando> Message-ID: <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com> On 10 March 2014 15:31, Steven D'Aprano <steve at pearwood.info> wrote: > On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote: > >> This is what I've been thinking about. Most non-expert users will be >> very happy with Decimal64 and a single fixed context. There could be a >> separate type called decimal64 in builtins that always used the same >> standard context. Literals could create decimal64 instances. >> >> Where this would get complicated is for people who also use the >> Decimal type. They'd need to keep track of which objects were of which >> type and so decimal literals might seem more annoying than useful. > > Hmmm. I don't think this should necessarily be complicated, or at least > no more complicated than dealing with any other mixed numeric types. If > you don't want to mix them, don't mix them :-) Exactly. It just means that you wouldn't want to use the literals in code that actually uses the decimal module. Consider: if some_condition: x = 1d else: x = Decimal(some_string) # ... y = x / 3 So now x / 3 rounds differently depending on whether x is a decimal64 or a Decimal. I probably don't want that. The solution: coerce to Decimal. But then why did I bother with the Decimal literal anyway? Decimal(1d) is hardly better than Decimal('1'). Oscar From wolfgang.maier at biologie.uni-freiburg.de Mon Mar 10 18:48:21 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Mon, 10 Mar 2014 17:48:21 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] Message-ID: <loom.20140310T181321-988@post.gmane.org> Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah: > [My apologies for being terse, I don't have much time to follow this discussion right now.] > Nick Coghlan <ncoghlan at gmail.com> wrote: >> I think users of decimal literals will just need to deal with the risk of unexpected rounding, as the alternatives are even more problematic. > That is why I think we should seriously consider moving to IEEE semantics for a decimal literal. Among other things: > - Always round the literal inputs. > - Supply IEEE contexts. > - Make Decimal64 the default. > - Add the missing rounding modes to sqrt() and exp(). > - Keep the ability to create exact Decimals through the constructor when no context is passed. > - Make the Decimal constructor use the context for rounding if it is passed. > - ... While I find this discussion about decimal literals extremely interesting, in my opinion, such a literal should have an underlying completely new numerical type, if it is really supposed to be for inexperienced users. Most of the discussions right now concern rounding issues that occur after the decimal point, but I think an at least equally big problem is rounding *to the left* of it as in (using current float): >>> 1e50 + 1000 1e+50 Importantly, Decimal is no cure here: >>> Decimal(10**50) + Decimal(100) Decimal('1.000000000000000000000000000E+50') (of course, you can debate context settings to make this particular example work, but, in general, it happens with big enough numbers.) The solution for this example is using ints of course: >>> 10**50 + 100 100000000000000000000000000000000000000000000000100 , but obviously this works only for whole numbers, so there currently is no built-in way to make this example work correctly: >>> 10**50 - 9999999999999999.5 1e+50 (or with Decimal: >>> Decimal(10**50) - Decimal('9999999999999999.5') Decimal('1.000000000000000000000000000E+50') ). If we are discussing a new number literal I would like to have it cope with this and my suggestion is a new numeric type that's sort of a hybrid between int and either Decimal or float, i.e., a type that behaves like int for digits left of the decimal point, but may truncate to the right. In pure Python, something similar (but not a literal form of course) could be implemented as a class that stores the left digits as an int and the right digits as a float/Decimal internally. Calculations involving this class would be slow due to the constant need of shifting digits between the integer?and the float part, and might be too slow for many users even when written in C, but its behavior would meet the expectation of inexperienced people better than the existing types. Going back to Mark Harris' initial proposal of unifying numeric types (which I am not trying to support in any way here), such a type would even allow to unify int and float since an ints could be considered a subset of the new type with a fractional part of zero. Cheers, Wolfgang From guido at python.org Mon Mar 10 18:56:20 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2014 10:56:20 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140310T181321-988@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> Message-ID: <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> On Mon, Mar 10, 2014 at 10:48 AM, Wolfgang Maier < wolfgang.maier at biologie.uni-freiburg.de> wrote: > Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah: > > > [My apologies for being terse, I don't have much time to follow this > discussion right now.] > > > Nick Coghlan <ncoghlan at gmail.com> wrote: > >> I think users of decimal literals will just need to deal with the risk > of > unexpected rounding, as the alternatives are even more problematic. > > > That is why I think we should seriously consider moving to IEEE semantics > for a decimal literal. Among other things: > > - Always round the literal inputs. > > - Supply IEEE contexts. > > - Make Decimal64 the default. > > - Add the missing rounding modes to sqrt() and exp(). > > - Keep the ability to create exact Decimals through the constructor when > no context is passed. > > - Make the Decimal constructor use the context for rounding if it is > passed. > > - ... > > While I find this discussion about decimal literals extremely interesting, > in my opinion, such a literal should have an underlying completely new > numerical type, if it is really supposed to be for inexperienced users. > > Most of the discussions right now concern rounding issues that occur after > the decimal point, but I think an at least equally big problem is rounding > *to the left* of it as in (using current float): > > >>> 1e50 + 1000 > 1e+50 > > Importantly, Decimal is no cure here: > > >>> Decimal(10**50) + Decimal(100) > Decimal('1.000000000000000000000000000E+50') > > (of course, you can debate context settings to make this particular example > work, but, in general, it happens with big enough numbers.) > > The solution for this example is using ints of course: > > >>> 10**50 + 100 > 100000000000000000000000000000000000000000000000100 > > , but obviously this works only for whole numbers, so there currently is no > built-in way to make this example work correctly: > > >>> 10**50 - 9999999999999999.5 > 1e+50 > > (or with Decimal: > >>> Decimal(10**50) - Decimal('9999999999999999.5') > Decimal('1.000000000000000000000000000E+50') > ). > > If we are discussing a new number literal I would like to have it cope with > this and my suggestion is a new numeric type that's sort of a hybrid > between > int and either Decimal or float, i.e., a type that behaves like int for > digits left of the decimal point, but may truncate to the right. > In pure Python, something similar (but not a literal form of course) could > be implemented as a class that stores the left digits as an int and the > right digits as a float/Decimal internally. Calculations involving this > class would be slow due to the constant need of shifting digits between the > integer?and the float part, and might be too slow for many users even when > written in C, but its behavior would meet the expectation of inexperienced > people better than the existing types. > Going back to Mark Harris' initial proposal of unifying numeric types > (which > I am not trying to support in any way here), such a type would even allow > to > unify int and float since an ints could be considered a subset of the new > type with a fractional part of zero. > I think what you're proposing here is variant of fixed-point numbers. The representation you seem to be looking for is an arbitrary-precision integer plus an exponent. A question for you: how would you treat results like 1/3? -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/e51f1b29/attachment-0001.html> From wolfgang.maier at biologie.uni-freiburg.de Mon Mar 10 19:03:12 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Mon, 10 Mar 2014 18:03:12 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> Message-ID: <loom.20140310T185941-251@post.gmane.org> Guido van Rossum <guido at ...> writes: > > > I think what you're proposing here is variant of fixed-point numbers. The representation you seem to be looking for is an arbitrary-precision integer plus an exponent. A question for you: how would you treat results like 1/3? > No, I don't think this is what I'm proposing. I said digits left of the decimal point should behave like a Python integer, i.e., have arbitrary precision, but to the right of it things should look like a float or Decimal with fixed precision. So 1/3 would just look like float(1/3) for example. Best, Wolfgang From stefan at bytereef.org Mon Mar 10 19:09:13 2014 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 10 Mar 2014 19:09:13 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> References: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> Message-ID: <20140310180913.GA13362@sleipnir.bytereef.org> Guido van Rossum <guido at python.org> wrote: > On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote: > On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote: > > That is why I think we should seriously consider moving to IEEE > semantics > for a decimal literal. > > I think that would make a *lot* of sense. > > What's the proposal? Just using decimal64 for decimal literals, or introducing > decimal64 as a new builtin type? (I could get behind either one.) IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows multiple discrete contexts: Decimal32, Decimal64, Decimal128, ... In theory this goes on ad infinitum, but the precision increases much slower than the exponents. Mike Cowlishaw's spec fills in the gaps by allowing arbitrary contexts. Additionally there are some minor differences, but if we make moderate changes to Decimal, we get strict IEEE behavior. So my specific proposal was: 1) Make those changes to Decimal (we can call the module decimal2 if backwards compatibility rules it out). The most important change relevant to this subthread is rounding all *literals* on input while preserving exact construction via Decimal(value). 2) Keep the arbitrary context facility. 3) Set IEEEContext(Decimal64) as the default. Users will most likely primarily use Decimal32, Decimal64 and Decimal128. (Many users will likely never change the context at all.) 4) Optional: Also use the function names from IEEE 754-2008. With these changes most users would /think/ that Decimal is a fixed width Decimal64 type. Advanced users can still change the context. I know for a fact that some users really like the option of increasing the precision temporarily. I have to think about the other solution (decimal64 only). At first glance it seems too restrictive, since I imagine users would at least want Decimal128, too. Additionally there is no speed benefit. Stefan Krah From guido at python.org Mon Mar 10 19:09:46 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2014 11:09:46 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140310T185941-251@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> Message-ID: <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> On Mon, Mar 10, 2014 at 11:03 AM, Wolfgang Maier < wolfgang.maier at biologie.uni-freiburg.de> wrote: > Guido van Rossum <guido at ...> writes: > > > > > > > I think what you're proposing here is variant of fixed-point numbers. The > representation you seem to be looking for is an arbitrary-precision integer > plus an exponent. A question for you: how would you treat results like 1/3? > > > > No, I don't think this is what I'm proposing. I said digits left of the > decimal point should behave like a Python integer, i.e., have arbitrary > precision, but to the right of it things should look like a float or > Decimal > with fixed precision. So 1/3 would just look like float(1/3) for example. > I'm sorry, I still don't understand your proposal. Suppose I have two numbers following that description and I divide them, and suppose the result cannot be represented as a decimal or binary fraction. What is your algorithm for deciding how many digits after the decimal point to keep? (Or how many bits after the binary point -- are you proposing a binary or a decimal representation?) Is the number of bits/digits to keep a constant, or does it vary by how many digits/bits are to the left of the point? And what about multiplications? Should they always produce an exact result or should they truncate if the result requires more digits/bits behind the point than a certain number? (If the latter, how is that limit determined?) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/316439d2/attachment.html> From tim.peters at gmail.com Mon Mar 10 20:08:49 2014 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 10 Mar 2014 14:08:49 -0500 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <20140310135824.GB10968@sleipnir.bytereef.org> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <20140310135824.GB10968@sleipnir.bytereef.org> Message-ID: <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com> [Mark Dickinson <dickinsm at gmail.com>] >> >>> -Decimal(0.0) # negative sign mysteriously disappears. >> >> Decimal('0') [Stefan Krah <stefan at bytereef.org>] > Unless the rounding mode is ROUND_FLOOR. :( > > >>> getcontext().rounding = ROUND_FLOOR > >>> -Decimal(0.0) > Decimal('-0') I may be missing something, but our docs don't appear to be clear about what unary minus means in Python's decimal implementation. Looks like Mike C's intent was that +x act like Decimal(0) + x and -x like Decimal(0) - x Then all the examples "make sense" (given that you think the standards' rules for adding/subtracting signed zeros make sense). Can't win! That -x loses the sign when x is a positive 0 (under most rounding modes) is jarring; but that the results of "-x" and "0-x" may not be the same would also be jarring. Not a fan of signed zeroes :-( From dickinsm at gmail.com Mon Mar 10 20:30:49 2014 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Mar 2014 19:30:49 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <20140310135824.GB10968@sleipnir.bytereef.org> <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com> Message-ID: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com> On Mon, Mar 10, 2014 at 7:08 PM, Tim Peters <tim.peters at gmail.com> wrote: > Looks like Mike C's intent was that > > +x > > act like > > Decimal(0) + x > [...] Yes, I'm pretty sure that's what he intended. It's clearly spelled out in the standard: "the operations plus(a) and minus(a) (where a and b refer to any numbers) are calculated as the operations add(?0?, a) and subtract(?0?, b)respectively", but not in our docs. Can't win! That -x loses the sign when x is a positive 0 (under most > rounding modes) is jarring; but that the results of "-x" and "0-x" may > not be the same would also be jarring. > Hmm. I find the second option a lot less jarring than the first, but maybe that's because I'm used to it (and because I'm too young to really remember non-IEEE floating-point formats; the only one I really encountered was the Sinclair ZX81's 40-bit format). Not a fan of signed zeroes :-( > Yeah, me neither, but with IEEE 754 world domination it looks like they're here to stay. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/26f2442a/attachment-0001.html> From oscar.j.benjamin at gmail.com Mon Mar 10 21:51:20 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 20:51:20 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140310180913.GA13362@sleipnir.bytereef.org> References: <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <20140310180913.GA13362@sleipnir.bytereef.org> Message-ID: <CAHVvXxQwqpgmdVeDh0PFVrxNg5Q3FmGExmNQGZgSSJuWOQGB6Q@mail.gmail.com> On 10 March 2014 18:09, Stefan Krah <stefan at bytereef.org> wrote: > Guido van Rossum <guido at python.org> wrote: >> On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >> On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote: >> >> That is why I think we should seriously consider moving to IEEE >> semantics >> for a decimal literal. >> >> I think that would make a *lot* of sense. >> >> What's the proposal? Just using decimal64 for decimal literals, or introducing >> decimal64 as a new builtin type? (I could get behind either one.) > > IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows > multiple discrete contexts: Decimal32, Decimal64, Decimal128, ... > > In theory this goes on ad infinitum, but the precision increases much > slower than the exponents. Mike Cowlishaw's spec fills in the gaps by > allowing arbitrary contexts. Additionally there are some minor differences, > but if we make moderate changes to Decimal, we get strict IEEE behavior. > > > So my specific proposal was: > > 1) Make those changes to Decimal (we can call the module decimal2 > if backwards compatibility rules it out). The most important > change relevant to this subthread is rounding all *literals* on > input while preserving exact construction via Decimal(value). > > 2) Keep the arbitrary context facility. > > 3) Set IEEEContext(Decimal64) as the default. Users will most > likely primarily use Decimal32, Decimal64 and Decimal128. > (Many users will likely never change the context at all.) > > 4) Optional: Also use the function names from IEEE 754-2008. I generally agree with the above except that... > With these changes most users would /think/ that Decimal is a > fixed width Decimal64 type. Advanced users can still change the > context. This is the problem I have with this particular proposal. Users would think that it's a fixed-width type and then write code that naively makes that assumption. Then the code blows up when someone else changes the arithmetic context. I don't think we should encourage non-expert users to think that they can safely rely on this behaviour without actually making it safe to rely on. > I know for a fact that some users really like the > option of increasing the precision temporarily. Agreed. I do this often. But I think it's the kind of thing that happens in a special library like Mark's pcdeclib rather than application code for a casual user. The option will always be there to promote your decimal64s to the Big Decimal type and do all your calculations in whichever precision you want. > I have to think about the other solution (decimal64 only). At first > glance it seems too restrictive, since I imagine users would at least > want Decimal128, too. Additionally there is no speed benefit. Decimal128 seems fine to me. I just think it should be a true fixed-width type. The benefits of this are: 1) Naive code doesn't get broken by different contexts. 2) I can tell by looking at a snippet exactly what it does without needing to wonder (or ask) whether or not the context has been fiddled with. 3) I can show code that uses decimal literals and the decimal128 constructor and guarantee that it works without caveats. 4) Since the meaning of any expression is known at compile time it is amenable to constant folding. 5) Unary + is a no-op and - is exact (as users would expect) so negative literals will have the same meaning regardless of the current context. Oscar From oscar.j.benjamin at gmail.com Mon Mar 10 21:58:18 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 10 Mar 2014 20:58:18 +0000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <20140310135824.GB10968@sleipnir.bytereef.org> <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com> <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com> Message-ID: <CAHVvXxT7ocoA2+q3d_j1YQN4zUXW_HRkLozELgbXjNz0nYcXzQ@mail.gmail.com> On 10 March 2014 19:30, Mark Dickinson <dickinsm at gmail.com> wrote: > On Mon, Mar 10, 2014 at 7:08 PM, Tim Peters <tim.peters at gmail.com> wrote: > >> Can't win! That -x loses the sign when x is a positive 0 (under most >> rounding modes) is jarring; but that the results of "-x" and "0-x" may >> not be the same would also be jarring. > > Hmm. I find the second option a lot less jarring than the first, I agree. Needing to call copy_negate just to flip the sign when unary - is exact for every other numeric seems very unfortunate to me. Oscar From ethan at stoneleaf.us Mon Mar 10 21:57:09 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 10 Mar 2014 13:57:09 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> Message-ID: <531E2725.5090907@stoneleaf.us> On 03/10/2014 11:09 AM, Guido van Rossum wrote: > > I'm sorry, I still don't understand your proposal. I think he's saying make a new type similarly to complex, only instead of two floats to make a complex number, have a (long) int and a decimal float to make this new type. The long int portion would have infinite precision, the float portion would have, say, 16 digits (or whatever). -- ~Ethan~ From rosuav at gmail.com Mon Mar 10 22:51:00 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 11 Mar 2014 08:51:00 +1100 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <531E2725.5090907@stoneleaf.us> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> Message-ID: <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at stoneleaf.us> wrote: > I think he's saying make a new type similarly to complex, only instead of > two floats to make a complex number, have a (long) int and a decimal float > to make this new type. The long int portion would have infinite precision, > the float portion would have, say, 16 digits (or whatever). That's plausible as a representation, and looks tempting, but basic arithmetic operations become more complicated. Addition and subtraction just need to worry about carries, but multiplication forks out into four multiplications (int*int, int*frac, frac*int, frac*frac), and division becomes similarly complicated. Would it really be beneficial? ChrisA From tim.peters at gmail.com Mon Mar 10 22:54:05 2014 From: tim.peters at gmail.com (Tim Peters) Date: Mon, 10 Mar 2014 16:54:05 -0500 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <20140310135824.GB10968@sleipnir.bytereef.org> <CAExdVNkzoqot7dxD0gOT05-ooDMTWoTDftFwXhWhs-QiQje+3w@mail.gmail.com> <CAAu3qLWHCn8gKMxA2_sHJ06XMzcAr-O6QePjwzMNnCbwUg8RTA@mail.gmail.com> Message-ID: <CAExdVN=XWsHUg1xjyWxjsV5XgeDWFGaXZgwH9v4BG9LQq7tY3g@mail.gmail.com> [Mark Dickinson] > Yes, I'm pretty sure that's what he intended. It's clearly spelled out in > the standard: "the operations plus(a) and minus(a) (where a and b refer to > any numbers) are calculated as the operations add('0', a) and subtract('0', > b)respectively", Ah. The great thing about standards is that there are so many of them :-( I was staring at 754-2008, and found nary a mention of "plus" or "minus" operations. But I found the quoted text at: http://speleotrove.com/decimal/daops.html#refplusmin where it also says they "correspond to the prefix minus and plus operators in programming languages". That doesn't leave much wiggle room for a conforming implementation. Next ;-) From guido at python.org Mon Mar 10 22:57:48 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2014 14:57:48 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> Message-ID: <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com> On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at stoneleaf.us> wrote: > > I think he's saying make a new type similarly to complex, only instead of > > two floats to make a complex number, have a (long) int and a decimal > float > > to make this new type. The long int portion would have infinite > precision, > > the float portion would have, say, 16 digits (or whatever). > > That's plausible as a representation, and looks tempting, but basic > arithmetic operations become more complicated. Addition and > subtraction just need to worry about carries, but multiplication forks > out into four multiplications (int*int, int*frac, frac*int, > frac*frac), and division becomes similarly complicated. Would it > really be beneficial? > It looks neither plausible nor tempting to me at all, and I hope that's not what he meant. It can represent numbers of any magnitude that have lots of zeros following the decimal point followed by up to 16 digits of precision, but not numbers that have e.g. lots of ones instead of those zeros -- the float portion would be used up for the first 16 ones. E.g. 111111111111111111111111111111.000000000000000000000000000000123456789 would be representable exactly but not 111111111111111111111111111111.111111111111111111111111111111123456789 What makes numbers in the vicinity of integers special? -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/2487bed8/attachment-0001.html> From rosuav at gmail.com Mon Mar 10 23:29:26 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 11 Mar 2014 09:29:26 +1100 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com> Message-ID: <CAPTjJmp08WNDiF=7w9ktxULr6gi0V6OKExmNpsHDDbAJO6LMDQ@mail.gmail.com> On Tue, Mar 11, 2014 at 8:57 AM, Guido van Rossum <guido at python.org> wrote: > It looks neither plausible nor tempting to me at all, and I hope that's not > what he meant. It can represent numbers of any magnitude that have lots of > zeros following the decimal point followed by up to 16 digits of precision, > but not numbers that have e.g. lots of ones instead of those zeros -- the > float portion would be used up for the first 16 ones. E.g. > > 111111111111111111111111111111.000000000000000000000000000000123456789 > > would be representable exactly but not > > 111111111111111111111111111111.111111111111111111111111111111123456789 > > What makes numbers in the vicinity of integers special? Hmm, good point. I was thinking this would give a predictable 16 digits of precision after the decimal, but the leading zeroes are somewhat special. But when I said "tempting" I meant that it looks initially nice, and then went on to show that it's not so nice on analysis - which latter part you're also demonstrating. ChrisA From wolfgang.maier at biologie.uni-freiburg.de Mon Mar 10 23:36:59 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Mon, 10 Mar 2014 22:36:59 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com> Message-ID: <loom.20140310T233009-70@post.gmane.org> Guido van Rossum <guido at ...> writes: > > > > On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at stoneleaf.us> wrote: > > > > I think he's saying make a new type similarly to complex, only instead of > > two floats to make a complex number, have a (long) int and a decimal float > > to make this new type. ?The long int portion would have infinite precision, > > the float portion would have, say, 16 digits (or whatever). > That's plausible as a representation, and looks tempting, but basic > arithmetic operations become more complicated. Addition and > subtraction just need to worry about carries, but multiplication forks > out into four multiplications (int*int, int*frac, frac*int, > frac*frac), and division becomes similarly complicated. Would it > really be beneficial? > > > It looks neither plausible nor tempting to me at all, and I hope that's not what he meant. It can represent numbers of any magnitude that have lots of zeros following the decimal point followed by up to 16 digits of precision, but not numbers that have e.g. lots of ones instead of those zeros -- the float portion would be used up for the first 16 ones. E.g.111111111111111111111111111111.000000000000000000000000000000123456789 > > would be representable exactly but not111111111111111111111111111111.111111111111111111111111111111123456789 > > > > > > What makes numbers in the vicinity of integers special? > I'm afraid it is exactly what I'm proposing. I don't see though how this is different from current behavior of lets say Decimal. Assuming default context with prec=28 you currently get: >>> +Decimal('0.000000000000000000000000000000123456789') Decimal('1.23456789E-31') , but with a "consumer 1" (one is enough, actually): >>> +Decimal('0.100000000000000000000000000000123456789') Decimal('0.1000000000000000000000000000') Cheers, Wolfgang From wolfgang.maier at biologie.uni-freiburg.de Mon Mar 10 23:42:38 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Mon, 10 Mar 2014 22:42:38 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> Message-ID: <loom.20140310T233952-563@post.gmane.org> Chris Angelico <rosuav at ...> writes: > > On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman <ethan at ...> wrote: > > I think he's saying make a new type similarly to complex, only instead of > > two floats to make a complex number, have a (long) int and a decimal float > > to make this new type. The long int portion would have infinite precision, > > the float portion would have, say, 16 digits (or whatever). > > That's plausible as a representation, and looks tempting, but basic > arithmetic operations become more complicated. Addition and > subtraction just need to worry about carries, but multiplication forks > out into four multiplications (int*int, int*frac, frac*int, > frac*frac), and division becomes similarly complicated. Would it > really be beneficial? > That's right, it would complicate calculations exactly as you are pointing out. That's why I said it might be too slow for most users even when implemented optimally, but then I'm not sure whether performance should be the very first thing to discuss. Best, Wolfgang From guido at python.org Mon Mar 10 23:47:03 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2014 15:47:03 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140310T233009-70@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> <CAP7+vJ+6Wi6EidX8GZwiCKhDwrdb5w8ja9tbBqpkkHo=ewbFNA@mail.gmail.com> <loom.20140310T185941-251@post.gmane.org> <CAP7+vJK-teHf5eCfjXcAcyNFOjLM=nDYcZ+BA-9_bG7dhxo=qQ@mail.gmail.com> <531E2725.5090907@stoneleaf.us> <CAPTjJmpX3c1vLUScEKVrZ5Louis8wbgK+DgB2aRZKftR088T+Q@mail.gmail.com> <CAP7+vJK6bC60r-bvwMOi-3uZsAS181iTZmbZJt_4BmgsgS_R0w@mail.gmail.com> <loom.20140310T233009-70@post.gmane.org> Message-ID: <CAP7+vJJtpcocJwDNE=h_HNYk23UwSLLFhWY3DcQDYRJ+L93FkQ@mail.gmail.com> On Mon, Mar 10, 2014 at 3:36 PM, Wolfgang Maier < wolfgang.maier at biologie.uni-freiburg.de> wrote: > Guido van Rossum <guido at ...> writes: > > > > > > > > > On Mon, Mar 10, 2014 at 2:51 PM, Chris Angelico > <rosuav at gmail.com> wrote: > > On Tue, Mar 11, 2014 at 7:57 AM, Ethan Furman > <ethan at stoneleaf.us> wrote: > > > > > > > I think he's saying make a new type similarly to complex, only instead > of > > > two floats to make a complex number, have a (long) int and a decimal > float > > > to make this new type. The long int portion would have infinite > precision, > > > the float portion would have, say, 16 digits (or whatever). > > That's plausible as a representation, and looks tempting, but basic > > arithmetic operations become more complicated. Addition and > > subtraction just need to worry about carries, but multiplication forks > > out into four multiplications (int*int, int*frac, frac*int, > > frac*frac), and division becomes similarly complicated. Would it > > really be beneficial? > > > > > > It looks neither plausible nor tempting to me at all, and I hope that's > not what he meant. It can represent numbers of any magnitude that have lots > of zeros following the decimal point followed by up to 16 digits of > precision, but not numbers that have e.g. lots of ones instead of those > zeros -- the float portion would be used up for the first 16 ones. > E.g.111111111111111111111111111111.000000000000000000000000000000123456789 > > > > would be representable exactly but > not111111111111111111111111111111.111111111111111111111111111111123456789 > > > > > > > > > > > > What makes numbers in the vicinity of integers special? > > > > > I'm afraid it is exactly what I'm proposing. I don't see though how this is > different from current behavior of lets say Decimal. > Assuming default context with prec=28 you currently get: > > >>> +Decimal('0.000000000000000000000000000000123456789') > Decimal('1.23456789E-31') > > , but with a "consumer 1" (one is enough, actually): > > >>> +Decimal('0.100000000000000000000000000000123456789') > Decimal('0.1000000000000000000000000000') > It is very different. Decimal with prec=28 counts the number of digits from the first non-zero digit, and the number of digits it gives you after the decimal point depends on how many digits there are before it. That is a sane perspective on precision (the total number of significant digits). But in your proposal the number of digits you get after the point depends on how close the value is to the nearest integer (in the direction of zero), not how many significant digits you have in total. That's why my examples started with lots of ones, not zeros. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140310/123c15de/attachment.html> From ncoghlan at gmail.com Tue Mar 11 00:02:31 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2014 09:02:31 +1000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> Message-ID: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> On 11 Mar 2014 01:02, "Guido van Rossum" <guido at python.org> wrote: > > On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson <dickinsm at gmail.com> wrote: >> >> On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah <stefan at bytereef.org> wrote: >> >>> That is why I think we should seriously consider moving to IEEE semantics >>> for a decimal literal. >> >> >> I think that would make a *lot* of sense. > > > What's the proposal? Just using decimal64 for decimal literals, or introducing decimal64 as a new builtin type? (I could get behind either one.) My take is that we're down to two main options: Stefan: use the existing decimal type, change default context to Decimal64, round all decimal literals to current context Oscar: new fixed width decimal64 builtin type, always uses Decimal64 context (allowing constant folding), interoperates with variable context decimal.Decimal values (producing a decimal.Decimal result) I lean towards Oscar's proposal, as it removes the hidden context dependent behaviour and makes the builtin decimals true constant values. Perhaps Stefan & Oscar would be willing to collaborate on a PEP that sums up the pros and cons of the two main alternatives (as well as some of the other variants that were discussed and rejected)? (Adding the IEEE contexts to the decimal module sounds like a good idea regardless) Cheers, Nick. > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/1d015b3d/attachment-0001.html> From ncoghlan at gmail.com Tue Mar 11 00:06:16 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2014 09:06:16 +1000 Subject: [Python-ideas] Changing Decimal.__pos__ and Decimal.__neg__ [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <20140310151923.GD12595@ando> References: <CAHVvXxT89zui=1oaT0Jpi9o9hfCSCkhtEtPk3ybnsDPAFAiPKg@mail.gmail.com> <CAAu3qLUUES2txFU_zk4oS6K+25gj7LCC5KH0TjwjbEfv-ipk7A@mail.gmail.com> <1394459709.30556.92666421.00C9A571@webmail.messagingengine.com> <20140310151923.GD12595@ando> Message-ID: <CADiSq7cAHtQ3SXGbSjAOrYRxzDv4h82qRShc4MaggFu3nfgrLw@mail.gmail.com> On 11 Mar 2014 01:20, "Steven D'Aprano" <steve at pearwood.info> wrote: > > On Mon, Mar 10, 2014 at 09:55:09AM -0400, random832 at fastmail.us wrote: > > On Mon, Mar 10, 2014, at 9:30, Mark Dickinson wrote: > > > I guess I have ulterior motives; (ab)using '+' to mean 'round to the > > > current context' has always felt way too implicit for my tastes, and I'd > > > jump at the chance to see it go away. > > > > It occurs to me that there is an unprecedented opportunity to make warts > > "go away" - making Decimal a "first-class citizen" suggests moving it > > from the decimal module to a built-in type. Any backwards-incompatible > > change you want to make, you can give programs importing decimal the old > > behavior, and reserve the new behavior for the new type in builtins. > > Historically, I don't think Python has changed the API for types when > they've moved from a module to a built-in. I can't think of any example > except set, but apart from a change in name from set.Set to set, I don't > think there were any changes. I believe there were a few changes in how it interoperated with other types, based on lessons learned from the module version (although I forget the details). Cheers, Nick. > > Any other precedents? > > On the other hand, the built-in doesn't need to be a replacement for the > decimal module, but rather an alternative solution. If decimal64 was a > built-in, presumably it wouldn't need contexts and would be a lot > simpler. The decimal module would still be available for those with > advanced needs. > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/62b899d9/attachment.html> From steve at pearwood.info Tue Mar 11 02:17:35 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 11 Mar 2014 12:17:35 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com> References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAHVvXxTSg1dG3OCRrGHd7w+2WA3s7QKjyMtAhfEFGXoqc6j2QA@mail.gmail.com> <20140310153127.GF12595@ando> <CAHVvXxSTK9+1P38SWuPAk+XSr2Gr5k1z-0QnSuvW4zDCALqQ0A@mail.gmail.com> Message-ID: <20140311011735.GJ12595@ando> On Mon, Mar 10, 2014 at 04:00:38PM +0000, Oscar Benjamin wrote: > On 10 March 2014 15:31, Steven D'Aprano <steve at pearwood.info> wrote: > > On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote: > >> Where this would get complicated is for people who also use the > >> Decimal type. They'd need to keep track of which objects were of which > >> type and so decimal literals might seem more annoying than useful. > > Hmmm. I don't think this should necessarily be complicated, or at least > > no more complicated than dealing with any other mixed numeric types. If > > you don't want to mix them, don't mix them :-) > > Exactly. It just means that you wouldn't want to use the literals in > code that actually uses the decimal module. Consider: If you're writing a function that accepts whatever random numeric type the caller passes in, you have to expect that you cannot control the behaviour precisely. (You can't control rounding or precision for floats; Fractions are exact; the proposed decimal64 will have a fixed precision and fixed rounding; decimal.Decimal you can control.) You might still write type-agnostic code, but the result you get may depend on the input data. > if some_condition: > x = 1d > else: > x = Decimal(some_string) Mixing different types within a single calculation/function, as shown above, sounds like a recipe for chaos to me. I wouldn't write it like that, any more than I would write: if some_condition: x = 1.1 else: x = Decimal("2.2") > # ... > y = x / 3 > > So now x / 3 rounds differently depending on whether x is a decimal64 > or a Decimal. I probably don't want that. Then don't do it! :-) > The solution: coerce to Decimal. Or write both branches as a Decimal in the first place. > But then why did I bother with the Decimal literal anyway? > Decimal(1d) is hardly better than Decimal('1'). Presumably you didn't use a decimal64 literal if you cared about controlling the rounding or precision, because that would be silly. (You might, on the other hand, use decimal64 literals everywhere in your function, if its rounding and precision was sufficient for your needs.) However, the caller of your function may pass you a decimal64 instance. I don't think there's much advantage to passing integer values with the d suffix, but passing fractional values is a different story: some_function(1.1) versus some_function(1.1d) I think this is a proposal I could get behind: leave the decimal module (mostly) as-is, perhaps with a few backwards-compatible tweaks, while adding an independent decimal64 or decimal128 built-in type and literal with fixed rounding and precision. The new literal type would satisfy the use-case Mark Harris is worried about, apart from having to add a "d" suffix to literals[1], while the decimal module still allows for fine control of the context. The built-in decimal type should convert floats using their repr, which ought to make Guido happy too :-) [1] Shifting to decimal floats by default is a major backwards-incompatible change. -- Steven From cfkaran2 at gmail.com Tue Mar 11 02:45:17 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Mon, 10 Mar 2014 21:45:17 -0400 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <531BB061.2060009@canterbury.ac.nz> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> <531BB061.2060009@canterbury.ac.nz> Message-ID: <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com> Is there any further interest in standardized annotations, or should the idea be abandoned? Thanks, Cem Karan On Mar 8, 2014, at 7:05 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Cem Karan wrote: >> There are a number of reasons I'm suggesting UUIDs instead of simple strings: > > I'm not talking about strings, I'm talking about > objects created and exported by the module defining > the annotations, and compared by identity. > > The Python module namespace then ensures they have > unique names within any given program. That's all > you need, because there's no requirement to persist > them from one program execution to another. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From tjreedy at udel.edu Tue Mar 11 07:15:23 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Mar 2014 02:15:23 -0400 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> Message-ID: <lfm9lu$7s3$1@ger.gmane.org> On 3/10/2014 7:02 PM, Nick Coghlan wrote: > My take is that we're down to two main options: > > Stefan: use the existing decimal type, change default context to > Decimal64, round all decimal literals to current context > > Oscar: new fixed width decimal64 builtin type, always uses Decimal64 > context (allowing constant folding), interoperates with variable context > decimal.Decimal values (producing a decimal.Decimal result) > > I lean towards Oscar's proposal, as it removes the hidden context > dependent behaviour and makes the builtin decimals true constant values. Ditto for me. An everyday replacement for binary floats should be about as simple as binary floats, and simpler in the sense of having less surprising behavior. -- Terry Jan Reedy From ron3200 at gmail.com Tue Mar 11 07:36:38 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 11 Mar 2014 01:36:38 -0500 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140310T181321-988@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> Message-ID: <lfmat9$j4s$1@ger.gmane.org> On 03/10/2014 12:48 PM, Wolfgang Maier wrote: > Going back to Mark Harris' initial proposal of unifying numeric types (which > I am not trying to support in any way here), such a type would even allow to > unify int and float since an ints could be considered a subset of the new > type with a fractional part of zero. This turned out a bit longer than I intended. I think it is the correct approach to what I think is the intent of the number unifications suggestions. Trim it down a bit, and add some examples and it's a good start to a PEP. ;-) I think adding a new number type isn't going to help that much, although it won't hurt either. (Hoping this is a more practical direction for these discussions.) General Proposal: ================= Add a BaseValue class for creating context dependent values like a CurrencyValue class, or StatisticValue class. The Value class should not specify any particular number type. That is an implementation detail of a subclass. The Value class will define how to round, compare, and track error ranges when used in calculations. And will use ValueContext objects to adjust that as needed. Add a ValueContext class to create context parameter objects. These objects will be used to set a Value class's context for rounding, number of significant digits, and tracking error ranges. The ValueContext objects should be very simple, and loosely defined in the same way slice objects are. They are only used for passing the context parameters. (Like slice objects pass index's, but probably by called methods rather than by syntax.) Introduction: ============= There have been a number of suggestions to unify numbers in order to make them easier to use for less mathematically inclined users to do more complex calculations without having to know quite as much about the underlying type of number. There is no magic wand that will make it completely painless, but we can make things easier to get right to an average programmer with average to good math skills. (Probably the normal group for most beginner programmer students.) The basic issue I see isn't with the number types, but with how we use them. (And how we can us them better) There are a lot of parts that need to work both independently and together at the same time. Currently It's up to the programmer to keep track of significant digits (and significant error), any rounding that may need to be done, and to do it all correctly. There are many places where it can go wrong, and where getting it right requires a good understanding of these concepts within the speciality they are being used in. The Decimal module tries to reduce that work, and also reduce errors, and increase overall accuracy, all at the same time. But that doesn't help the existing number types. Also while just using the decimal type isn't that difficult, using the other features it has correctly isn't as easy. It's not easy to mix and match these ideas and come up with a usable, and reusable, interface for doing many similar types of calculations correctly as each value in a calculation (or function call), may not need the same exact requirements. Possibly the broadest, but still practical viewpoint, gives three kinds of numbers. A number kind as described here, refers to how they are used, and is independent of the way it's stored in the computer. The number kind is what determines some of the requirements of a calculation or comparison. If we consider that how a number is used as independent (as much as is reasonable) from how the number is stored in memory, then maybe the type is less important than the context. Kinds of Values: ================ 1. Measurements: Values which have limited accuracy and need both proper rounding and tracking of error ranges. 2. Exact numbers: Values that really do represent exactly what they are. Usually smaller sized numbers, and usually not fractional amounts. (includes counters and index's for example) 3. Ideal numbers: Values that are not measurements but can be calculated as accurately as needed and generally don't require keeping track of error because the significant digits can far exceed any measurements. - (Unless you are a theoretical physicist.) - (Low quality approximations should be considered a measurement, even if it could be calculated accurately to any nth digit.) Ints, and floating point actually do a very nice job of representing 2. and 3 in most cases. And they generally don't need to have a contexts associated to them, and usually don't need rounding. For example, we don't want to round index's and counters, they are in the exact group. And we can calculate Pi to what ever we need so that any error range in the value is less than significant. Equality Tests: =============== Another issue is with equality and how a value's exactness effects that. For example if we have two values with an error of +- 1. We can say it has a width of 3. (Or inexactness of 3) So we might have this case where two values overlap when you consider the width of the value. (it's +- amount) (23 +- 1) < (24 +- 1) --> False Of course those would be objects instead of literals. (weather or not they are decimal, int, or float, isn't important.) This can be extended to all the other comparison operators. The usual way of dealing with it is to do an error range test, which adds a lot of noise to an otherwise simple operation. Especially if you need to put those in many different places. They aren't quite complex enough to justify functions, but complex enough to be bothersome and can be a common source of errors. Exactness, (and rounding, as it's directly related to exactness), are independent concepts from how the number is stored in a computer. (Unless you use the wrong type of course, but then I would consider it a bug.) Values vs Numbers: ================= One approach is to have a Value class, and a ValueContext object. The ValueContext object may be more like a slice object. It would be up to the Value class to interpret it so that it can know how to do the operator methods correctly... __round__, __add__, __eq__, etc... A base Value class could be used to define types such as a Measurement class, Or a Currency class, (and others). The context API needs to be on a separate Value class/type rather than on the number in my opinion. I think trying to make numbers type also be a value is what gets us mixed up. The number is just a scaler for the value. It might also be called vector, but that is a technical term that can be confused with other concepts. Benefits of a Value class: ========================== * Doesn't change any existing number type. * Value objects work with any number types. * Value objects can have a repr that is much nicer, and more meaningful than the underlying number type. * Context sub-class's define how they are used (or act) rather than by what kind of computer number they may contain. * Context (slice like objects) that can be shared, and/or passed between Value objects as needed to keep track of rounding, significant digits, and error range. * Can be a library module. If Value objects is something that can be included into python, then a simpler decimal64 type without all the context settings could be used as it's just another computer number type with a bit more accuracy if you need it. A Currency Value type could use Decimal, and the type can be swapped out later for the decimal64, as that becomes just an implementation detail for he Currency Value class. Users could do something like that now by defining classes, and probably have done it within their applications. I consider that is a supporting argument for this approach, rather than an argument against it. It still takes a fair amount of math knowledge how to get it right. Making context dependent calculations easier to do, and use, would be very nice, and modules using these Value objects could possibly work together better. A good test might be how "Value" objects could be used in the statistics module. Or maybe Mark Harris could try out this idea in his new module. I think this is the sort of thing Mark was initially looking for. It's really not that complicated if the order of dependencies is correctly ordered. (probably he's got something close to this in it already, just with a different name. (Just a guess, but any solution that solves some of this may not be that far off.) BTW, The new literals would be helpful with this too. +1 Cheers, Ron From solipsis at pitrou.net Tue Mar 11 08:24:49 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Mar 2014 08:24:49 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> Message-ID: <20140311082449.4f0c0fdb@fsol> On Tue, 11 Mar 2014 09:02:31 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote: > > My take is that we're down to two main options: > > Stefan: use the existing decimal type, change default context to Decimal64, > round all decimal literals to current context > > Oscar: new fixed width decimal64 builtin type, always uses Decimal64 > context (allowing constant folding), interoperates with variable context > decimal.Decimal values (producing a decimal.Decimal result) > > I lean towards Oscar's proposal, as it removes the hidden context dependent > behaviour and makes the builtin decimals true constant values. Yuck. Is this some kind of joke? We've gone through the trouble of unifying long and int in Python 3 and now people are proposing two different Decimal types, one with fixed precision and one with arbitrary precision? I'm completely -1 on this. Regards Antoine. From stephen at xemacs.org Tue Mar 11 10:28:32 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 11 Mar 2014 18:28:32 +0900 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> <531BB061.2060009@canterbury.ac.nz> <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com> Message-ID: <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp> Cem Karan writes: > Is there any further interest in standardized annotations, or > should the idea be abandoned? Obviously there's interest; standards are a good thing when you're trying to share. But not if they end up getting in the way of sharing because they're too limited or you end up with a bunch of standards such that no program can conform with all of them. To avoid the latter, you need to provide an implementation and show that it's useful by waiting for it to be used. You're not going to get a standard in to the stdlib at this point because there's not enough usage of *any* proposed annotation standard. If you want to make progress on this, just do it, and worry about getting it in to the stdlib later. To see what it takes to go directly into the stdlib, consider the PEP 461 debate. There was no need to provide an implementation and wait for usage to follow *because %-formatting for binary was already in widespread practical use in Python 2*. It was pretty clear that the default was going to be "just like Python 2", and that's how it ended up -- with the exception of "%r" because that would do the wrong thing in the intended use case (and "%a" does an equivalent right thing). From cfkaran2 at gmail.com Tue Mar 11 10:45:33 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 11 Mar 2014 05:45:33 -0400 Subject: [Python-ideas] Suggestion for standardized annotations In-Reply-To: <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp> References: <C77F2E59-0E8E-424A-9A23-BEECBB49DAD2@gmail.com> <CACac1F_e8YKCVc_U0c2v52iR37jozKwD6iPB_1+GF47rurtAvw@mail.gmail.com> <CADiSq7cC6JO-G_v5siayomSJS+2x=jD5UQ7J2Z+vUS-9_w29hg@mail.gmail.com> <CAENTz_UPN-GF807nr5f-m46yXXMj2igDVtVUs77Ztv+cChgchg@mail.gmail.com> <5317FDB6.5000806@canterbury.ac.nz> <F85B0726-4698-40EF-B5CC-739B5BEEA771@gmail.com> <531BB061.2060009@canterbury.ac.nz> <9434953A-4005-4B50-AF8F-04136F0779E8@gmail.com> <87d2htnkxr.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <3C102356-A26B-46FF-8FDB-9FE00AFD6260@gmail.com> Oh, I didn't think it would get into the standard library in one shot, that's for sure! I just wanted to gauge interest to see if I should continue working on it and promoting it. I'll go ahead and do so, and put it up on pypi. Thanks, Cem Karan On Mar 11, 2014, at 5:28 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > Cem Karan writes: > >> Is there any further interest in standardized annotations, or >> should the idea be abandoned? > > Obviously there's interest; standards are a good thing when you're > trying to share. But not if they end up getting in the way of sharing > because they're too limited or you end up with a bunch of standards > such that no program can conform with all of them. > > To avoid the latter, you need to provide an implementation and show > that it's useful by waiting for it to be used. You're not going to > get a standard in to the stdlib at this point because there's not > enough usage of *any* proposed annotation standard. > > If you want to make progress on this, just do it, and worry about > getting it in to the stdlib later. > > To see what it takes to go directly into the stdlib, consider the PEP > 461 debate. There was no need to provide an implementation and wait > for usage to follow *because %-formatting for binary was already in > widespread practical use in Python 2*. It was pretty clear that the > default was going to be "just like Python 2", and that's how it ended > up -- with the exception of "%r" because that would do the wrong thing > in the intended use case (and "%a" does an equivalent right thing). From oscar.j.benjamin at gmail.com Tue Mar 11 11:51:28 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 10:51:28 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140311082449.4f0c0fdb@fsol> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> Message-ID: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> On 11 March 2014 07:24, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Tue, 11 Mar 2014 09:02:31 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: >> >> My take is that we're down to two main options: >> >> Stefan: use the existing decimal type, change default context to Decimal64, >> round all decimal literals to current context >> >> Oscar: new fixed width decimal64 builtin type, always uses Decimal64 >> context (allowing constant folding), interoperates with variable context >> decimal.Decimal values (producing a decimal.Decimal result) >> >> I lean towards Oscar's proposal, as it removes the hidden context dependent >> behaviour and makes the builtin decimals true constant values. > > Yuck. Is this some kind of joke? We've gone through the trouble of > unifying long and int in Python 3 and now people are proposing two > different Decimal types, one with fixed precision and one with > arbitrary precision? > > I'm completely -1 on this. I understand your objection Antoine. There is a big difference though between integer and non-integer types in this respect though. For integers is is not hard to define a single integer type that satisfies the vast majority of use-cases (focusing purely on semantics rather than detailed implementation aspects and performance). The only contentious issue is how to handle inexact division. int and long were not really "unified" in Python 3. The long type *replaced* the int type as it has those universal integer semantics that are really wanted. When it comes to non-integer types there's just no single good way of doing it. So just in the stdlib we already have float, Fraction and Decimal. The additional libraries that I use have many more numeric types than that. The idea here is that for most users decimal128 just is *the* decimal type. The decimal.Decimal type is IMO overly complex and should really be for niche use-cases like other non-stdlib multi-precision numeric types. Java's BigDecimal is more user friendly in some important ways. It performs all calculations exactly until you explicitly round. When you ask it to do something that must be inexact without explicitly supplying a rounding context it throws an exception. Naturally it is more verbose than calculations with decimal.Decimal but it is possible to always know what's going on just by looking at the relevant code. C# has a (non-standard I think) fixed-width decimal type. Once again the precision and exponent range of this type are not governed by a hidden modifiable arithmetic context so it is possible to know what a snippet of code does before run-time. C++ is introducing decimal32, decimal64, and decimal128 as fixed-width decimal floating point types as according to IEEE-754-2008. These will be fixed-width types whose precision is always known so that a static analysis can explain what code does (excepting traps and flags - I'm not clear on how they will work in C++). The problem as I see it is that if Decimal is brought into the core of the language in its current format then many people will be confused by its behaviour. The behaviour of Python's Decimal type is governed by the hidden global variable that is the arithmetic context so that even trivial looking code cannot fully be understood without thinking through all the possible contexts that might be in effect when it is executed. I've used a few multi-precision libraries and I find Decimal hard to use because of the action at a distance effect of the contexts and the fact that it is possible for Decimals to exist whose precision exceeds the context so that even the expression -x is subject to rounding. The idea is here is to make decimal floating point accessible for non-expert users. I'm concerned that the current Decimal type is overly complex for this use-case. Oscar From ncoghlan at gmail.com Tue Mar 11 12:46:27 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2014 21:46:27 +1000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> Message-ID: <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> On 11 March 2014 20:51, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > On 11 March 2014 07:24, Antoine Pitrou <solipsis at pitrou.net> wrote: >> On Tue, 11 Mar 2014 09:02:31 +1000 >> Nick Coghlan <ncoghlan at gmail.com> wrote: >>> >>> My take is that we're down to two main options: >>> >>> Stefan: use the existing decimal type, change default context to Decimal64, >>> round all decimal literals to current context >>> >>> Oscar: new fixed width decimal64 builtin type, always uses Decimal64 >>> context (allowing constant folding), interoperates with variable context >>> decimal.Decimal values (producing a decimal.Decimal result) >>> >>> I lean towards Oscar's proposal, as it removes the hidden context dependent >>> behaviour and makes the builtin decimals true constant values. >> >> Yuck. Is this some kind of joke? We've gone through the trouble of >> unifying long and int in Python 3 and now people are proposing two >> different Decimal types, one with fixed precision and one with >> arbitrary precision? >> >> I'm completely -1 on this. > > I understand your objection Antoine. There is a big difference though > between integer and non-integer types in this respect though. For > integers is is not hard to define a single integer type that satisfies > the vast majority of use-cases (focusing purely on semantics rather > than detailed implementation aspects and performance). The only > contentious issue is how to handle inexact division. int and long were > not really "unified" in Python 3. The long type *replaced* the int > type as it has those universal integer semantics that are really > wanted. > > When it comes to non-integer types there's just no single good way of > doing it. So just in the stdlib we already have float, Fraction and > Decimal. The additional libraries that I use have many more numeric > types than that. The idea here is that for most users decimal128 just > is *the* decimal type. The decimal.Decimal type is IMO overly complex > and should really be for niche use-cases like other non-stdlib > multi-precision numeric types. Right. I believe the appropriate comparison here should be with builtin floats (which are essentially 64 bit C doubles, with all the limitations that implies), rather than with the historical int/long situation. The reason I see Oscar's proposal as arguably different from the int/long case is that we *wouldn't* be trying to pretend that builtin decimals are the same type as the existing variable context decimals. While they would implicitly interoperate under addition, etc (as numbers generally do, with variable precision decimals being treated as the resulting type for mixed arithmetic), there would be no way to implicitly turn two fixed precision decimals into variable precision ones. That is, don't think "int vs long", think "float16 vs float32 vs float64 vs float128", with Oscar's proposal being that picking just one decimal floating point precision as *the* floating point precision, just as Python already does for binary floating point, makes more sense than trying to provide variable precision support in a builtin type. That approach buys us a couple of key benefits: - they would be true constants, so constant folding, etc, would work normally, and you could tell from reading the code exactly what it will do, without needing to worry about the impact of thread local state - you could generally use the builtin decimal literals without learning anything about decimal contexts, because decimal literals wouldn't be context dependent (although their behaviour would be formally defined in terms of an IEEE decimal context, for most users it would just be "this is how Python decimal literals behave") The guidance to new users would then be *don't* use the decimal module, use decimal literals instead. If you need more configurable behaviour, *then* reach for the decimal module. However, the explicit methods on decimal context objects should probably still be updated to accept both fixed and variable precision decimals under this model. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mal at egenix.com Tue Mar 11 13:33:10 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 11 Mar 2014 13:33:10 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> Message-ID: <531F0286.8090306@egenix.com> On 11.03.2014 12:46, Nick Coghlan wrote: > ...[Oscar's proposal being that picking just one decimal floating > point precision as *the* floating point precision]... > > That approach buys us a couple of key benefits: > > - they would be true constants, so constant folding, etc, would work > normally, and you could tell from reading the code exactly what it > will do, without needing to worry about the impact of thread local > state > - you could generally use the builtin decimal literals without > learning anything about decimal contexts, because decimal literals > wouldn't be context dependent (although their behaviour would be > formally defined in terms of an IEEE decimal context, for most users > it would just be "this is how Python decimal literals behave") > > The guidance to new users would then be *don't* use the decimal > module, use decimal literals instead. If you need more configurable > behaviour, *then* reach for the decimal module. However, the explicit > methods on decimal context objects should probably still be updated to > accept both fixed and variable precision decimals under this model. I think you are leaving out one of the most important use cases for decimal values: data input and output. The literals would only appear in programs. However, in most use cases, you want to read decimal data from some source, process it and then write it back again. This would most likely also require using a few decimal literals, but the main source of decimals would still be constructors that you choose when writing the tools. Coming back to floats and the decimal constructor: In investment banking, for example, people usually work with floats all the time. You only convert back to decimals at the very end of some calculation. The reason here being either that the calculations involve complex operations which are not available for decimals, trying to keep error intervals small, or a combination of both. In general, you try to use as much precision as you can afford (in terms of memory and speed), to keep those error intervals small. In accounting, you often use decimals for storing data with an (usually contractually or law based) agreed upon precision and rounding logic. However, there situations where you have to go to floats as well in order to run calculations, e.g. for interest, taxes, etc. In both situations, you want to have the decimal constructor take the float values with full precision and only then apply the necessary rounding to turn the value into a form which complies with the agreed upon rules. It's also not uncommon to add correctional bookings to address rounding issues explicitly (e.g. when calculating VAT of a large number of individual items). In short: you try to prevent rounding from happening as much as possible and when using it, you use it in a controlled way. Based on this, the choice to have the decimal constructor use full precision when reading floats is a good one, even though it may not feel right for the novice, the casual decimal user or as human concept :-) For decimal literals, I'd argue that if you enter a value 1.2500d, you are expecting a decimal with 4 decimal places precision, not 64 or 128 bits :-) The information about the intended precision is implicit in the literal. You see this done in exchange rates, stock prices, prices at your grocery store, etc. The decimals can then be transformed into ones with higher precision during calculations and then possibly back to lower precision, but this is an explicit decision by the system doing the calculation. Anyway, just throwing in some additional entropy into this discussion. Probably not all that helpful, since you're already converging on two possible solutions :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 11 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-04-09: PyCon 2014, Montreal, Canada ... 29 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From oscar.j.benjamin at gmail.com Tue Mar 11 13:44:00 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 12:44:00 +0000 Subject: [Python-ideas] %-formatting with Decimals Message-ID: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> During recent discussions I came across something unfortunate with formatting Decimals. Formatting with .format works as I would want: >>> from decimal import Decimal as D >>> '{:1.0e}'.format(D('1.123e+1000')) '1e+1000' >>> '{:1.0e}'.format(D('1.123e-1000')) '1e-1000' >>> '{:.50f}'.format(D('1.1')) '1.10000000000000000000000000000000000000000000000000' But %-formatting coerces to float: >>> '%1.0e' % D('1.123e-1000') '0e+00' >>> '%1.0e' % D('1.123e+1000') 'inf' >>> '%.50f' % D('1.1') '1.10000000000000008881784197001252323389053344726562' Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved? Oscar From oscar.j.benjamin at gmail.com Tue Mar 11 14:04:50 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 13:04:50 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <531F0286.8090306@egenix.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> <531F0286.8090306@egenix.com> Message-ID: <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com> On 11 March 2014 12:33, M.-A. Lemburg <mal at egenix.com> wrote: > On 11.03.2014 12:46, Nick Coghlan wrote: >> >> ...[Oscar's proposal being that picking just one decimal floating >> point precision as *the* floating point precision]... > > I think you are leaving out one of the most important use cases > for decimal values: data input and output. > > The literals would only appear in programs. However, in most use > cases, you want to read decimal data from some source, process it > and then write it back again. This would most likely also require > using a few decimal literals, but the main source of decimals > would still be constructors that you choose when writing the > tools. True, and the Decimal constructor does what you want. The decimal128 constructor will also do what you want provided you don't need more than 34 digits. In my proposal you would still be able to trap Inexact with decimal128 if desired. > Coming back to floats and the decimal constructor: > > In investment banking, for example, people usually work with floats > all the time. You only convert back to decimals at the very end of some > calculation. The reason here being either that the calculations > involve complex operations which are not available for decimals, > trying to keep error intervals small, or a combination of both. > In general, you try to use as much precision as you can afford > (in terms of memory and speed), to keep those error intervals > small. > > In accounting, you often use decimals for storing data with > an (usually contractually or law based) agreed upon precision > and rounding logic. Well this is a situation where you would definitely want all of the bells and whistles of the full Decimal module. You could still use decimal literals in this code and they would inter-operate as expected. > However, there situations where you have > to go to floats as well in order to run calculations, e.g. > for interest, taxes, etc. > > In both situations, you want to have the decimal constructor > take the float values with full precision and only then apply > the necessary rounding to turn the value into a form which > complies with the agreed upon rules. It's also not uncommon to > add correctional bookings to address rounding issues explicitly > (e.g. when calculating VAT of a large number of individual > items). > > In short: you try to prevent rounding from happening as > much as possible and when using it, you use it in a > controlled way. > > Based on this, the choice to have the decimal constructor > use full precision when reading floats is a good one, even > though it may not feel right for the novice, the casual > decimal user or as human concept :-) Don't worry, I think that the suggestion to change the way that Decimal(float) works is settled now. > For decimal literals, I'd argue that if you enter a > value 1.2500d, you are expecting a decimal with 4 decimal > places precision, not 64 or 128 bits :-) The decimal32/64/128 types can store the number in this form. Unlike with binary floats the trailing zeros are not discarded. > The information about the intended precision is implicit in the > literal. You see this done in exchange rates, stock prices, > prices at your grocery store, etc. > > The decimals can then be transformed into ones with higher > precision during calculations and then possibly back to lower > precision, but this is an explicit decision by the system doing > the calculation. The current behaviour will not be changed in this respect. There are two separate things that you are referring to though. One is the precision information associated with an individual Decimal e.g. the number of significant digits even if some are trailing zeros. The Decimal type preserves this information and so do the decimal64/128 formats: >>> Decimal('10.1000') Decimal('10.1000') >>> Decimal('10.1000e-5') Decimal('0.000101000') The other precision is the limiting precision of the current arithmetic context that is used when rounding. If you have a legal obligation to make that rounding be just so then you're into what I would consider to be expert usage of the decimal module. > Anyway, just throwing in some additional entropy into this > discussion. Probably not all that helpful, since you're already > converging on two possible solutions :-) Thanks! Oscar From storchaka at gmail.com Tue Mar 11 14:15:16 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Tue, 11 Mar 2014 15:15:16 +0200 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> Message-ID: <lfn280$851$1@ger.gmane.org> 11.03.14 14:44, Oscar Benjamin ???????(??): > Am I doing this wrong? Is this just a limitation of the old-style % > formatting or something that could possibly be improved? Yes, this is a limitation of the old-style % formatting. Another example: >>> '{:.2f}'.format(1.234j) '0.00+1.23j' >>> '%.2f' % 1.234j Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can't convert complex to float From solipsis at pitrou.net Tue Mar 11 14:30:03 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Mar 2014 14:30:03 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> Message-ID: <lfn34f$k4q$1@ger.gmane.org> Le 11/03/2014 11:51, Oscar Benjamin a ?crit : > > When it comes to non-integer types there's just no single good way of > doing it. So just in the stdlib we already have float, Fraction and > Decimal. The additional libraries that I use have many more numeric > types than that. The idea here is that for most users decimal128 just > is *the* decimal type. Who are those "most users"? Those who are currently happy with float? Defining two different but similar types, one for "beginner use" and one for "expert use" is a terrible design pattern. People *will* have to know about both types. Some libraries will choose to return the beginner type, others will choose to return the expert type. Users will have to learn the differences, will have to learn how conversions work, etc. > The decimal.Decimal type is IMO overly complex > and should really be for niche use-cases like other non-stdlib > multi-precision numeric types. What is complex about it? The notion of a decimal's precision in number of digits is understandable by a 10 year old. It is arguably much simpler than unicode and character encodings. Besides, for most applications Decimal's current precision will be good enough: changing the current context (which, I believe, is thread-local, not process-global) is only necessary in special cases. > Java's BigDecimal is more user friendly in some important ways. It > performs all calculations exactly until you explicitly round. So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-) (or perhaps it doesn't have .sqrt()?) > C++ is introducing decimal32, decimal64, and decimal128 as fixed-width > decimal floating point types as according to IEEE-754-2008. C++ has tons of integer types too, but Python has a single one. Regards Antoine. From ncoghlan at gmail.com Tue Mar 11 14:42:19 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2014 23:42:19 +1000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfn34f$k4q$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> Message-ID: <CADiSq7c4jpyA1NnXRBd9yfvZ7bcOS0TQdERcE7sYOkBx0nNTzQ@mail.gmail.com> On 11 Mar 2014 23:31, "Antoine Pitrou" <solipsis at pitrou.net> wrote: > > Le 11/03/2014 11:51, Oscar Benjamin a ?crit : > > >> The decimal.Decimal type is IMO overly complex >> and should really be for niche use-cases like other non-stdlib >> multi-precision numeric types. > > > What is complex about it? What is the result of "Decimal('1.0') + Decimal('1e70')"? Correct answer: insufficient data (since we don't know the current precision). That becomes a substantially more questionable answer for "1.0d + 1e70d". Having apparent numeric literals that can't be constant folded would be a much bigger design wart than having two different kinds of decimal. Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/193e0481/attachment.html> From oscar.j.benjamin at gmail.com Tue Mar 11 15:40:09 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 14:40:09 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfn34f$k4q$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> Message-ID: <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> On 11 March 2014 13:30, Antoine Pitrou <solipsis at pitrou.net> wrote: > Le 11/03/2014 11:51, Oscar Benjamin a ?crit : >> >> The decimal.Decimal type is IMO overly complex >> and should really be for niche use-cases like other non-stdlib >> multi-precision numeric types. > > What is complex about it? > The notion of a decimal's precision in number of digits is understandable by > a 10 year old. I agree, that's why I say that we just have a type with a fixed maximum number of digits. I think lots of people will be able to understand that. > It is arguably much simpler than unicode and character > encodings. Besides, for most applications Decimal's current precision will > be good enough: changing the current context (which, I believe, is > thread-local, not process-global) is only necessary in special cases. The problem is that the context cannot be depended upon so the result of trivial calculations or even a negative decimal literal will depend on something that can be changed *anywhere* else in the code. The expression -1.123456789d will evaluate differently depending on the context and I think many people will be surprised by that. I expect that this would lead to naive code that breaks when the context is changed. Great care needs to be taken to get this right, or otherwise it should just be considered generally unsafe to change the context. >> Java's BigDecimal is more user friendly in some important ways. It >> performs all calculations exactly until you explicitly round. > > So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-) > > (or perhaps it doesn't have .sqrt()?) Apparently it does not: http://stackoverflow.com/questions/13649703/square-root-of-bigdecimal-in-java But let's take something similar: BigDecimal(1).divide(BigDecimal(3)). This expression cannot be computed exactly in decimal format so it raises ArithmeticError. If you want it to round to some precision then you can provide a context with BigDecimal(1).divide(BigDecimal(3), mymathcontext). This rounds as specified in mymathcontext and the result of the expression is not affected by someone else changing a global arithmetic context. Oscar From solipsis at pitrou.net Tue Mar 11 16:14:34 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Mar 2014 16:14:34 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> Message-ID: <lfn98c$4v9$1@ger.gmane.org> >> It is arguably much simpler than unicode and character >> encodings. Besides, for most applications Decimal's current precision will >> be good enough: changing the current context (which, I believe, is >> thread-local, not process-global) is only necessary in special cases. > > The problem is that the context cannot be depended upon so the result > of trivial calculations or even a negative decimal literal will depend > on something that can be changed *anywhere* else in the code. The > expression -1.123456789d will evaluate differently depending on the > context and I think many people will be surprised by that. I expect > that this would lead to naive code that breaks when the context is > changed. Great care needs to be taken to get this right, or otherwise > it should just be considered generally unsafe to change the context. This is a good point, but I still don't think it outweighs the large cognitive baggage (and maintenance overhead for library authors as well as ourselves) of having two different types doing mostly similar things. It is also not clear that a decimal type would be really beneficial for non-expert users. Most people get by with floats fine. > But let's take something similar: BigDecimal(1).divide(BigDecimal(3)). > This expression cannot be computed exactly in decimal format so it > raises ArithmeticError. If you want it to round to some precision then > you can provide a context with BigDecimal(1).divide(BigDecimal(3), > mymathcontext). But then it becomes, ironically, more cumbersome to spell than Decimal(1) / Decimal(3) (I'm talking about the explicit context, of course, not the .divide() method which needn't exist in Python) Regards Antoine. From p.f.moore at gmail.com Tue Mar 11 16:26:34 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 11 Mar 2014 15:26:34 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfn98c$4v9$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> Message-ID: <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> On 11 March 2014 15:14, Antoine Pitrou <solipsis at pitrou.net> wrote: >> The problem is that the context cannot be depended upon so the result >> of trivial calculations or even a negative decimal literal will depend >> on something that can be changed *anywhere* else in the code. The >> expression -1.123456789d will evaluate differently depending on the >> context and I think many people will be surprised by that. I expect >> that this would lead to naive code that breaks when the context is >> changed. Great care needs to be taken to get this right, or otherwise >> it should just be considered generally unsafe to change the context. > > This is a good point, but I still don't think it outweighs the large > cognitive baggage (and maintenance overhead for library authors as well as > ourselves) of having two different types doing mostly similar things. It feels to me as if what we're discovering here is why dynamic scoping is a bad thing. If context was lexically scoped, all of this concern about "action at a distance" and user code impacting library behaviour would go away. I presume that ship has sailed, though, and there's no practical way of switching to lexical scoping of decimal contexts... Paul From solipsis at pitrou.net Tue Mar 11 16:45:28 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Mar 2014 16:45:28 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> Message-ID: <lfnb29$sbb$1@ger.gmane.org> Le 11/03/2014 16:26, Paul Moore a ?crit : > > It feels to me as if what we're discovering here is why dynamic > scoping is a bad thing. If context was lexically scoped, all of this > concern about "action at a distance" and user code impacting library > behaviour would go away. I don't think that's the problem. What you call "dynamic scoping" is a feature here, and "lexical scoping" wouldn't solve the issue: simply, the parser and compiler run before executing any Python code, so they can't see any changes in decimal precision triggered by a library call. Regards Antoine. From mertz at gnosis.cx Tue Mar 11 17:11:16 2014 From: mertz at gnosis.cx (David Mertz) Date: Tue, 11 Mar 2014 09:11:16 -0700 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfnb29$sbb$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> <lfnb29$sbb$1@ger.gmane.org> Message-ID: <CAEbHw4Zr5zYsPa+sCk2A4cG24TQtOAS-=7aX0w7v-d2x-VsGFg@mail.gmail.com> I think that the dynamic scoping of getcontext() *is* a problem, but lexical scoping wouldn't actually solve the issue at hand with decimal literals. Even if we imagined a lexically scoped future with local causality, this code would still produce unpredictable results and would not allow constant folding: def add_contst(d): decimal.lexical_getcontext().prec(random.randint(1,28)) return d + 1234.567890d I'm strongly in favor of Oscar's approach of making a decimal literal be fixed precision (e.g. decimal128; I prefer that to decimal64). More advanced users are welcome to cast that literal to decimal.Decimal with a particular context, but basic users will find this the least surprising approach. On Tue, Mar 11, 2014 at 8:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > Le 11/03/2014 16:26, Paul Moore a ?crit : > > >> It feels to me as if what we're discovering here is why dynamic >> scoping is a bad thing. If context was lexically scoped, all of this >> concern about "action at a distance" and user code impacting library >> behaviour would go away. >> > > I don't think that's the problem. What you call "dynamic scoping" is a > feature here, and "lexical scoping" wouldn't solve the issue: simply, the > parser and compiler run before executing any Python code, so they can't see > any changes in decimal precision triggered by a library call. > > Regards > > Antoine. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/c318ae20/attachment.html> From oscar.j.benjamin at gmail.com Tue Mar 11 18:03:01 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 17:03:01 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfnb29$sbb$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> <lfnb29$sbb$1@ger.gmane.org> Message-ID: <CAHVvXxRwCsKTZfWzVZW2=XEyWrY-yCj8z=KXz97-LVx_nr2Q=A@mail.gmail.com> On 11 March 2014 15:45, Antoine Pitrou <solipsis at pitrou.net> wrote: > Le 11/03/2014 16:26, Paul Moore a ?crit : > >> >> It feels to me as if what we're discovering here is why dynamic >> scoping is a bad thing. If context was lexically scoped, all of this >> concern about "action at a distance" and user code impacting library >> behaviour would go away. > > > I don't think that's the problem. What you call "dynamic scoping" is a > feature here, I think it would more appropriately be called "global scoping" but yes it is a feature when used carefully. > and "lexical scoping" wouldn't solve the issue: simply, the > parser and compiler run before executing any Python code, so they can't see > any changes in decimal precision triggered by a library call. The issue is bigger than constant folding. The problem is about being able to understand a simple looking expression or block of code. With the decimal module it becomes necessary to consider many different cases. What if the precision is less than X or higher than Y? What if this particular Decimal has a precision exceeding that of the current context? (The extra precision is not ignored; rounding effectively occurs *after* each calculation.) The issue applies to all programmers but consider the case where an inexperienced user posts code that doesn't work asking for help. On the one hand you don't know if they've modified the context and can't be sure that you understand what the code does. On the other hand you don't want to have to bombard them with technical details about whether the context was modified or explain how to check if they're unsure. Oscar From oscar.j.benjamin at gmail.com Tue Mar 11 18:09:37 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 17:09:37 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfn98c$4v9$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> Message-ID: <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com> On 11 March 2014 15:14, Antoine Pitrou <solipsis at pitrou.net> wrote: > >>> It is arguably much simpler than unicode and character >>> encodings. Besides, for most applications Decimal's current precision >>> will >>> be good enough: changing the current context (which, I believe, is >>> thread-local, not process-global) is only necessary in special cases. >> >> >> The problem is that the context cannot be depended upon so the result >> of trivial calculations or even a negative decimal literal will depend >> on something that can be changed *anywhere* else in the code. The >> expression -1.123456789d will evaluate differently depending on the >> context and I think many people will be surprised by that. I expect >> that this would lead to naive code that breaks when the context is >> changed. Great care needs to be taken to get this right, or otherwise >> it should just be considered generally unsafe to change the context. > > This is a good point, but I still don't think it outweighs the large > cognitive baggage (and maintenance overhead for library authors as well as > ourselves) of having two different types doing mostly similar things. > > It is also not clear that a decimal type would be really beneficial for > non-expert users. Most people get by with floats fine. These are good arguments. I guess we'll just have to see what all the pros and cons look like taken together. (There's always the possibility of doing nothing of course!) >> But let's take something similar: BigDecimal(1).divide(BigDecimal(3)). >> This expression cannot be computed exactly in decimal format so it >> raises ArithmeticError. If you want it to round to some precision then >> you can provide a context with BigDecimal(1).divide(BigDecimal(3), >> mymathcontext). > > But then it becomes, ironically, more cumbersome to spell than > Decimal(1) / Decimal(3) > (I'm talking about the explicit context, of course, not the .divide() method > which needn't exist in Python) That's why I propose something more along the lines of what the other languages have done. I'm not sure if any other languages have the same model as Python's decimal module except possibly REXX. I don't have any experience of REXX but from a quick read of some docs the "numeric digits" instruction seems to have the same global effect as the decimal module's contexts. Oscar From rosuav at gmail.com Tue Mar 11 18:19:08 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Mar 2014 04:19:08 +1100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CAHVvXxRJOa5P9MPa4bmw7W_OPDEKN0YOiX5NL98Da3CdqgSi-A@mail.gmail.com> Message-ID: <CAPTjJmrVWZ579gQz2VQVBS4fw5n9QsKzbRoCj=u4HtodHtLCSg@mail.gmail.com> On Wed, Mar 12, 2014 at 4:09 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > I'm not sure if any other languages have the same model as Python's > decimal module except possibly REXX. I don't have any experience of > REXX but from a quick read of some docs the "numeric digits" > instruction seems to have the same global effect as the decimal > module's contexts. Yes, it does. I'm more familiar with numeric digits than with Python's decimal contexts, but they definitely seem to be doing the same thing. (Does Python have an equivalent of 'numeric fuzz N', which basically defines numeric equality as "reduce precision by N digits, round both numbers, then see if they're equal"?) ChrisA From solipsis at pitrou.net Tue Mar 11 18:43:28 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Mar 2014 18:43:28 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> <lfnb29$sbb$1@ger.gmane.org> <CAHVvXxRwCsKTZfWzVZW2=XEyWrY-yCj8z=KXz97-LVx_nr2Q=A@mail.gmail.com> Message-ID: <20140311184328.52518261@fsol> On Tue, 11 Mar 2014 17:03:01 +0000 Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > > The issue is bigger than constant folding. The problem is about being > able to understand a simple looking expression or block of code. With > the decimal module it becomes necessary to consider many different > cases. What if the precision is less than X or higher than Y? What if > this particular Decimal has a precision exceeding that of the current > context? Many APIs have context-dependent meaning, especially in a duck-typed languages such as Python. I don't think decimals are a special-case here. OTOH, *if you know that d and e are decimals*, then what e.g. `d + e` does is pretty clear, regardless of the context. Knowing the precision isn't necessary for *understanding* what the code does; it may be necessary to track an unexpectedly "wrong" result, but that's all. And, besides, I think much of this argument is based on FUD. Probably in most real-world situations people don't set the decimal context to some arbitrarily low precision, so I don't know what the *practical* problem is. > The issue applies to all programmers but consider the case where an > inexperienced user posts code that doesn't work asking for help. Define "doesn't work". If the stated result simply isn't precise enough (or is too precise for the user's expectation), then it will be obvious what the cause is! No need for some heroic spelunking here, AFAICT. Regards Antoine. From stefan at bytereef.org Tue Mar 11 19:41:59 2014 From: stefan at bytereef.org (Stefan Krah) Date: Tue, 11 Mar 2014 19:41:59 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> Message-ID: <20140311184159.GA948@sleipnir.bytereef.org> Nick Coghlan <ncoghlan at gmail.com> wrote: > Stefan: use the existing decimal type, change default context to Decimal64, > round all decimal literals to current context I'm open to many options, including doing nothing or somehow deal with the UNARY_MINUS issue in another way. First of all, I don't think that having a decimal literal is in any way essential -- professional users probably deal with Decimals much like Marc-Andre described and I cannot see why we should add a literal primarily for inexperienced users. So, if we add a literal, professional users should be able to use it in a meaningful way, too. I understand the benefits of a context with fixed precision, Emax and Emin, so here are the drawbacks: 1) It must be possible to set rounding and traps, so the context is not really fixed. 2) Code that mixes Decimal64 and Decimal (virtually all professional code) will get hard to read and to understand, since we now have two contexts (problems) instead of one. Even tracking the flags will be challenging. 3) We must decide between Decimal64 and Decimal128. The latter is quite a bit slower, the former may not have sufficient precision. Now, to the argument that a context may be changed by a third party package: This is inherently true for all global (thread local) state and is really a problem of the package. Packages can do that with locales (which would also affect the "n" formatting for Decimal64), packages can do that right now with the existing Decimal type. In fact, for a highly robust package, people should not use the global context at all [1]. IIRC potential problems with coroutines were pointed out on python-dev recently. And that is another problem: If a package is using fixed Decimal64 literals, is it then supposed to use only the official global context? I think that would be a bad idea, unless the official context is completely frozen. That on the other hand would deprive users of controlling Overflow, clamping etc. Stefan Krah [1] If you read and implement this, do watch performance a little (method lookups can ruin a lot, using the operators is usually faster). From tinchester at gmail.com Tue Mar 11 19:46:07 2014 From: tinchester at gmail.com (=?UTF-8?B?VGluIFR2cnRrb3ZpxIc=?=) Date: Tue, 11 Mar 2014 19:46:07 +0100 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <mailman.40053.1394557800.18129.python-ideas@python.org> References: <mailman.40053.1394557800.18129.python-ideas@python.org> Message-ID: <531F59EF.8030808@gmail.com> Hello, it occured to me today that it's not currently possible to parse ISO 8601 dates using the datetime.datetime.strftime function (I was parsing datetimes generated by Postgres). The problem is in the semicolon in the time zone offset. '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time representation, the likes of which can be generated by the datetime module itself (the isoformat method). The %z strftime directive only recognizes offsets without the semicolon. The Python docs also direct users to inspect the platform strftime documentation; on my system the man page clearly states %z is "... The +hhmm or -hhmm numeric timezone", again, no semicolon support. Googling around, most common suggestions are to use a third party library (such as dateutil), so either this functionality really doesn't exist in a simple form in the datetime module, or is really undiscoverable. It seems to me ISO 8601 is significant enough (since there are even special methods for it in the datetime module) for it to be parsable, in a non-complex way, by the standard library. I guess it's interesting to point out a new flag was added to Java's SimpleDateFormat's date and time patterns in Java 7 - the X directive stands for "ISO 8601 time zone" (-08; -0800; -08:00, Z). I've worked with this so I happen to know it off the top of my head. Thanks in advance for comments. Tin From guido at python.org Tue Mar 11 19:57:24 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 11 Mar 2014 11:57:24 -0700 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <531F59EF.8030808@gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> Message-ID: <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> Can you show or point us to some examples of the types of dates that cannot be parsed? Do you happen to have the specification for those? "ISO 8601" alone doesn't tell me much, there are many different formats endorsed by the standard, and the standard is hard to read. I can't find it in Wikipedia either, nor in the Postgres docs. Is it something that a simple regular expression could extract into pieces that can be parsed with Python's strftime? On Tue, Mar 11, 2014 at 11:46 AM, Tin Tvrtkovi? <tinchester at gmail.com>wrote: > Hello, > > it occured to me today that it's not currently possible to parse ISO 8601 > dates using the datetime.datetime.strftime function (I was parsing > datetimes generated by Postgres). The problem is in the semicolon in the > time zone offset. > > '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time > representation, the likes of which can be generated by the datetime module > itself (the isoformat method). The %z strftime directive only recognizes > offsets without the semicolon. The Python docs also direct users to inspect > the platform strftime documentation; on my system the man page clearly > states %z is "... The +hhmm or -hhmm numeric timezone", again, no semicolon > support. > > Googling around, most common suggestions are to use a third party library > (such as dateutil), so either this functionality really doesn't exist in a > simple form in the datetime module, or is really undiscoverable. It seems > to me ISO 8601 is significant enough (since there are even special methods > for it in the datetime module) for it to be parsable, in a non-complex way, > by the standard library. > > I guess it's interesting to point out a new flag was added to Java's > SimpleDateFormat's date and time patterns in Java 7 - the X directive > stands for "ISO 8601 time zone" (-08; -0800; -08:00, Z). I've worked with > this so I happen to know it off the top of my head. > > Thanks in advance for comments. > > Tin > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/c42dee5d/attachment-0001.html> From alexander.belopolsky at gmail.com Tue Mar 11 20:08:06 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 11 Mar 2014 15:08:06 -0400 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> Message-ID: <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com> On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum <guido at python.org> wrote: > Can you show or point us to some examples of the types of dates that > cannot be parsed? Do you happen to have the specification for those? This is a known limitation. We don't have an strftime code for TZ in hh:mm format. GNU date uses ':z' for this. See http://bugs.python.org/issue5207 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/8896d1cc/attachment.html> From guido at python.org Tue Mar 11 20:10:51 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 11 Mar 2014 12:10:51 -0700 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com> Message-ID: <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com> Ah, colon, not semicolon. On Mar 11, 2014 12:08 PM, "Alexander Belopolsky" < alexander.belopolsky at gmail.com> wrote: > > On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum <guido at python.org>wrote: > >> Can you show or point us to some examples of the types of dates that >> cannot be parsed? Do you happen to have the specification for those? > > > This is a known limitation. We don't have an strftime code for TZ in hh:mm > format. GNU date uses ':z' for this. > > See http://bugs.python.org/issue5207 > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/5979d87d/attachment.html> From random832 at fastmail.us Tue Mar 11 20:14:22 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 11 Mar 2014 15:14:22 -0400 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <531F59EF.8030808@gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> Message-ID: <1394565262.23609.93273945.5C970096@webmail.messagingengine.com> On Tue, Mar 11, 2014, at 14:46, Tin Tvrtkovi? wrote: > '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time > representation, the likes of which can be generated by the datetime > module itself (the isoformat method). The %z strftime directive only > recognizes offsets without the semicolon. The Python docs also direct > users to inspect the platform strftime documentation; on my system the > man page clearly states %z is "... The +hhmm or -hhmm numeric timezone", > again, no semicolon support. Maybe Python should switch to platform-independent time parsing/formatting implementations. These can have a %z that accepts this format. I will incidentally note that as of SUSv7 systems are not required to support %z for strptime at all (nor does it therefore impose any requirement to reject formats including a colon, or 'Z'.) From random832 at fastmail.us Tue Mar 11 20:17:37 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 11 Mar 2014 15:17:37 -0400 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <lfn280$851$1@ger.gmane.org> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> Message-ID: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote: > 11.03.14 14:44, Oscar Benjamin ???????(??): > > Am I doing this wrong? Is this just a limitation of the old-style % > > formatting or something that could possibly be improved? > > Yes, this is a limitation of the old-style % formatting. Another example: Is % formatting not going to be updated to support new features or fix bugs like this? Is it going to be deprecated? Will a tool be provided to convert usages (at least with static format strings) to str.format()? From random832 at fastmail.us Tue Mar 11 20:22:43 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 11 Mar 2014 15:22:43 -0400 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <lfnb29$sbb$1@ger.gmane.org> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <lfn34f$k4q$1@ger.gmane.org> <CAHVvXxTKDwf-sTuOJK3wzgm1nR_wWAG4DzWdQWVLZ_K7hpnHww@mail.gmail.com> <lfn98c$4v9$1@ger.gmane.org> <CACac1F-sRWxeacy5EWOujw-ghURFMo2d5ZRXn-5qXjba1PX9UA@mail.gmail.com> <lfnb29$sbb$1@ger.gmane.org> Message-ID: <1394565763.26478.93278337.000AD481@webmail.messagingengine.com> On Tue, Mar 11, 2014, at 11:45, Antoine Pitrou wrote: > I don't think that's the problem. What you call "dynamic scoping" is a > feature here, and "lexical scoping" wouldn't solve the issue: simply, > the parser and compiler run before executing any Python code, so they > can't see any changes in decimal precision triggered by a library call. They could if all such changes were made by keyworded statements or expressions (which the compiler can understand just fine) rather than library calls. Which they would necessarily have to be if contexts were lexically scoped, since they would have to affect the execution of operators within their scope (probably by passing the context as a third argument to the operator implementations) without affecting functions called by them. From pyideas at rebertia.com Tue Mar 11 20:28:48 2014 From: pyideas at rebertia.com (Chris Rebert) Date: Tue, 11 Mar 2014 12:28:48 -0700 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <531F59EF.8030808@gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> Message-ID: <CAMZYqRSJpZ=n6RLLk0UREQE_sFAk3MhbnzuaBA4x_AW-2roXdw@mail.gmail.com> On Tue, Mar 11, 2014 at 11:46 AM, Tin Tvrtkovi? <tinchester at gmail.com> wrote: > Hello, > > it occured to me today that it's not currently possible to parse ISO 8601 > dates using the datetime.datetime.strftime function (I was parsing datetimes > generated by Postgres). The problem is in the semicolon in the time zone > offset. > > '1997-07-16T19:20:30.45+01:00' is a valid ISO 8601 date and time > representation, the likes of which can be generated by the datetime module > itself (the isoformat method). The %z strftime directive only recognizes > offsets without the semicolon. The Python docs also direct users to inspect > the platform strftime documentation; on my system the man page clearly > states %z is "... The +hhmm or -hhmm numeric timezone", again, no semicolon > support. > > Googling around, most common suggestions are to use a third party library > (such as dateutil), so either this functionality really doesn't exist in a > simple form in the datetime module, or is really undiscoverable. It seems to > me ISO 8601 is significant enough (since there are even special methods for > it in the datetime module) for it to be parsable, in a non-complex way, by > the standard library. There's already an open bug about this (well, technically RFC 3339, but close enough): http://bugs.python.org/issue15873 Cheers, Chris From random832 at fastmail.us Tue Mar 11 20:38:08 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 11 Mar 2014 15:38:08 -0400 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140311184159.GA948@sleipnir.bytereef.org> References: <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311184159.GA948@sleipnir.bytereef.org> Message-ID: <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote: > 1) It must be possible to set rounding and traps, so the context > is not really fixed. It _must_ be possible? If there's a way to set rounding for floats, I can't find it. As for traps, I can't find that either, but a constant operation that may (will?) trap disables constant-folding. >>> dis.dis(lambda x: 1.0/0.0) 1 0 LOAD_CONST 1 (1.0) 3 LOAD_CONST 2 (0.0) 6 BINARY_TRUE_DIVIDE 7 RETURN_VALUE Maybe what needs to be done is to rigorously define what is a constant expression (C does this) and explicitly state what context (including rounding and traps) they take place in, independent of the runtime context - which, then, an implementation that does _not_ do constant folding must use to perform operations within a constant expression at runtime. Incidentally, did you know there's apparently no limit to how large an integer ** will constant-fold? I say this because I assume if there were it would be far below 2**9999999. On the other hand, 2.0**1024 doesn't constant-fold. From tinchester at gmail.com Tue Mar 11 20:39:28 2014 From: tinchester at gmail.com (=?UTF-8?B?VGluIFR2cnRrb3ZpxIc=?=) Date: Tue, 11 Mar 2014 20:39:28 +0100 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com> <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com> Message-ID: <531F6670.9010304@gmail.com> Ugh, I need to get into the habit of checking the issue tracker first. Sorry for not doing that this time. The resolution of that issue is what I'm after, yeah. Also, yes, I meant colon instead of semicolon. Thanks! On 11.03.2014 20:10, Guido van Rossum wrote: > > Ah, colon, not semicolon. > > On Mar 11, 2014 12:08 PM, "Alexander Belopolsky" > <alexander.belopolsky at gmail.com > <mailto:alexander.belopolsky at gmail.com>> wrote: > > > On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum > <guido at python.org <mailto:guido at python.org>> wrote: > > Can you show or point us to some examples of the types of > dates that cannot be parsed? Do you happen to have the > specification for those? > > > This is a known limitation. We don't have an strftime code for TZ > in hh:mm format. GNU date uses ':z' for this. > > See http://bugs.python.org/issue5207 > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/59a62b1e/attachment.html> From stefan at bytereef.org Tue Mar 11 20:46:25 2014 From: stefan at bytereef.org (Stefan Krah) Date: Tue, 11 Mar 2014 20:46:25 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> References: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311184159.GA948@sleipnir.bytereef.org> <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> Message-ID: <20140311194625.GA2110@sleipnir.bytereef.org> random832 at fastmail.us <random832 at fastmail.us> wrote: > On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote: > > 1) It must be possible to set rounding and traps, so the context > > is not really fixed. > > It _must_ be possible? If there's a way to set rounding for floats, I > can't find it. As for traps, I can't find that either, but a constant > operation that may (will?) trap disables constant-folding. If we use IEEE, the default context would not raise anything -- infinities and NaNs would silently propagate. This is valuable, but it should be possible switch it off. Settting rounding is obviously valuable, too. Stefan Krah From breamoreboy at yahoo.co.uk Tue Mar 11 20:51:17 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 11 Mar 2014 19:51:17 +0000 Subject: [Python-ideas] Parsing ISO 8601 with the standard library In-Reply-To: <531F6670.9010304@gmail.com> References: <mailman.40053.1394557800.18129.python-ideas@python.org> <531F59EF.8030808@gmail.com> <CAP7+vJKwcarMxB17V=7KGw0NTv+Y11kLEKtmEOX8PAgjZjMr2w@mail.gmail.com> <CAP7h-xY4qrDS8MgX47aNaPxA7Q8uNnAP2dmXAmHmZJnRfm_NvQ@mail.gmail.com> <CAP7+vJLy0m2mhySfr+wnrDXkvKcXbf-sN59g2SB7mNB0X31Wyw@mail.gmail.com> <531F6670.9010304@gmail.com> Message-ID: <lfnpfl$lvb$1@ger.gmane.org> On 11/03/2014 19:39, Tin Tvrtkovi? wrote: > Ugh, I need to get into the habit of checking the issue tracker first. > Sorry for not doing that this time. > > The resolution of that issue is what I'm after, yeah. Also, yes, I meant > colon instead of semicolon. > > Thanks! > > On 11.03.2014 20:10, Guido van Rossum wrote: >> >> Ah, colon, not semicolon. >> >> On Mar 11, 2014 12:08 PM, "Alexander Belopolsky" >> <alexander.belopolsky at gmail.com >> <mailto:alexander.belopolsky at gmail.com>> >> wrote: >> >> >> On Tue, Mar 11, 2014 at 2:57 PM, Guido van Rossum >> <guido at python.org >> <mailto:guido at python.org>> wrote: >> >> Can you show or point us to some examples of the types of >> dates that cannot be parsed? Do you happen to have the >> specification for those? >> >> >> This is a known limitation. We don't have an strftime code for TZ >> in hh:mm format. GNU date uses ':z' for this. >> >> See http://bugs.python.org/issue5207 >> > In case you haven't seen it there's also http://bugs.python.org/issue15873 - whether both need to remain open I've no idea. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From abarnert at yahoo.com Tue Mar 11 21:10:01 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Mar 2014 13:10:01 -0700 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> Message-ID: <27743BD2-22A8-4194-A177-0AE63D338384@yahoo.com> On Mar 11, 2014, at 12:17, random832 at fastmail.us wrote: > On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote: >> 11.03.14 14:44, Oscar Benjamin ???????(??): >>> Am I doing this wrong? Is this just a limitation of the old-style % >>> formatting or something that could possibly be improved? >> >> Yes, this is a limitation of the old-style % formatting. Another example: > > Is % formatting not going to be updated to support new features or fix > bugs like this? That's an inherent limitation of % formatting: it uses printf-style type codes, whose meaning is baked into the % operator, rather than calling __format__ on its arguments to handle it polymorphically. %f means treat the argument as a float. That limited flexibility is part of what allows it to be simpler, faster, able to share some format strings with C, etc. It's doing what it was designed to do; the fact that it wasn't designed the same as str.format isn't a bug. That's not to say that it couldn't be extended with new type codes for decimals. If C1x adds printf type codes (or if gcc or some other compiler creates its own extension), you could make a pretty good argument for % formatting to borrow them. > Is it going to be deprecated? It wasn't deprecated in 3.0, and after the issue was raised again, I'm pretty sure the decision was that deprecation is off the table until at least Python 4000. And if you go back and read the arguments, the reason it wasn't deprecated is that for some use cases it's better, in part because it's not extensible, and has the corresponding advantages mentioned above. > Will a tool be provided to > convert usages (at least with static format strings) to str.format()? I'm pretty sure people have already written such tools, back when str.format was first created. I don't think too many people use them. If %-formatting is working for your existing code, or appropriate for some new code, there's no reason to change it. If {}-formatting is appropriate for some new code, it's easier to write it directly than to try to write a different format and then convert it. From oscar.j.benjamin at gmail.com Tue Mar 11 21:26:28 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 20:26:28 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140311194625.GA2110@sleipnir.bytereef.org> References: <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311184159.GA948@sleipnir.bytereef.org> <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> <20140311194625.GA2110@sleipnir.bytereef.org> Message-ID: <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com> On 11 March 2014 19:46, Stefan Krah <stefan at bytereef.org> wrote: > random832 at fastmail.us <random832 at fastmail.us> wrote: >> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote: >> > 1) It must be possible to set rounding and traps, so the context >> > is not really fixed. >> >> It _must_ be possible? If there's a way to set rounding for floats, I >> can't find it. As for traps, I can't find that either, but a constant >> operation that may (will?) trap disables constant-folding. > > If we use IEEE, the default context would not raise anything -- infinities > and NaNs would silently propagate. This is valuable, but it should be > possible switch it off. Settting rounding is obviously valuable, too. I was intending that it would be possible to use the traps and flags on this context. Perhaps Stefan knows something I don't but I thought that the constant folder could just use a hyper-sensitive context and check for exceptions (or otherwise check the flags). I assumed this is how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not folded so that they can raise exceptions at the appropriate time. Rounding is a trickier one. It would be possible to do the same thing by having the constant folder trap Inexact but it's a little more limiting since it applies to expressions that are more likely to have useful values. Oscar From ethan at stoneleaf.us Tue Mar 11 21:25:00 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 11 Mar 2014 13:25:00 -0700 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> Message-ID: <531F711C.1020106@stoneleaf.us> On 03/11/2014 12:17 PM, random832 at fastmail.us wrote: > On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote: >> 11.03.14 14:44, Oscar Benjamin ???????(??): >>> Am I doing this wrong? Is this just a limitation of the old-style % >>> formatting or something that could possibly be improved? >> >> Yes, this is a limitation of the old-style % formatting. Another example: > > Is % formatting not going to be updated to support new features No. > or fix bugs like this? It's not a bug. Decimal is not float. > Is it going to be deprecated? No. %-formatting has its place. > Will a tool be provided to convert usages (at least with static format strings) > to str.format()? No. Feel free to write one and share it with the community. :) Basically, %-formatting is primitive, .format-formatting is advanced. If you want advanced behavior, use the advanced method. -- ~Ethan~ From stefan at bytereef.org Tue Mar 11 21:53:31 2014 From: stefan at bytereef.org (Stefan Krah) Date: Tue, 11 Mar 2014 21:53:31 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com> References: <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311184159.GA948@sleipnir.bytereef.org> <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> <20140311194625.GA2110@sleipnir.bytereef.org> <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com> Message-ID: <20140311205331.GA2508@sleipnir.bytereef.org> Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > >> On Tue, Mar 11, 2014, at 14:41, Stefan Krah wrote: > >> > 1) It must be possible to set rounding and traps, so the context > >> > is not really fixed. > >> > >> It _must_ be possible? If there's a way to set rounding for floats, I > >> can't find it. As for traps, I can't find that either, but a constant > >> operation that may (will?) trap disables constant-folding. > > > > If we use IEEE, the default context would not raise anything -- infinities > > and NaNs would silently propagate. This is valuable, but it should be > > possible switch it off. Settting rounding is obviously valuable, too. > > I was intending that it would be possible to use the traps and flags > on this context. Perhaps Stefan knows something I don't but I thought > that the constant folder could just use a hyper-sensitive context and > check for exceptions (or otherwise check the flags). I assumed this is > how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not > folded so that they can raise exceptions at the appropriate time. I meant that an external package that modifies e.g. rounding in the global context would also affect results, so the problem (which was used as an argument in favor of the fixed width decimal) is not limited to exponents and precision. Stefan Krah From oscar.j.benjamin at gmail.com Tue Mar 11 21:59:17 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 20:59:17 +0000 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140311205331.GA2508@sleipnir.bytereef.org> References: <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311184159.GA948@sleipnir.bytereef.org> <1394566688.30715.93281121.23981AE1@webmail.messagingengine.com> <20140311194625.GA2110@sleipnir.bytereef.org> <CAHVvXxQQOh1w+t7qF2+PTsKuFO29tmnKXk0wsQUaHJidyQ2HTQ@mail.gmail.com> <20140311205331.GA2508@sleipnir.bytereef.org> Message-ID: <CAHVvXxT6NdpQdUdhyOxBZS+aFCbUpK6mW0-FevBu4XyODW7Eow@mail.gmail.com> On 11 March 2014 20:53, Stefan Krah <stefan at bytereef.org> wrote: > Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: >> >> I was intending that it would be possible to use the traps and flags >> on this context. Perhaps Stefan knows something I don't but I thought >> that the constant folder could just use a hyper-sensitive context and >> check for exceptions (or otherwise check the flags). I assumed this is >> how it handled cases like '1 / 0' and '1.0 * 10 ** 1000' which are not >> folded so that they can raise exceptions at the appropriate time. > > I meant that an external package that modifies e.g. rounding in the global > context would also affect results, so the problem (which was used as an > argument in favor of the fixed width decimal) is not limited to exponents > and precision. Could it not be handled by having the constant folder trap Inexact? I though that if all calculations can be exact within the fixed precision limit and exponent range then no rounding occurs so the result is independent of the rounding mode. Or have I missed something? Oscar From ron3200 at gmail.com Tue Mar 11 22:44:58 2014 From: ron3200 at gmail.com (Ron Adam) Date: Tue, 11 Mar 2014 16:44:58 -0500 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <531F0286.8090306@egenix.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzY d8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> <531F0286.8090306@egenix.com> Message-ID: <lfo04c$960$1@ger.gmane.org> On 03/11/2014 07:33 AM, M.-A. Lemburg wrote: > On 11.03.2014 12:46, Nick Coghlan wrote: >> >...[Oscar's proposal being that picking just one decimal floating >> > point precision as*the* floating point precision]... >> > >> >That approach buys us a couple of key benefits: >> > >> >- they would be true constants, so constant folding, etc, would work >> >normally, and you could tell from reading the code exactly what it >> >will do, without needing to worry about the impact of thread local >> >state >> >- you could generally use the builtin decimal literals without >> >learning anything about decimal contexts, because decimal literals >> >wouldn't be context dependent (although their behaviour would be >> >formally defined in terms of an IEEE decimal context, for most users >> >it would just be "this is how Python decimal literals behave") >> > >> >The guidance to new users would then be*don't* use the decimal >> >module, use decimal literals instead. If you need more configurable >> >behaviour,*then* reach for the decimal module. However, the explicit >> >methods on decimal context objects should probably still be updated to >> >accept both fixed and variable precision decimals under this model. > I think you are leaving out one of the most important use cases > for decimal values: data input and output. > > The literals would only appear in programs. However, in most use > cases, you want to read decimal data from some source, process it > and then write it back again. This would most likely also require > using a few decimal literals, but the main source of decimals > would still be constructors that you choose when writing the > tools. > > Coming back to floats and the decimal constructor: > > In investment banking, for example, people usually work with floats > all the time. You only convert back to decimals at the very end of some > calculation. The reason here being either that the calculations > involve complex operations which are not available for decimals, > trying to keep error intervals small, or a combination of both. > In general, you try to use as much precision as you can afford > (in terms of memory and speed), to keep those error intervals > small. > > In accounting, you often use decimals for storing data with > an (usually contractually or law based) agreed upon precision > and rounding logic. However, there situations where you have > to go to floats as well in order to run calculations, e.g. > for interest, taxes, etc. > > In both situations, you want to have the decimal constructor > take the float values with full precision and only then apply > the necessary rounding to turn the value into a form which > complies with the agreed upon rules. It's also not uncommon to > add correctional bookings to address rounding issues explicitly > (e.g. when calculating VAT of a large number of individual > items). > > In short: you try to prevent rounding from happening as > much as possible and when using it, you use it in a > controlled way. > > Based on this, the choice to have the decimal constructor > use full precision when reading floats is a good one, even > though it may not feel right for the novice, the casual > decimal user or as human concept:-) > > For decimal literals, I'd argue that if you enter a > value 1.2500d, you are expecting a decimal with 4 decimal > places precision, not 64 or 128 bits:-) > > The information about the intended precision is implicit in the > literal. You see this done in exchange rates, stock prices, > prices at your grocery store, etc. > > The decimals can then be transformed into ones with higher > precision during calculations and then possibly back to lower > precision, but this is an explicit decision by the system doing > the calculation. > > Anyway, just throwing in some additional entropy into this > discussion. Probably not all that helpful, since you're already > converging on two possible solutions:-) I like django's measurement object approach. https://docs.djangoproject.com/en/dev/ref/contrib/gis/measure/ It just needs a way to track accuracy (significant digits). Which can be independent of the underlying numeric type. (I think django has that too.) Possibly, it could promote it's underlying type (int, float, decimal) if it needs to, or raise an exception if the underlying type isn't accurate enough to provide the requested precision. The same concept could be used for currency calculations, or time for that matter. This would do much more than adding a new underlying type to python. It encapsulates expert knowledge into objects that can be used by user who's expertise may be in other areas. The actual underlying numeric type could depend on the amount of accuracy needed rather than some preconceived notion of greater accuracy that probably isn't needed in most cases. Ie.. Decimal128 for calculations that only require 6 significant digits. Cheers, Ron From random832 at fastmail.us Tue Mar 11 22:51:06 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Tue, 11 Mar 2014 17:51:06 -0400 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <531F711C.1020106@stoneleaf.us> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> Message-ID: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote: > > Is % formatting not going to be updated to support new features > > No. > > > or fix bugs like this? > > It's not a bug. Decimal is not float. The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does. > > Is it going to be deprecated? > > No. %-formatting has its place. What place is that? There should be one-- and preferably only one --obvious way to do it. We've got two syntaxes, and two functions, that do _almost_ the same thing, but with two completely different microlanguages. Why? From ncoghlan at gmail.com Tue Mar 11 23:30:40 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 12 Mar 2014 08:30:40 +1000 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> Message-ID: <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com> On 12 Mar 2014 07:51, <random832 at fastmail.us> wrote: > > On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote: > > > Is % formatting not going to be updated to support new features > > > > No. > > > > > or fix bugs like this? > > > > It's not a bug. Decimal is not float. > > The bug to which I was referring involves not calling __format__ on > whatever object is passed in as str.format does - Decimal not being > float is no impediment to str.format and Decimal.__format__ accepting > the same format strings that float.__format__ does. > > > > Is it going to be deprecated? > > > > No. %-formatting has its place. > > What place is that? > > There should be one-- and preferably only one --obvious way to do > it. > > We've got two syntaxes, and two functions, that do _almost_ the same > thing, but with two completely different microlanguages. Why? Start by reading PEP 3101, which added str.format to address the limitations of mod-formatting. For new users, the recommendation is unconditional: use str.format. mod-formatting is now a power tool for optimisation, consistency with C implementations, templating format strings without excessive escaping and (the main reason it still exists), backwards compatibility (both for existing code and existing programmers). The original plan was to eventually deprecate mod-formatting entirely, but not only did we discover that was harder than expected, we also found it was actually incredibly useful in some cases to have a formatting system available that relied on coercion rather than polymorphism, since it could be faster, was more predictable and also more easily mapped to other languages like C (especially since Python's mod-formatting is based on C's printf syntax). That's why the binary interpolation proposal for 3.5 now *only* proposes restoring mod-formatting for the binary domain, leaving str.format as a text domain only operation. Cheers, Nick. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/b19fc92a/attachment.html> From ethan at stoneleaf.us Tue Mar 11 22:58:42 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 11 Mar 2014 14:58:42 -0700 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> Message-ID: <531F8712.3010805@stoneleaf.us> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote: > On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote: >> at some point prior, random832 at fastmail.us wrote: >>> >>> Is % formatting not going to be updated to support new features >> >> No. >> >>> or fix bugs like this? >> >> It's not a bug. Decimal is not float. > > The bug to which I was referring involves not calling __format__ on > whatever object is passed in as str.format does - Decimal not being > float is no impediment to str.format and Decimal.__format__ accepting > the same format strings that float.__format__ does. That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not. >>> Is it going to be deprecated? >> >> No. %-formatting has its place. > > What place is that? > > There should be one-- and preferably only one --obvious way to do > it. You'll notice the word 'preferably' in there. The preferred method is format. %-formatting has a good place in lazy evaluation (in logging, for example), as well as other areas. > We've got two syntaxes, and two functions, that do _almost_ the same > thing, but with two completely different microlanguages. Why? Because % was there first, but had some failings that are addressed by format. -- ~Ethan~ From rosuav at gmail.com Tue Mar 11 23:47:33 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Mar 2014 09:47:33 +1100 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <CADiSq7eLvXKYT1pnL0+rmH=9Ovdt2zN5dLkfM5Vha1VjMwgH0A@mail.gmail.com> Message-ID: <CAPTjJmrbz1o7+XyqSHTEOY_jtm6bz0jFYryjYAsELtUDR9fgGg@mail.gmail.com> On Wed, Mar 12, 2014 at 9:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: > mod-formatting is now a power tool for optimisation, consistency with C > implementations, templating format strings without excessive escaping and > (the main reason it still exists), backwards compatibility (both for > existing code and existing programmers). And consistency with other C-derived languages, too. The only weirdness of Python's that isn't found in others is the "hey look isn't it cool" use of the % operator, which results in edge cases because the operator always takes exactly one argument on the RHS - which can be solved by turning it into a function: >>> def sprintf(fmt, *args): return fmt%args >>> sprintf("foo %s bar",(1,2,3)) 'foo (1, 2, 3) bar' >>> sprintf("foo %d bar %d quux",1,2) 'foo 1 bar 2 quux' ChrisA From oscar.j.benjamin at gmail.com Tue Mar 11 23:49:23 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 11 Mar 2014 22:49:23 +0000 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <531F8712.3010805@stoneleaf.us> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <531F8712.3010805@stoneleaf.us> Message-ID: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote: > On 03/11/2014 02:51 PM, random832 at fastmail.us wrote: >> >> The bug to which I was referring involves not calling __format__ on >> whatever object is passed in as str.format does - Decimal not being >> float is no impediment to str.format and Decimal.__format__ accepting >> the same format strings that float.__format__ does. > > That is not a bug. str.__mod__ is not documented as calling an object's > __format__ method, and indeed it does not. Instead it calls float on the object: >>> class A: ... def __float__(self): ... print('__float__ called') ... >>> '%f' % A() __float__ called Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: nb_float should return float object There's no mention of that anywhere in the docs though: http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting There's also no mention of this in the decimal docs. Oscar From ethan at stoneleaf.us Tue Mar 11 23:46:45 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 11 Mar 2014 15:46:45 -0700 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <531F8712.3010805@stoneleaf.us> <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> Message-ID: <531F9255.1090407@stoneleaf.us> On 03/11/2014 03:49 PM, Oscar Benjamin wrote: > On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote: >> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote: >>> >>> The bug to which I was referring involves not calling __format__ on >>> whatever object is passed in as str.format does - Decimal not being >>> float is no impediment to str.format and Decimal.__format__ accepting >>> the same format strings that float.__format__ does. >> >> That is not a bug. str.__mod__ is not documented as calling an object's >> __format__ method, and indeed it does not. > > Instead it calls float on the object: > >>>> class A: > ... def __float__(self): > ... print('__float__ called') > ... >>>> '%f' % A() > __float__ called Well, you did ask for a float, and how else are you going to get one? ;) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: nb_float should return float object > > There's no mention of that anywhere in the docs though: > http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting We should probably fix that. -- ~Ethan~ From abarnert at yahoo.com Wed Mar 12 01:45:38 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 11 Mar 2014 17:45:38 -0700 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <531F8712.3010805@stoneleaf.us> <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> Message-ID: <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com> On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote: >> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote: >>> >>> The bug to which I was referring involves not calling __format__ on >>> whatever object is passed in as str.format does - Decimal not being >>> float is no impediment to str.format and Decimal.__format__ accepting >>> the same format strings that float.__format__ does. >> >> That is not a bug. str.__mod__ is not documented as calling an object's >> __format__ method, and indeed it does not. > > Instead it calls float on the object: > >>>> class A: > ... def __float__(self): > ... print('__float__ called') > ... >>>> '%f' % A() > __float__ called > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: nb_float should return float object > > There's no mention of that anywhere in the docs though: > http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't. It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()". Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)? > There's also no mention of this in the decimal docs. Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals. From harrismh777 at gmail.com Wed Mar 12 06:33:48 2014 From: harrismh777 at gmail.com (Mark H. Harris) Date: Tue, 11 Mar 2014 22:33:48 -0700 (PDT) Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <20140311082449.4f0c0fdb@fsol> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <20140309003925.GC28804@ando> <85a717e0-28f3-4590-b42c-6ef4cda7a28b@googlegroups.com> <CAPTjJmr_sWuM7=vyS5gN04KhMDnncG6K7so24vA2z=T1dzEgyg@mail.gmail.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> Message-ID: <35c7108e-2757-4420-80fd-a7f6e6d2a495@googlegroups.com> On Tuesday, March 11, 2014 2:24:49 AM UTC-5, Antoine Pitrou wrote: > > Yuck. Is this some kind of joke? We've gone through the trouble of > unifying long and int in Python 3 and now people are proposing two > different Decimal types, one with fixed precision and one with > arbitrary precision? I also agree --in the long term someday-- that unification of number is key. Sometime in the distant future where there is no panic, and life is good, and things are safe. At some point the correct approach IMHO is to calculate with decimal floating point by default. But, this is not that day. Finding a way (as everyone is talking) to merge the use of binary floating point, and decimal floating point calcs, that is flexible &efficient, and caters to the most users (both experienced and novice alike) should be the goal... even if there are caveats and engineering compromises. It just makes sense. Thanks for all you are doing guys. kind regards, m harris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140311/071c57cb/attachment.html> From oscar.j.benjamin at gmail.com Wed Mar 12 11:13:23 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 12 Mar 2014 10:13:23 +0000 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <531F8712.3010805@stoneleaf.us> <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com> Message-ID: <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com> On 12 March 2014 00:45, Andrew Barnert <abarnert at yahoo.com> wrote: > > On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > >> On 11 March 2014 21:58, Ethan Furman <ethan at stoneleaf.us> wrote: >>> On 03/11/2014 02:51 PM, random832 at fastmail.us wrote: >>>> >>>> The bug to which I was referring involves not calling __format__ on >>>> whatever object is passed in as str.format does - Decimal not being >>>> float is no impediment to str.format and Decimal.__format__ accepting >>>> the same format strings that float.__format__ does. >>> >>> That is not a bug. str.__mod__ is not documented as calling an object's >>> __format__ method, and indeed it does not. >> >> Instead it calls float on the object: >> >>>>> class A: >> ... def __float__(self): >> ... print('__float__ called') >> ... >>>>> '%f' % A() >> __float__ called >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> TypeError: nb_float should return float object >> >> There's no mention of that anywhere in the docs though: >> http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatting > > I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't. > > It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()". > > Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)? Yes that's what I meant. The exact words used there to describe the 'f' format code are 'Floating point decimal format'. There's no mention that it will first need to coerce to a binary float before converting that to decimal. (Perhaps those words predate the decimal module). >> There's also no mention of this in the decimal docs. > > Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals. Because it's a relevant issue to consider when working with Decimals. In simple cases it will work but in other cases it won't. Oscar From eric at trueblade.com Wed Mar 12 12:36:50 2014 From: eric at trueblade.com (Eric V. Smith) Date: Wed, 12 Mar 2014 07:36:50 -0400 Subject: [Python-ideas] %-formatting with Decimals In-Reply-To: <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com> References: <CAHVvXxRov=+zf_HHimPpFe7n+HX5QpxQVBa-+zX0Bvr7-vrmTA@mail.gmail.com> <lfn280$851$1@ger.gmane.org> <1394565457.24653.93276933.3EEF5E6E@webmail.messagingengine.com> <531F711C.1020106@stoneleaf.us> <1394574666.6084.93336933.004BE02D@webmail.messagingengine.com> <531F8712.3010805@stoneleaf.us> <CAHVvXxTW6dz5CD_GNPsxmGytSuCoO-b9NaRY6LaD=dw5QGfzyQ@mail.gmail.com> <1FC44C3A-8FC7-4AA9-9E6E-8EC4EBAD88E0@yahoo.com> <CAHVvXxQ5DowK0VCZPt3t9ak9n5YR8uP3FUmhXo-hrnYPwFR1AA@mail.gmail.com> Message-ID: <532046D2.6050101@trueblade.com> On 03/12/2014 06:13 AM, Oscar Benjamin wrote: > On 12 March 2014 00:45, Andrew Barnert <abarnert at yahoo.com> wrote: >> >> On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: >>> There's also no mention of this in the decimal docs. >> >> Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals. > > Because it's a relevant issue to consider when working with Decimals. > In simple cases it will work but in other cases it won't. In the hopefully unlikely event we do get decimal literals, I think it would be reasonable to extend %f-formatting (and its friends e and g) to cover decimals as well as floats. Eric. From mal at egenix.com Wed Mar 12 14:11:15 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 12 Mar 2014 14:11:15 +0100 Subject: [Python-ideas] Python Numbers as Human Concept Decimal System In-Reply-To: <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com> References: <95c4efe0-b086-4bf2-b6c6-7d9bcc58f2b3@googlegroups.com> <CAP7+vJ+jq14x3G8jcW6fSxGOdrqZe+K6GVPWRCGxsjKr-qEyjw@mail.gmail.com> <CAHVvXxT8J7+y6T-gonxB6crUzSW9WGODLe_akyxqNKbtH0s_cQ@mail.gmail.com> <CAP7+vJJ_P2B+CWF_2tu=oLY3c77VJAd1cMOuE6MoDgNMnS1f_A@mail.gmail.com> <CAHVvXxQAMj3Hhmigwnc1mzzOeotnPgAKpn7trbLfVkEiVi-HQw@mail.gmail.com> <20140310105942.GB12595@ando> <CADiSq7dFS6KRGbYcqMw1VaC3SnprQEq0nEquPWc5cihX4zjzDw@mail.gmail.com> <20140310135342.GA10968@sleipnir.bytereef.org> <CAAu3qLXwF0D6ek73LNJtVD7hicsEu-i-Kns58PbREORbNq5QVg@mail.gmail.com> <CAP7+vJK1Vn_bv+W80RTZPGUD6fSxrFri7_VyckKAhB8sUn2_mQ@mail.gmail.com> <CADiSq7feEwvtHQ08OpsYjdrmt9hUNwn5RDx7sHB8cozzYd8WAQ@mail.gmail.com> <20140311082449.4f0c0fdb@fsol> <CAHVvXxQH-vEOqUq4TgTQW6dD2vCB98dzC8MDYh3-ZDe7QArXSQ@mail.gmail.com> <CADiSq7cEg3B+M+VqSAoRjwWpT1DS+W=N4ntxhUCyMtrhUGfQ0A@mail.gmail.com> <531F0286.8090306@egenix.com> <CAHVvXxRMEeiFeg1jTE_oM-22oORKv4vXzzdBOXFX-uQzjKJ=hg@mail.gmail.com> Message-ID: <53205CF3.1000100@egenix.com> On 11.03.2014 14:04, Oscar Benjamin wrote: > On 11 March 2014 12:33, M.-A. Lemburg <mal at egenix.com> wrote: >> >> [...examples from finance...] >> >> Anyway, just throwing in some additional entropy into this >> discussion. Probably not all that helpful, since you're already >> converging on two possible solutions :-) > > Thanks! Thanks for the detailed response. It's good to know that things will work out fine :-) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 12 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 17 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 28 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From wolfgang.maier at biologie.uni-freiburg.de Wed Mar 12 16:33:49 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 12 Mar 2014 15:33:49 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?numerical_type_combining_integer_and_flo?= =?utf-8?q?at/decimal=09properties_=5BWas=3A_Re=3A_Python_Numbers_a?= =?utf-8?q?s_Human_Concept_Decimal_System=5D?= References: <loom.20140310T181321-988@post.gmane.org> Message-ID: <loom.20140312T162210-499@post.gmane.org> Wolfgang Maier <wolfgang.maier at ...> writes: > > Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah: > > > [My apologies for being terse, I don't have much time to follow this > discussion right now.] > > > Nick Coghlan <ncoghlan <at> gmail.com> wrote: > >> I think users of decimal literals will just need to deal with the risk of > unexpected rounding, as the alternatives are even more problematic. > > > That is why I think we should seriously consider moving to IEEE semantics > for a decimal literal. Among other things: > > - Always round the literal inputs. > > - Supply IEEE contexts. > > - Make Decimal64 the default. > > - Add the missing rounding modes to sqrt() and exp(). > > - Keep the ability to create exact Decimals through the constructor when > no context is passed. > > - Make the Decimal constructor use the context for rounding if it is passed. > > - ... > > While I find this discussion about decimal literals extremely interesting, > in my opinion, such a literal should have an underlying completely new > numerical type, if it is really supposed to be for inexperienced users. > > Most of the discussions right now concern rounding issues that occur after > the decimal point, but I think an at least equally big problem is rounding > *to the left* of it as in (using current float): > > >>> 1e50 + 1000 > 1e+50 > > Importantly, Decimal is no cure here: > > >>> Decimal(10**50) + Decimal(100) > Decimal('1.000000000000000000000000000E+50') > > (of course, you can debate context settings to make this particular example > work, but, in general, it happens with big enough numbers.) > > The solution for this example is using ints of course: > > >>> 10**50 + 100 > 100000000000000000000000000000000000000000000000100 > > , but obviously this works only for whole numbers, so there currently is no > built-in way to make this example work correctly: > > >>> 10**50 - 9999999999999999.5 > 1e+50 > > (or with Decimal: > >>> Decimal(10**50) - Decimal('9999999999999999.5') > Decimal('1.000000000000000000000000000E+50') > ). > > If we are discussing a new number literal I would like to have it cope with > this and my suggestion is a new numeric type that's sort of a hybrid between > int and either Decimal or float, i.e., a type that behaves like int for > digits left of the decimal point, but may truncate to the right. > In pure Python, something similar (but not a literal form of course) could > be implemented as a class that stores the left digits as an int and the > right digits as a float/Decimal internally. Calculations involving this > class would be slow due to the constant need of shifting digits between the > integer?and the float part, and might be too slow for many users even when > written in C, but its behavior would meet the expectation of inexperienced > people better than the existing types. > Going back to Mark Harris' initial proposal of unifying numeric types (which > I am not trying to support in any way here), such a type would even allow to > unify int and float since an ints could be considered a subset of the new > type with a fractional part of zero. > So, I've tried my hands on the pure Python implementation of this (code below) to make it easier for people to play around with this idea. I found it easier in the end to represent both the integral and the fractional part of the new type as Python ints (and restricting the precision for the fractional component). In the compound object the integral part has arbitrary precision, the fractional part a fixed precision of 28 digits by default (and in response to Guido: this is constant even with leading zeros), but this can be changed temporarily through a context manager (a very simple one though). This is a very preliminary version that is incomplete still and could certainly be optimized in many places. It's also slow, though maybe not as slow as I thought (it's definitely fun to work with it on simple problems in interactive mode). Feedback is welcome, Wolfgang Here's the code: """ Test of a new numeric class PyNumber. Usage: Generate a new PyNumber with a fractional component of 0 (equal to an int): >>> PyNumber(3) PyNumber('3.0') Generate a PyNumber with a fractional component Decimal('0.1'): >>> PyNumber(3, Decimal('0.1')) PyNumber('3.1') Generate the same PyNumber by specifying fractional as int and exponent: >>> PyNumber(3, 1, exp = 1) PyNumber('3.1') Generate the same PyNumber again using a string: >>> PyNumber('3.1') PyNumber('3.1') Control decimal precision through context manager (default precision is 28): >>> with Prec(3): PyNumber('0.333333333333333') PyNumber('0.333') >>> with Prec(7): PyNumber('1') / PyNumber('3') PyNumber('0.3333333') Addition, subtraction, multiplication and division work as expected, but currently only between PyNumbers. """ from decimal import * from math import log, floor max_exp = 28 class Prec (object): def __init__ (self, prec): self.prec = prec def __enter__ (self): global max_exp self.old_prec = max_exp max_exp = self.prec def __exit__ (self, *_): global max_exp max_exp = self.old_prec class PyNumber (object): """Numeric type for intuitive calculations. Supports arbitrary precision of integral component, i.e., left of the decimal point, and provides fixed precision (module default: 28 digits) for the fractional component, i.e. right of the decimal point. Behaves well with calculations involving huge and small numbers, for which float and Decimal produce rounding errors. Modified sum example from statistics module: >>> sum([1e50, 1.1, -1e50] * 1000) # floating point calculation gives zero. 0.0 >>> sum([PyNumber(10**50), PyNumber('1.1'), PyNumber(-10**50)] * 1000, PyNumber(0)) PyNumber('1100.0') """ def __init__ (self, integral, fractional = 0, exp = 0): # PyNumber objects are composed of two integers, one unbounded for # representing the integral part of a number, the other restricted to # max_exp digits for representing the fractional component. if isinstance(integral, int): if integral >= 0: self.integral = integral self.sign = 1 else: self.integral = -integral self.sign = -1 if fractional < 0: raise TypeError ('negative fraction not allowed') if isinstance(fractional, int): self.fractional = fractional if fractional > 0: if exp == 0: self.exp = floor(log(fractional, 10)+1) else: self.exp = exp elif fractional == 0: self.exp = 0 else: raise TypeError ('Unknown error caused by fractional.') elif isinstance(fractional, Decimal): if fractional >= 1: raise ValueError ('need a fraction < 1.') sign, digits, exp = fractional.as_tuple() self.fractional = 0 for digit in digits: self.fractional = self.fractional*10 + digit self.exp = -exp else: raise TypeError ('Expected int or Decimal for fractional part.') elif isinstance(integral, str): x = integral.split('.') if len(x) > 2: raise ValueError('Float format string expected.') self.integral = int(x[0]) if self.integral >= 0: self.sign = 1 else: self.integral = -self.integral self.sign = -1 if len(x) == 2: self.fractional = int(x[1]) else: self.fractional = 0 if self.fractional > 0: self.exp = len(x[1]) else: self.exp = 0 else: raise TypeError('Expected int or str as first argument.') if self.exp > max_exp: self.fractional //= 10**(self.exp-max_exp) self.exp = max_exp def __add__ (self, other): if self.sign == -1: if other.sign == 1: # -1 + 2 = 2 - 1 return PyNumber(*other._sub_absolute(self)) if other.sign == -1: if self.sign == 1: # 1 + (-2) = 1 - 2 return PyNumber(*self._sub_absolute(other)) if self.exp >= other.exp: integral, fractional = self._add_absolute(other) else: integral, fractional = other._add_absolute(self) return PyNumber(self.sign * integral, fractional, self.exp) def _add_absolute(self, other): # add integral components. integral = self.integral + other.integral # adjust and add fractional components fractional = self.fractional + other.fractional * 10**(self.exp-other.exp) # determine overflow of fractional sum into integral part overflow = fractional // 10**self.exp if overflow: integral += overflow fractional -= overflow * 10**self.exp return integral, fractional def __sub__ (self, other): if self.sign == -1 and other.sign == 1: # -1 - 2 = -(1 + 2) integral, fractional = self._add_absolute(other) return PyNumber(-integral, fractional, self.exp) if other.sign == -1 and self.sign == 1: # 1 - (-2) = 1 + 2 return PyNumber(*self._add_absolute(other), exp = self.exp) if other.sign == -1 and self.sign == -1: # -1 - (-2) = 2 - 1 return PyNumber(*other._sub_absolute(self)) return PyNumber(*self._sub_absolute(other)) def _sub_absolute (self, other): sign = 1 # merge integral and fractional components into big ints. a = self.integral * 10**self.exp + self.fractional b = other.integral * 10**other.exp + other.fractional # adjust the two representations. if self.exp > other.exp: b *= 10**(self.exp-other.exp) elif other.exp > self.exp: a *= 10**(other.exp-self.exp) # subtract. diff = a-b if diff < 0: diff = -diff sign = -1 # split the big int back into integral and fractional. shift = max(self.exp, other.exp) integral, fractional = divmod(diff, 10**shift) return sign * integral, fractional, shift def __mul__ (self, other): if self.exp < other.exp: return other * self # multiply the integral components. p1 = self.integral * other.integral # adjust the fractional parts. f1 = self.fractional f2 = other.fractional * 10**(self.exp-other.exp) # cross-multiply integral and fractional components. p2 = ( self.integral * f2 * 10**self.exp + other.integral * f1 * 10**self.exp + f1 * f2) # determine overflow of cross-multiplication into integral part. shift = 2 * self.exp overflow = p2 // 10**shift if overflow > 0: p1 += overflow p2 -= overflow * 10**shift # truncate trailing zeros. while not p2 % 10: p2 //= 10 shift -= 1 sign = self.sign * other.sign return PyNumber(sign*p1, p2, shift) def __truediv__ (self, other): # merge integral and fractional components into big ints. n = self.integral * 10**self.exp + self.fractional d = other.integral * 10**other.exp + other.fractional # adjust the two representations. if self.exp > other.exp: d *= 10**(self.exp-other.exp) elif other.exp > self.exp: n *= 10**(other.exp-self.exp) # divide. integral, remainder = divmod(n, d) # turn remainder into fractional component. remainder *= 10 shift = 1 while remainder % d and shift < max_exp: remainder *= 10 shift += 1 fractional = remainder // d sign = self.sign * other.sign return PyNumber(sign * integral, fractional, shift) def __int__ (self): return self.integral def __float__ (self): return self.sign * (self.integral + self.fractional / 10**self.exp) def __repr__ (self): return "PyNumber('{0}.{1}{2}')".format(self.sign * self.integral, '0'*(self.exp-len(str(self.fractional))), self.fractional) From guido at python.org Wed Mar 12 16:37:52 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 12 Mar 2014 08:37:52 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140312T162210-499@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> Message-ID: <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> This representation makes more sense, it is fixed point. But you can just use a single integer and keep track of where the point should be. On Mar 12, 2014 8:34 AM, "Wolfgang Maier" < wolfgang.maier at biologie.uni-freiburg.de> wrote: > Wolfgang Maier <wolfgang.maier at ...> writes: > > > > > Am Montag, 10. M?rz 2014 14:53:42 UTC+1 schrieb Stefan Krah: > > > > > [My apologies for being terse, I don't have much time to follow this > > discussion right now.] > > > > > Nick Coghlan <ncoghlan <at> gmail.com> wrote: > > >> I think users of decimal literals will just need to deal with the > risk of > > unexpected rounding, as the alternatives are even more problematic. > > > > > That is why I think we should seriously consider moving to IEEE > semantics > > for a decimal literal. Among other things: > > > - Always round the literal inputs. > > > - Supply IEEE contexts. > > > - Make Decimal64 the default. > > > - Add the missing rounding modes to sqrt() and exp(). > > > - Keep the ability to create exact Decimals through the constructor > when > > no context is passed. > > > - Make the Decimal constructor use the context for rounding if it is > passed. > > > - ... > > > > While I find this discussion about decimal literals extremely > interesting, > > in my opinion, such a literal should have an underlying completely new > > numerical type, if it is really supposed to be for inexperienced users. > > > > Most of the discussions right now concern rounding issues that occur > after > > the decimal point, but I think an at least equally big problem is > rounding > > *to the left* of it as in (using current float): > > > > >>> 1e50 + 1000 > > 1e+50 > > > > Importantly, Decimal is no cure here: > > > > >>> Decimal(10**50) + Decimal(100) > > Decimal('1.000000000000000000000000000E+50') > > > > (of course, you can debate context settings to make this particular > example > > work, but, in general, it happens with big enough numbers.) > > > > The solution for this example is using ints of course: > > > > >>> 10**50 + 100 > > 100000000000000000000000000000000000000000000000100 > > > > , but obviously this works only for whole numbers, so there currently is > no > > built-in way to make this example work correctly: > > > > >>> 10**50 - 9999999999999999.5 > > 1e+50 > > > > (or with Decimal: > > >>> Decimal(10**50) - Decimal('9999999999999999.5') > > Decimal('1.000000000000000000000000000E+50') > > ). > > > > If we are discussing a new number literal I would like to have it cope > with > > this and my suggestion is a new numeric type that's sort of a hybrid > between > > int and either Decimal or float, i.e., a type that behaves like int for > > digits left of the decimal point, but may truncate to the right. > > In pure Python, something similar (but not a literal form of course) > could > > be implemented as a class that stores the left digits as an int and the > > right digits as a float/Decimal internally. Calculations involving this > > class would be slow due to the constant need of shifting digits between > the > > integer?and the float part, and might be too slow for many users even > when > > written in C, but its behavior would meet the expectation of > inexperienced > > people better than the existing types. > > Going back to Mark Harris' initial proposal of unifying numeric types > (which > > I am not trying to support in any way here), such a type would even > allow to > > unify int and float since an ints could be considered a subset of the new > > type with a fractional part of zero. > > > > So, I've tried my hands on the pure Python implementation of this (code > below) to make it easier for people to play around with this idea. I found > it easier in the end to represent both the integral and the fractional part > of the new type as Python ints (and restricting the precision for the > fractional component). > In the compound object the integral part has arbitrary precision, the > fractional part a fixed precision of 28 digits by default (and in response > to Guido: this is constant even with leading zeros), but this can be > changed > temporarily through a context manager (a very simple one though). > > This is a very preliminary version that is incomplete still and could > certainly be optimized in many places. It's also slow, though maybe not as > slow as I thought (it's definitely fun to work with it on simple problems > in > interactive mode). > > Feedback is welcome, > Wolfgang > > Here's the code: > > """ > Test of a new numeric class PyNumber. > > Usage: > > Generate a new PyNumber with a fractional component of 0 (equal to an int): > > >>> PyNumber(3) > PyNumber('3.0') > > Generate a PyNumber with a fractional component Decimal('0.1'): > > >>> PyNumber(3, Decimal('0.1')) > PyNumber('3.1') > > Generate the same PyNumber by specifying fractional as int and exponent: > > >>> PyNumber(3, 1, exp = 1) > PyNumber('3.1') > > Generate the same PyNumber again using a string: > > >>> PyNumber('3.1') > PyNumber('3.1') > > Control decimal precision through context manager (default precision is > 28): > > >>> with Prec(3): > PyNumber('0.333333333333333') > PyNumber('0.333') > > >>> with Prec(7): > PyNumber('1') / PyNumber('3') > PyNumber('0.3333333') > > Addition, subtraction, multiplication and division work as expected, but > currently only between PyNumbers. > > """ > > from decimal import * > from math import log, floor > > max_exp = 28 > > class Prec (object): > def __init__ (self, prec): > self.prec = prec > > def __enter__ (self): > global max_exp > self.old_prec = max_exp > max_exp = self.prec > > def __exit__ (self, *_): > global max_exp > max_exp = self.old_prec > > class PyNumber (object): > """Numeric type for intuitive calculations. > > Supports arbitrary precision of integral component, i.e., left of the > decimal point, and provides fixed precision (module default: 28 digits) > for the fractional component, i.e. right of the decimal point. > > Behaves well with calculations involving huge and small numbers, for > which > float and Decimal produce rounding errors. > > Modified sum example from statistics module: > > >>> sum([1e50, 1.1, -1e50] * 1000) # floating point calculation gives > zero. > 0.0 > > >>> sum([PyNumber(10**50), PyNumber('1.1'), PyNumber(-10**50)] * 1000, > PyNumber(0)) > PyNumber('1100.0') > > """ > > def __init__ (self, integral, fractional = 0, exp = 0): > # PyNumber objects are composed of two integers, one unbounded for > # representing the integral part of a number, the other restricted > to > # max_exp digits for representing the fractional component. > > if isinstance(integral, int): > if integral >= 0: > self.integral = integral > self.sign = 1 > else: > self.integral = -integral > self.sign = -1 > if fractional < 0: > raise TypeError ('negative fraction not allowed') > if isinstance(fractional, int): > self.fractional = fractional > if fractional > 0: > if exp == 0: > self.exp = floor(log(fractional, 10)+1) > else: > self.exp = exp > elif fractional == 0: > self.exp = 0 > else: > raise TypeError ('Unknown error caused by fractional.') > elif isinstance(fractional, Decimal): > if fractional >= 1: > raise ValueError ('need a fraction < 1.') > sign, digits, exp = fractional.as_tuple() > self.fractional = 0 > for digit in digits: > self.fractional = self.fractional*10 + digit > self.exp = -exp > else: > raise TypeError ('Expected int or Decimal for fractional > part.') > elif isinstance(integral, str): > x = integral.split('.') > if len(x) > 2: > raise ValueError('Float format string expected.') > self.integral = int(x[0]) > if self.integral >= 0: > self.sign = 1 > else: > self.integral = -self.integral > self.sign = -1 > if len(x) == 2: > self.fractional = int(x[1]) > else: > self.fractional = 0 > if self.fractional > 0: > self.exp = len(x[1]) > else: > self.exp = 0 > else: > raise TypeError('Expected int or str as first argument.') > if self.exp > max_exp: > self.fractional //= 10**(self.exp-max_exp) > self.exp = max_exp > > def __add__ (self, other): > if self.sign == -1: > if other.sign == 1: > # -1 + 2 = 2 - 1 > return PyNumber(*other._sub_absolute(self)) > if other.sign == -1: > if self.sign == 1: > # 1 + (-2) = 1 - 2 > return PyNumber(*self._sub_absolute(other)) > if self.exp >= other.exp: > integral, fractional = self._add_absolute(other) > else: > integral, fractional = other._add_absolute(self) > return PyNumber(self.sign * integral, fractional, self.exp) > > def _add_absolute(self, other): > # add integral components. > integral = self.integral + other.integral > # adjust and add fractional components > fractional = self.fractional + other.fractional * > 10**(self.exp-other.exp) > # determine overflow of fractional sum into integral part > overflow = fractional // 10**self.exp > if overflow: > integral += overflow > fractional -= overflow * 10**self.exp > return integral, fractional > > def __sub__ (self, other): > if self.sign == -1 and other.sign == 1: > # -1 - 2 = -(1 + 2) > integral, fractional = self._add_absolute(other) > return PyNumber(-integral, fractional, self.exp) > if other.sign == -1 and self.sign == 1: > # 1 - (-2) = 1 + 2 > return PyNumber(*self._add_absolute(other), exp = self.exp) > if other.sign == -1 and self.sign == -1: > # -1 - (-2) = 2 - 1 > return PyNumber(*other._sub_absolute(self)) > return PyNumber(*self._sub_absolute(other)) > > def _sub_absolute (self, other): > sign = 1 > # merge integral and fractional components into big ints. > a = self.integral * 10**self.exp + self.fractional > b = other.integral * 10**other.exp + other.fractional > # adjust the two representations. > if self.exp > other.exp: > b *= 10**(self.exp-other.exp) > elif other.exp > self.exp: > a *= 10**(other.exp-self.exp) > # subtract. > diff = a-b > if diff < 0: > diff = -diff > sign = -1 > # split the big int back into integral and fractional. > shift = max(self.exp, other.exp) > integral, fractional = divmod(diff, 10**shift) > return sign * integral, fractional, shift > > def __mul__ (self, other): > if self.exp < other.exp: > return other * self > # multiply the integral components. > p1 = self.integral * other.integral > # adjust the fractional parts. > f1 = self.fractional > f2 = other.fractional * 10**(self.exp-other.exp) > # cross-multiply integral and fractional components. > p2 = ( > self.integral * f2 * 10**self.exp + > other.integral * f1 * 10**self.exp + > f1 * f2) > # determine overflow of cross-multiplication into integral part. > shift = 2 * self.exp > overflow = p2 // 10**shift > if overflow > 0: > p1 += overflow > p2 -= overflow * 10**shift > # truncate trailing zeros. > while not p2 % 10: > p2 //= 10 > shift -= 1 > sign = self.sign * other.sign > return PyNumber(sign*p1, p2, shift) > > def __truediv__ (self, other): > # merge integral and fractional components into big ints. > n = self.integral * 10**self.exp + self.fractional > d = other.integral * 10**other.exp + other.fractional > # adjust the two representations. > if self.exp > other.exp: > d *= 10**(self.exp-other.exp) > elif other.exp > self.exp: > n *= 10**(other.exp-self.exp) > # divide. > integral, remainder = divmod(n, d) > # turn remainder into fractional component. > remainder *= 10 > shift = 1 > while remainder % d and shift < max_exp: > remainder *= 10 > shift += 1 > fractional = remainder // d > sign = self.sign * other.sign > return PyNumber(sign * integral, fractional, shift) > > def __int__ (self): > return self.integral > > def __float__ (self): > return self.sign * (self.integral + self.fractional / 10**self.exp) > > def __repr__ (self): > return "PyNumber('{0}.{1}{2}')".format(self.sign * self.integral, > '0'*(self.exp-len(str(self.fractional))), self.fractional) > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/b71cbc99/attachment-0001.html> From wolfgang.maier at biologie.uni-freiburg.de Wed Mar 12 16:48:39 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 12 Mar 2014 15:48:39 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> Message-ID: <loom.20140312T164301-427@post.gmane.org> Guido van Rossum <guido at ...> writes: > > This representation makes more sense, it is fixed point. But you can just use a single integer and keep track of where the point should be. Right, the main reason why I haven't tried that is that I started out with an int for representing the integral part and a Decimal for the rest, so from the beginning I always had two separate objects in the code. On the other hand, I assume that a single int might also slow down the trimming of the fractional part, which is necessary to keep its precision fixed ? From random832 at fastmail.us Wed Mar 12 16:49:09 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Wed, 12 Mar 2014 11:49:09 -0400 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> Message-ID: <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com> On Wed, Mar 12, 2014, at 11:37, Guido van Rossum wrote: > This representation makes more sense, it is fixed point. But you can just > use a single integer and keep track of where the point should be. If you're keeping track of it other than implicitly, isn't that floating point? Just of a NNNNNNN*10^-X variety instead of 1.NNNNNN*2^X. From guido at python.org Wed Mar 12 17:13:29 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 12 Mar 2014 09:13:29 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <loom.20140312T164301-427@post.gmane.org> References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> <loom.20140312T164301-427@post.gmane.org> Message-ID: <CAP7+vJ+RS+XpnLaPhcVawLoN2PGhUii91AZwYqvbHfmhPiS5VA@mail.gmail.com> On Wed, Mar 12, 2014 at 8:48 AM, Wolfgang Maier < wolfgang.maier at biologie.uni-freiburg.de> wrote: > Guido van Rossum <guido at ...> writes: > > This representation makes more sense, it is fixed point. But you can just > use a single integer and keep track of where the point should be. > > Right, the main reason why I haven't tried that is that I started out with > an int for representing the integral part and a Decimal for the rest, so > from the beginning I always had two separate objects in the code. > On the other hand, I assume that a single int might also slow down the > trimming of the fractional part, which is necessary to keep its precision > fixed ? > You can't be sure without timing it, but my expectation is that unless the number of digits is really large (like in the 1000s or more) the fewer number of operations will always be quicker. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/cfbd5403/attachment.html> From guido at python.org Wed Mar 12 17:15:15 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 12 Mar 2014 09:15:15 -0700 Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] In-Reply-To: <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com> References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> <1394639349.11461.93643329.0D43CD2F@webmail.messagingengine.com> Message-ID: <CAP7+vJLacrRWK8ewVDo9aCMk6XdM+fvMiaexk10TD0MZfmGWjA@mail.gmail.com> On Wed, Mar 12, 2014 at 8:49 AM, <random832 at fastmail.us> wrote: > On Wed, Mar 12, 2014, at 11:37, Guido van Rossum wrote: > > This representation makes more sense, it is fixed point. But you can just > > use a single integer and keep track of where the point should be. > > If you're keeping track of it other than implicitly, isn't that floating > point? Just of a NNNNNNN*10^-X variety instead of 1.NNNNNN*2^X. > No, his rules for how many digits to keep (everything to the left, only 28 to the right) make it fixed-point. Keeping track of the number explicitly just makes it simpler to create variants where instead of 28 you use another number of digits. But the use cases for this kind of representation are pretty limited. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140312/7702906a/attachment.html> From ericsnowcurrently at gmail.com Thu Mar 13 08:43:59 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 13 Mar 2014 01:43:59 -0600 Subject: [Python-ideas] re-organizing the sys module Message-ID: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> The sys module is perhaps the most core module to Python and one of the oldest. It has grown relatively organically over the years and its contents could use a little clean up. Similar proposals have come up before but without enough interest to keep them rolling. My motivation for cleaning up sys is a combination of things: * I'm working on a proposal that may include adding a sub-namespace to sys (similar to what we did with sys.implementation) and removing-ish some attributes. * The namespace is large enough that scanning quickly through it, particularly alphabetically a la help() or the docs, isn't very helpful. * Other outstanding proposals like PEP 432 may restructure underlying APIs and having the sys module mirror the resulting APIs would help reduce the cognitive overhead in this space. * It would be nice to support descriptors on the module. * The current structure does not communicate the role of the module very well. Changing the sys module should be done carefully and as little as possible. The status quo has something like a +10 modifier when it comes to sys changes. The current API is pretty stable and undoubtedly a lot of code relies on it. This proposal reflects an understanding of those considerations and addresses them in an entirely backward compatible way (including impact on alternate Python implementations). At the moment I'm not proposing any big changes to sys nor any changes at all to its API. That can come later. <wink> Instead, this proposal is about setting the stage for better flexibility in making changes to sys when the need arises. Consequently I propose just 2.5000000000000001 relatively simple changes: 1. the existing sys module be renamed to _sys (and the stdlib be changed accordingly where appropriate); 2. a pure Python sys module be added to the stdlib that wraps the existing API; 2.5. the new module should use the replace-itself-in-sys.modules technique to provide a module subclass that supports descriptors. Thoughts? -eric From ericsnowcurrently at gmail.com Thu Mar 13 09:01:03 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 13 Mar 2014 02:01:03 -0600 Subject: [Python-ideas] a standard approach to module-replace-itself-in-sys.modules Message-ID: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com> Not that long ago we had a discussion about the esoteric, yet totally supported and not going anywhere, trick of having a module replace itself in sys.modules. Presently there isn't any convention surrounding the technique. It's simply a matter of sticking something else into sys.modules[__name__]. The problem is that there are some subtle things that can go wrong in the import system if you aren't careful and people generally won't know that. (Then again the whole technique isn't exactly in the mainstream!) To help reduce the risk here, I propose that we provide a helper in importlib.util and an associated convention. The helper will be a class decorator and the convention will be to decorate 1 class at the bottom of the module, where the class will contain any relevant customizations. The decorator will do the following: 1. create a new module subclass; 2. merge the decorated class namespace into the subclass; 3. create an instance of the subclass; 4. merge the original module namespace into the instance; 5. set sys.modules[__name__] to the new instance. For example: @importlib.util.replace_module class CustomModule: CHEDDAR = None @property def limburger (self): return None There's more to it but that captures the gist. I've been exploring API ideas, but ultimately we don't need much to get the ball rolling. In the meantime, I've published an unrefined but mostly working version of replace_module() online: https://bitbucket.org/ericsnowcurrently/odds_and_ends/src/default/replace_module.py -eric From solipsis at pitrou.net Thu Mar 13 10:17:47 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 13 Mar 2014 10:17:47 +0100 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> Message-ID: <lfrt3c$62p$1@ger.gmane.org> Le 13/03/2014 08:43, Eric Snow a ?crit : > At the moment I'm not proposing any big changes to sys nor any changes > at all to its API. That can come later. <wink> Instead, this > proposal is about setting the stage for better flexibility in making > changes to sys when the need arises. Consequently I propose just > 2.5000000000000001 relatively simple changes: > > 1. the existing sys module be renamed to _sys (and the stdlib be > changed accordingly where appropriate); > 2. a pure Python sys module be added to the stdlib that wraps the existing API; I'm not sure what that brings until the need arises. However, making it a pure Python sys module means it won't be immediately available at startup like sys currently is (i.e. it's a built-in module). I don't know if there are any scenarios where it may matter. > 2.5. the new module should use the replace-itself-in-sys.modules > technique to provide a module subclass that supports descriptors. I don't think any module should be specific in that regard. Special cases generally make the language more complicated to learn. Also, you haven't explained the rationale. Regards Antoine. From ncoghlan at gmail.com Thu Mar 13 10:29:09 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Mar 2014 19:29:09 +1000 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> Message-ID: <CADiSq7eTqkjHe0Q2777bHE66ccgPpMT8VD3S2OgfdWaQ5CcS9A@mail.gmail.com> On 13 March 2014 17:43, Eric Snow <ericsnowcurrently at gmail.com> wrote: > At the moment I'm not proposing any big changes to sys nor any changes > at all to its API. That can come later. <wink> Instead, this > proposal is about setting the stage for better flexibility in making > changes to sys when the need arises. Consequently I propose just > 2.5000000000000001 relatively simple changes: > > 1. the existing sys module be renamed to _sys (and the stdlib be > changed accordingly where appropriate); > 2. a pure Python sys module be added to the stdlib that wraps the existing API; As Antoine suggests, we shouldn't do that until we have a concrete use case (although I'm generally approving of the idea, there are also benefits to having it as entirely a builtin module). It may seem counterintuitive, but it's actually easier to sell non-invasive prep work after already making the case for the subsequent change :) > 2.5. the new module should use the replace-itself-in-sys.modules > technique to provide a module subclass that supports descriptors. I'm generally against doing that without a clear rationale for why defining a new object to use isn't more appropriate (we've survived attribute -> function transitions before, like the one to using the thread safe sys.exc_info()). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From storchaka at gmail.com Thu Mar 13 13:04:15 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 13 Mar 2014 14:04:15 +0200 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <lfrt3c$62p$1@ger.gmane.org> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> <lfrt3c$62p$1@ger.gmane.org> Message-ID: <lfs6qo$tvh$1@ger.gmane.org> 13.03.14 11:17, Antoine Pitrou ???????(??): >> 1. the existing sys module be renamed to _sys (and the stdlib be >> changed accordingly where appropriate); >> 2. a pure Python sys module be added to the stdlib that wraps the >> existing API; > > I'm not sure what that brings until the need arises. > However, making it a pure Python sys module means it won't be > immediately available at startup like sys currently is (i.e. it's a > built-in module). I don't know if there are any scenarios where it may > matter. Many sys attributes (argv, displayhook, excepthook, modules, path, path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout, tracebacklimit) are used implicit in builtins and in interpreter core. Even import system doesn't work without the sys module. So it should be initialized very early and finalized very late. From brett at python.org Thu Mar 13 13:54:16 2014 From: brett at python.org (Brett Cannon) Date: Thu, 13 Mar 2014 08:54:16 -0400 Subject: [Python-ideas] a standard approach to module-replace-itself-in-sys.modules In-Reply-To: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com> References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com> Message-ID: <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com> On Thu, Mar 13, 2014 at 4:01 AM, Eric Snow <ericsnowcurrently at gmail.com>wrote: > Not that long ago we had a discussion about the esoteric, yet totally > supported and not going anywhere, trick of having a module replace > itself in sys.modules. Presently there isn't any convention > surrounding the technique. It's simply a matter of sticking something > else into sys.modules[__name__]. > > The problem is that there are some subtle things that can go wrong in > the import system if you aren't careful and people generally won't > know that. (Then again the whole technique isn't exactly in the > mainstream!) To help reduce the risk here, I propose that we provide > a helper in importlib.util and an associated convention. > Do we actually want to promote this practice by codifying support for it in the stdlib? If you're doing this to get decorators for module attributes then you should say this is for that goal and the solution is getting the module swapping hack to be easier to use (and might call for a better solution). Otherwise this proposal feels like it's trying to codify behaviour that is rarely necessary. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/18b8eb42/attachment.html> From wolfgang.maier at biologie.uni-freiburg.de Thu Mar 13 18:28:49 2014 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Thu, 13 Mar 2014 17:28:49 +0000 (UTC) Subject: [Python-ideas] numerical type combining integer and float/decimal properties [Was: Re: Python Numbers as Human Concept Decimal System] References: <loom.20140310T181321-988@post.gmane.org> <loom.20140312T162210-499@post.gmane.org> <CAP7+vJJTROJ94o_KsvcZFmzx0g0fu3n1QF=-xNZzoBST7akcNg@mail.gmail.com> Message-ID: <loom.20140313T175526-176@post.gmane.org> Guido van Rossum <guido at ...> writes: > > This representation makes more sense, it is fixed point. But you can just use a single integer and keep track of where the point should be. One of the reasons I started out with two separately stored parts has to do with my initial motivation for forking the parallel Python Numbers as Human Concept Decimal System discussion. That one is about a new numeric 'd' literal and the discussion now seems to go in the direction of having this generate a new number type closely related to the current decimal.Decimal, but somewhat simplified. I definitely agree with Oscar Benjamin on his point that decimal.Decimal from the stdlib is too complex (especially with Contexts) to be made accessible through a literal constructor, but I would even go a step further than him and say that if such a new number type gets introduced it should have additional properties that decimal.Decimal doesn't have. One of the concerns about a new type seems to be that it makes Python's number system even more complex rather than simplifying it, but my suggestion would be to make the new literal create a compound number type behaving like my PyNumber class. Please regard my use of two integers as an implementation detail here. My preference for a built-in type would actually be an integer for the integral part, but a decimal for the fractional part, it was just a bit harder to code up quickly. I envision this decimal part to be essentially the simplified decimal that Oscar Benjamin advocates for, so it would have most of the power of decimal.Decimal, but it would only be concerned with the fractional part of numbers. The user would never directly encounter this new decimal type, but instead work with the compound "PyNumber" type. I agree, this still introduces a new type, but, importantly, this new type could be considered a superset of int. In fact, every int could be thought of as a "PyNumber" with a fractional part of 0 (just like real numbers could be thought of as complex with an imaginary part of 0j). In the long run, it would thus be possible to make, e.g., division of integers return a "PyNumber" and to replace float in conversions from an integral number to a fractional one. Then there would be no need anymore to expose the skeleton of number representations by things like: >>> float(math.factorial(500)) Traceback (most recent call last): File "<pyshell#6>", line 1, in <module> float(math.factorial(500)) OverflowError: long int too large to convert to float and there would be no more precision loss like this occurring anymore: >>> a = 26313083693369353016721801216 >>> a + .5 2.631308369336935e+28 So, essentially I would prefer a potential new number type to improve the user-experience with Python numbers in more ways than currently discussed with the 'd' literal - and to come back full circle now: Having a separate representation for the type's integral component could be advantageous when this is expected to be combined with ints a lot. Cheers, Wolfgang From barry at python.org Thu Mar 13 21:03:53 2014 From: barry at python.org (Barry Warsaw) Date: Thu, 13 Mar 2014 16:03:53 -0400 Subject: [Python-ideas] re-organizing the sys module References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org> Message-ID: <20140313160353.745d9659@anarchist.wooz.org> On Mar 13, 2014, at 02:04 PM, Serhiy Storchaka wrote: >Many sys attributes (argv, displayhook, excepthook, modules, path, >path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout, >tracebacklimit) are used implicit in builtins and in interpreter core. Even >import system doesn't work without the sys module. So it should be >initialized very early and finalized very late. If the C API PySys_* functions were updated to use _sys instead, hopefully most stuff in the core would still work, right? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/c06a142f/attachment.sig> From barry at python.org Thu Mar 13 21:06:29 2014 From: barry at python.org (Barry Warsaw) Date: Thu, 13 Mar 2014 16:06:29 -0400 Subject: [Python-ideas] a standard approach to module-replace-itself-in-sys.modules References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com> <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com> Message-ID: <20140313160629.5a01af85@anarchist.wooz.org> On Mar 13, 2014, at 08:54 AM, Brett Cannon wrote: >Do we actually want to promote this practice by codifying support for it in >the stdlib? If you're doing this to get decorators for module attributes >then you should say this is for that goal and the solution is getting the >module swapping hack to be easier to use (and might call for a better >solution). Otherwise this proposal feels like it's trying to codify >behaviour that is rarely necessary. I don't think it necessarily has to be promoted as something we want people to generally do, but if they're going to do it I do think it would be nice to have a bless (i.e. stdlib) way. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/09953f01/attachment-0001.sig> From njs at pobox.com Fri Mar 14 02:59:14 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 01:59:14 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> Message-ID: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Hi all, In the process of cleaning out old design... peculiarities... in numpy, I happened to look into the history of attempts to add syntax for matrix multiplication to Python, since the lack of this is (as you'll see) at the root of various intractable problems we have. I was pretty surprised; it turns out that even though numerical folks have been whinging about missing this operator for ~15 years, the only two attempts that have been made to add it were: PEP 211, which instead adds an operator for itertools.product, aka, "maybe we can sneak matrix multiply past Guido in some sort of... large, wooden rabbit..." and PEP 225, aka "let's add 12 new operators and figure out what to do with them later" I'd have rejected these too! So I thought, maybe we should try the radical tactic of writing down what we actually want, carefully explaining why we want it, and then asking for it. And at least this way, if it gets rejected, we'll know that it was rejected for the right reasons... You'll notice that this draft is rather more developed than the average first-round PEP posting, because it's already been the rounds of all the various numerical package mailing lists to build consensus; no point in asking for the wrong thing. Don't let that slow you down, though. I think what we have here is fairly convincing and covers a lot of the design space (at least it convinced me, which I wasn't sure of at the start), but I'm still totally open to changing anything here based on comments and feedback. AFAICT the numerical community would walk over hot coals if there were an infix matrix multiplication operator on the other side. (BTW, since this is python-ideas -- have you considered adding hot coals to python 3? It might do wonders for uptake.) ...Anyway, the point is, I'm sure I can wrangle them into accepting any useful suggestions or other changes deemed necessary by the broader Python community. -n --- [begin draft PEP -- monospace font recommended] --- PEP: XXXX Title: Dedicated infix operators for matrix multiplication and matrix power Version: $Revision$ Last-Modified: $Date$ Author: Nathaniel J. Smith <njs at pobox.com> Status: Draft Type: Standards Track Python-Version: 3.5 Content-Type: text/x-rst Created: 20-Feb-2014 Post-History: Abstract ======== This PEP proposes two new binary operators dedicated to matrix multiplication and matrix power, spelled ``@`` and ``@@`` respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) Specification ============= Two new binary operators are added to the Python language, together with corresponding in-place versions: ======= ========================= =============================== Op Precedence/associativity Methods ======= ========================= =============================== ``@`` Same as ``*`` ``__matmul__``, ``__rmatmul__`` ``@@`` Same as ``**`` ``__matpow__``, ``__rmatpow__`` ``@=`` n/a ``__imatmul__`` ``@@=`` n/a ``__imatpow__`` ======= ========================= =============================== No implementations of these methods are added to the builtin or standard library types. However, a number of projects have reached consensus on the recommended semantics for these operations; see `Intended usage details`_ below. Motivation ========== Executive summary ----------------- In numerical code, there are two important operations which compete for use of Python's ``*`` operator: elementwise multiplication, and matrix multiplication. In the nearly twenty years since the Numeric library was first proposed, there have been many attempts to resolve this tension [#hugunin]_; none have been really satisfactory. Currently, most numerical Python code uses ``*`` for elementwise multiplication, and function/method syntax for matrix multiplication; however, this leads to ugly and unreadable code in common circumstances. The problem is bad enough that significant amounts of code continue to use the opposite convention (which has the virtue of producing ugly and unreadable code in *different* circumstances), and this API fragmentation across codebases then creates yet more problems. There does not seem to be any *good* solution to the problem of designing a numerical API within current Python syntax -- only a landscape of options that are bad in different ways. The minimal change to Python syntax which is sufficient to resolve these problems is the addition of a single new infix operator for matrix multiplication. Matrix multiplication has a singular combination of features which distinguish it from other binary operations, which together provide a uniquely compelling case for the addition of a dedicated infix operator: * Just as for the existing numerical operators, there exists a vast body of prior art supporting the use of infix notation for matrix multiplication across all fields of mathematics, science, and engineering; ``@`` harmoniously fills a hole in Python's existing operator system. * ``@`` greatly clarifies real-world code. * ``@`` provides a smoother onramp for less experienced users, who are particularly harmed by hard-to-read code and API fragmentation. * ``@`` benefits a substantial and growing portion of the Python user community. * ``@`` will be used frequently -- in fact, evidence suggests it may be used more frequently than ``//`` or the bitwise operators. * ``@`` allows the Python numerical community to reduce fragmentation, and finally standardize on a single consensus duck type for all numerical array objects. And, given the existence of ``@``, it makes more sense than not to have ``@@``, ``@=``, and ``@@=``, so they are added as well. Background: What's wrong with the status quo? --------------------------------------------- When we crunch numbers on a computer, we usually have lots and lots of numbers to deal with. Trying to deal with them one at a time is cumbersome and slow -- especially when using an interpreted language. Instead, we want the ability to write down simple operations that apply to large collections of numbers all at once. The *n-dimensional array* is the basic object that all popular numeric computing environments use to make this possible. Python has several libraries that provide such arrays, with numpy being at present the most prominent. When working with n-dimensional arrays, there are two different ways we might want to define multiplication. One is elementwise multiplication:: [[1, 2], [[11, 12], [[1 * 11, 2 * 12], [3, 4]] x [13, 14]] = [3 * 13, 4 * 14]] and the other is `matrix multiplication`_: .. _matrix multiplication: https://en.wikipedia.org/wiki/Matrix_multiplication :: [[1, 2], [[11, 12], [[1 * 11 + 2 * 13, 1 * 12 + 2 * 14], [3, 4]] x [13, 14]] = [3 * 11 + 4 * 13, 3 * 12 + 4 * 14]] Elementwise multiplication is useful because it lets us easily and quickly perform many multiplications on a large collection of values, without writing a slow and cumbersome ``for`` loop. And this works as part of a very general schema: when using the array objects provided by numpy or other numerical libraries, all Python operators work elementwise on arrays of all dimensionalities. The result is that one can write functions using straightforward code like ``a * b + c / d``, treating the variables as if they were simple values, but then immediately use this function to efficiently perform this calculation on large collections of values, while keeping them organized using whatever arbitrarily complex array layout works best for the problem at hand. Matrix multiplication is more of a special case. It's only defined on 2d arrays (also known as "matrices"), and multiplication is the only operation that has a meaningful "matrix" version -- "matrix addition" is the same as elementwise addition; there is no such thing as "matrix bitwise-or" or "matrix floordiv"; "matrix division" can be defined but is not very useful, etc. However, matrix multiplication is still used very heavily across all numerical application areas; mathematically, it's one of the most fundamental operations there is. Because Python syntax currently allows for only a single multiplication operator ``*``, libraries providing array-like objects must decide: either use ``*`` for elementwise multiplication, or use ``*`` for matrix multiplication. And, unfortunately, it turns out that when doing general-purpose number crunching, both operations are used frequently, and there are major advantages to using infix rather than function call syntax in both cases. Thus it is not at all clear which convention is optimal, or even acceptable; often it varies on a case-by-case basis. Nonetheless, network effects mean that it is very important that we pick *just one* convention. In numpy, for example, it is technically possible to switch between the conventions, because numpy provides two different types with different ``__mul__`` methods. For ``numpy.ndarray`` objects, ``*`` performs elementwise multiplication, and matrix multiplication must use a function call (``numpy.dot``). For ``numpy.matrix`` objects, ``*`` performs matrix multiplication, and elementwise multiplication requires function syntax. Writing code using ``numpy.ndarray`` works fine. Writing code using ``numpy.matrix`` also works fine. But trouble begins as soon as we try to integrate these two pieces of code together. Code that expects an ``ndarray`` and gets a ``matrix``, or vice-versa, may crash or return incorrect results. Keeping track of which functions expect which types as inputs, and return which types as outputs, and then converting back and forth all the time, is incredibly cumbersome and impossible to get right at any scale. Functions that defensively try to handle both types as input and DTRT, find themselves floundering into a swamp of ``isinstance`` and ``if`` statements. PEP 238 split ``/`` into two operators: ``/`` and ``//``. Imagine the chaos that would have resulted if it had instead split ``int`` into two types: ``classic_int``, whose ``__div__`` implemented floor division, and ``new_int``, whose ``__div__`` implemented true division. This, in a more limited way, is the situation that Python number-crunchers currently find themselves in. In practice, the vast majority of projects have settled on the convention of using ``*`` for elementwise multiplication, and function call syntax for matrix multiplication (e.g., using ``numpy.ndarray`` instead of ``numpy.matrix``). This reduces the problems caused by API fragmentation, but it doesn't eliminate them. The strong desire to use infix notation for matrix multiplication has caused a number of specialized array libraries to continue to use the opposing convention (e.g., scipy.sparse, pyoperators, pyviennacl) despite the problems this causes, and ``numpy.matrix`` itself still gets used in introductory programming courses, often appears in StackOverflow answers, and so forth. Well-written libraries thus must continue to be prepared to deal with both types of objects, and, of course, are also stuck using unpleasant funcall syntax for matrix multiplication. After nearly two decades of trying, the numerical community has still not found any way to resolve these problems within the constraints of current Python syntax (see `Rejected alternatives to adding a new operator`_ below). This PEP proposes the minimum effective change to Python syntax that will allow us to drain this swamp. It splits ``*`` into two operators, just as was done for ``/``: ``*`` for elementwise multiplication, and ``@`` for matrix multiplication. (Why not the reverse? Because this way is compatible with the existing consensus, and because it gives us a consistent rule that all the built-in numeric operators also apply in an elementwise manner to arrays; the reverse convention would lead to more special cases.) So that's why matrix multiplication doesn't and can't just use ``*``. Now, in the the rest of this section, we'll explain why it nonetheless meets the high bar for adding a new operator. Why should matrix multiplication be infix? ------------------------------------------ Right now, most numerical code in Python uses syntax like ``numpy.dot(a, b)`` or ``a.dot(b)`` to perform matrix multiplication. This obviously works, so why do people make such a fuss about it, even to the point of creating API fragmentation and compatibility swamps? Matrix multiplication shares two features with ordinary arithmetic operations like addition and multiplication on numbers: (a) it is used very heavily in numerical programs -- often multiple times per line of code -- and (b) it has an ancient and universally adopted tradition of being written using infix syntax. This is because, for typical formulas, this notation is dramatically more readable than any function call syntax. Here's an example to demonstrate: One of the most useful tools for testing a statistical hypothesis is the linear hypothesis test for OLS regression models. It doesn't really matter what all those words I just said mean; if we find ourselves having to implement this thing, what we'll do is look up some textbook or paper on it, and encounter many mathematical formulas that look like: .. math:: S = (H \beta - r)^T (H V H^T)^{-1} (H \beta - r) Here the various variables are all vectors or matrices (details for the curious: [#lht]_). Now we need to write code to perform this calculation. In current numpy, matrix multiplication can be performed using either the function or method call syntax. Neither provides a particularly readable translation of the formula:: import numpy as np from numpy.linalg import inv, solve # Using dot function: S = np.dot((np.dot(H, beta) - r).T, np.dot(inv(np.dot(np.dot(H, V), H.T)), np.dot(H, beta) - r)) # Using dot method: S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r) With the ``@`` operator, the direct translation of the above formula becomes:: S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) Notice that there is now a transparent, 1-to-1 mapping between the symbols in the original formula and the code that implements it. Of course, an experienced programmer will probably notice that this is not the best way to compute this expression. The repeated computation of :math:`H \beta - r` should perhaps be factored out; and, expressions of the form ``dot(inv(A), B)`` should almost always be replaced by the more numerically stable ``solve(A, B)``. When using ``@``, performing these two refactorings gives us:: # Version 1 (as above) S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) # Version 2 trans_coef = H @ beta - r S = trans_coef.T @ inv(H @ V @ H.T) @ trans_coef # Version 3 S = trans_coef.T @ solve(H @ V @ H.T, trans_coef) Notice that when comparing between each pair of steps, it's very easy to see exactly what was changed. If we apply the equivalent transformations to the code using the .dot method, then the changes are much harder to read out or verify for correctness:: # Version 1 (as above) S = (H.dot(beta) - r).T.dot(inv(H.dot(V).dot(H.T))).dot(H.dot(beta) - r) # Version 2 trans_coef = H.dot(beta) - r S = trans_coef.T.dot(inv(H.dot(V).dot(H.T))).dot(trans_coef) # Version 3 S = trans_coef.T.dot(solve(H.dot(V).dot(H.T)), trans_coef) Readability counts! The statements using ``@`` are shorter, contain more whitespace, can be directly and easily compared both to each other and to the textbook formula, and contain only meaningful parentheses. This last point is particularly important for readability: when using function-call syntax, the required parentheses on every operation create visual clutter that makes it very difficult to parse out the overall structure of the formula by eye, even for a relatively simple formula like this one. Eyes are terrible at parsing non-regular languages. I made and caught many errors while trying to write out the 'dot' formulas above. I know they still contain at least one error, maybe more. (Exercise: find it. Or them.) The ``@`` examples, by contrast, are not only correct, they're obviously correct at a glance. If we are even more sophisticated programmers, and writing code that we expect to be reused, then considerations of speed or numerical accuracy might lead us to prefer some particular order of evaluation. Because ``@`` makes it possible to omit irrelevant parentheses, we can be certain that if we *do* write something like ``(H @ V) @ H.T``, then our readers will know that the parentheses must have been added intentionally to accomplish some meaningful purpose. In the ``dot`` examples, it's impossible to know which nesting decisions are important, and which are arbitrary. Infix ``@`` dramatically improves matrix code usability at all stages of programmer interaction. Transparent syntax is especially crucial for non-expert programmers ------------------------------------------------------------------- A large proportion of scientific code is written by people who are experts in their domain, but are not experts in programming. And there are many university courses run each year with titles like "Data analysis for social scientists" which assume no programming background, and teach some combination of mathematical techniques, introduction to programming, and the use of programming to implement these mathematical techniques, all within a 10-15 week period. These courses are more and more often being taught in Python rather than special-purpose languages like R or Matlab. For these kinds of users, whose programming knowledge is fragile, the existence of a transparent mapping between formulas and code often means the difference between succeeding and failing to write that code at all. This is so important that such classes often use the ``numpy.matrix`` type which defines ``*`` to mean matrix multiplication, even though this type is buggy and heavily disrecommended by the rest of the numpy community for the fragmentation that it causes. This pedagogical use case is, in fact, the *only* reason ``numpy.matrix`` remains a supported part of numpy. Adding ``@`` will benefit both beginning and advanced users with better syntax; and furthermore, it will allow both groups to standardize on the same notation from the start, providing a smoother on-ramp to expertise. But isn't matrix multiplication a pretty niche requirement? ----------------------------------------------------------- The world is full of continuous data, and computers are increasingly called upon to work with it in sophisticated ways. Arrays are the lingua franca of finance, machine learning, 3d graphics, computer vision, robotics, operations research, econometrics, meteorology, computational linguistics, recommendation systems, neuroscience, astronomy, bioinformatics (including genetics, cancer research, drug discovery, etc.), physics engines, quantum mechanics, geophysics, network analysis, and many other application areas. In most or all of these areas, Python is rapidly becoming a dominant player, in large part because of its ability to elegantly mix traditional discrete data structures (hash tables, strings, etc.) on an equal footing with modern numerical data types and algorithms. We all live in our own little sub-communities, so some Python users may be surprised to realize the sheer extent to which Python is used for number crunching -- especially since much of this particular sub-community's activity occurs outside of traditional Python/FOSS channels. So, to give some rough idea of just how many numerical Python programmers are actually out there, here are two numbers: In 2013, there were 7 international conferences organized specifically on numerical Python [#scipy-conf]_ [#pydata-conf]_. At PyCon 2014, ~20% of the tutorials appear to involve the use of matrices [#pycon-tutorials]_. To quantify this further, we used Github's "search" function to look at what modules are actually imported across a wide range of real-world code (i.e., all the code on Github). We checked for imports of several popular stdlib modules, a variety of numerically oriented modules, and various other extremely high-profile modules like django and lxml (the latter of which is the #1 most downloaded package on PyPI). Starred lines indicate packages which export array- or matrix-like objects which will adopt ``@`` if this PEP is approved:: Count of Python source files on Github matching given search terms (as of 2014-04-10, ~21:00 UTC) ================ ========== =============== ======= =========== module "import X" "from X import" total total/numpy ================ ========== =============== ======= =========== sys 2374638 63301 2437939 5.85 os 1971515 37571 2009086 4.82 re 1294651 8358 1303009 3.12 numpy ************** 337916 ********** 79065 * 416981 ******* 1.00 warnings 298195 73150 371345 0.89 subprocess 281290 63644 344934 0.83 django 62795 219302 282097 0.68 math 200084 81903 281987 0.68 threading 212302 45423 257725 0.62 pickle+cPickle 215349 22672 238021 0.57 matplotlib 119054 27859 146913 0.35 sqlalchemy 29842 82850 112692 0.27 pylab *************** 36754 ********** 41063 ** 77817 ******* 0.19 scipy *************** 40829 ********** 28263 ** 69092 ******* 0.17 lxml 19026 38061 57087 0.14 zlib 40486 6623 47109 0.11 multiprocessing 25247 19850 45097 0.11 requests 30896 560 31456 0.08 jinja2 8057 24047 32104 0.08 twisted 13858 6404 20262 0.05 gevent 11309 8529 19838 0.05 pandas ************** 14923 *********** 4005 ** 18928 ******* 0.05 sympy 2779 9537 12316 0.03 theano *************** 3654 *********** 1828 *** 5482 ******* 0.01 ================ ========== =============== ======= =========== These numbers should be taken with several grains of salt (see footnote for discussion: [#github-details]_), but, to the extent they can be trusted, they suggest that ``numpy`` might be the single most-imported non-stdlib module in the entire Pythonverse; it's even more-imported than such stdlib stalwarts as ``subprocess``, ``math``, ``pickle``, and ``threading``. And numpy users represent only a subset of the broader numerical community that will benefit from the ``@`` operator. Matrices may once have been a niche data type restricted to Fortran programs running in university labs and military clusters, but those days are long gone. Number crunching is a mainstream part of modern Python usage. In addition, there is some precedence for adding an infix operator to handle a more-specialized arithmetic operation: the floor division operator ``//``, like the bitwise operators, is very useful under certain circumstances when performing exact calculations on discrete values. But it seems likely that there are many Python programmers who have never had reason to use ``//`` (or, for that matter, the bitwise operators). ``@`` is no more niche than ``//``. So ``@`` is good for matrix formulas, but how common are those really? ---------------------------------------------------------------------- We've seen that ``@`` makes matrix formulas dramatically easier to work with for both experts and non-experts, that matrix formulas appear in many important applications, and that numerical libraries like numpy are used by a substantial proportion of Python's user base. But numerical libraries aren't just about matrix formulas, and being important doesn't necessarily mean taking up a lot of code: if matrix formulas only occured in one or two places in the average numerically-oriented project, then it still wouldn't be worth adding a new operator. So how common is matrix multiplication, really? When the going gets tough, the tough get empirical. To get a rough estimate of how useful the ``@`` operator will be, the table below shows the rate at which different Python operators are actually used in the stdlib, and also in two high-profile numerical packages -- the scikit-learn machine learning library, and the nipy neuroimaging library -- normalized by source lines of code (SLOC). Rows are sorted by the 'combined' column, which pools all three code bases together. The combined column is thus strongly weighted towards the stdlib, which is much larger than both projects put together (stdlib: 411575 SLOC, scikit-learn: 50924 SLOC, nipy: 37078 SLOC). [#sloc-details]_ The ``dot`` row (marked ``******``) counts how common matrix multiply operations are in each codebase. :: ==== ====== ============ ==== ======== op stdlib scikit-learn nipy combined ==== ====== ============ ==== ======== = 2969 5536 4932 3376 / 10,000 SLOC - 218 444 496 261 + 224 201 348 231 == 177 248 334 196 * 156 284 465 192 % 121 114 107 119 ** 59 111 118 68 != 40 56 74 44 / 18 121 183 41 > 29 70 110 39 += 34 61 67 39 < 32 62 76 38 >= 19 17 17 18 <= 18 27 12 18 dot ***** 0 ********** 99 ** 74 ****** 16 | 18 1 2 15 & 14 0 6 12 << 10 1 1 8 // 9 9 1 8 -= 5 21 14 8 *= 2 19 22 5 /= 0 23 16 4 >> 4 0 0 3 ^ 3 0 0 3 ~ 2 4 5 2 |= 3 0 0 2 &= 1 0 0 1 //= 1 0 0 1 ^= 1 0 0 0 **= 0 2 0 0 %= 0 0 0 0 <<= 0 0 0 0 >>= 0 0 0 0 ==== ====== ============ ==== ======== These two numerical packages alone contain ~780 uses of matrix multiplication. Within these packages, matrix multiplication is used more heavily than most comparison operators (``<`` ``!=`` ``<=`` ``>=``). Even when we dilute these counts by including the stdlib into our comparisons, matrix multiplication is still used more often in total than any of the bitwise operators, and 2x as often as ``//``. This is true even though the stdlib, which contains a fair amount of integer arithmetic and no matrix operations, makes up more than 80% of the combined code base. By coincidence, the numeric libraries make up approximately the same proportion of the 'combined' codebase as numeric tutorials make up of PyCon 2014's tutorial schedule, which suggests that the 'combined' column may not be *wildly* unrepresentative of new Python code in general. While it's impossible to know for certain, from this data it seems entirely possible that across all Python code currently being written, matrix multiplication is already used more often than ``//`` and the bitwise operations. But isn't it weird to add an operator with no stdlib uses? ---------------------------------------------------------- It's certainly unusual (though ``Ellipsis`` was also added without any stdlib uses). But the important thing is whether a change will benefit users, not where the software is being downloaded from. It's clear from the above that ``@`` will be used, and used heavily. And this PEP provides the critical piece that will allow the Python numerical community to finally reach consensus on a standard duck type for all array-like objects, which is a necessary precondition to ever adding a numerical array type to the stdlib. Matrix power and in-place operators ----------------------------------- The primary motivation for this PEP is ``@``; the other proposed operators don't have nearly as much impact. The matrix power operator ``@@`` is useful and well-defined, but not really necessary. It is still included, though, for consistency: if we have an ``@`` that is analogous to ``*``, then it would be weird and surprising to *not* have an ``@@`` that is analogous to ``**``. Similarly, the in-place operators ``@=`` and ``@@=`` provide limited value -- it's more common to write ``a = (b @ a)`` than it is to write ``a = (a @ b)``, and in-place matrix operations still generally have to allocate substantial temporary storage -- but they are included for completeness and symmetry. Compatibility considerations ============================ Currently, the only legal use of the ``@`` token in Python code is at statement beginning in decorators. The new operators are all infix; the one place they can never occur is at statement beginning. Therefore, no existing code will be broken by the addition of these operators, and there is no possible parsing ambiguity between decorator-@ and the new operators. Another important kind of compatibility is the mental cost paid by users to update their understanding of the Python language after this change, particularly for users who do not work with matrices and thus do not benefit. Here again, ``@`` has minimal impact: even comprehensive tutorials and references will only need to add a sentence or two to fully document this PEP's changes for a non-numerical audience. Intended usage details ====================== This section is informative, rather than normative -- it documents the consensus of a number of libraries that provide array- or matrix-like objects on how the ``@`` and ``@@`` operators will be implemented. This section uses the numpy terminology for describing arbitrary multidimensional arrays of data, because it is a superset of all other commonly used models. In this model, the *shape* of any array is represented by a tuple of integers. Because matrices are two-dimensional, they have len(shape) == 2, while 1d vectors have len(shape) == 1, and scalars have shape == (), i.e., they are "0 dimensional". Any array contains prod(shape) total entries. Notice that `prod(()) == 1`_ (for the same reason that sum(()) == 0); scalars are just an ordinary kind of array, not a special case. Notice also that we distinguish between a single scalar value (shape == (), analogous to ``1``), a vector containing only a single entry (shape == (1,), analogous to ``[1]``), a matrix containing only a single entry (shape == (1, 1), analogous to ``[[1]]``), etc., so the dimensionality of any array is always well-defined. Other libraries with more restricted representations (e.g., those that support 2d arrays only) might implement only a subset of the functionality described here. .. _prod(()) == 1: https://en.wikipedia.org/wiki/Empty_product Semantics --------- The recommended semantics for ``@`` for different inputs are: * 2d inputs are conventional matrices, and so the semantics are obvious: we apply conventional matrix multiplication. If we write ``arr(2, 3)`` to represent an arbitrary 2x3 array, then ``arr(3, 4) @ arr(4, 5)`` returns an array with shape (3, 5). * 1d vector inputs are promoted to 2d by prepending or appending a '1' to the shape, the operation is performed, and then the added dimension is removed from the output. The 1 is always added on the "outside" of the shape: prepended for left arguments, and appended for right arguments. The result is that matrix @ vector and vector @ matrix are both legal (assuming compatible shapes), and both return 1d vectors; vector @ vector returns a scalar. This is clearer with examples. * ``arr(2, 3) @ arr(3, 1)`` is a regular matrix product, and returns an array with shape (2, 1), i.e., a column vector. * ``arr(2, 3) @ arr(3)`` performs the same computation as the previous (i.e., treats the 1d vector as a matrix containing a single *column*, shape = (3, 1)), but returns the result with shape (2,), i.e., a 1d vector. * ``arr(1, 3) @ arr(3, 2)`` is a regular matrix product, and returns an array with shape (1, 2), i.e., a row vector. * ``arr(3) @ arr(3, 2)`` performs the same computation as the previous (i.e., treats the 1d vector as a matrix containing a single *row*, shape = (1, 3)), but returns the result with shape (2,), i.e., a 1d vector. * ``arr(1, 3) @ arr(3, 1)`` is a regular matrix product, and returns an array with shape (1, 1), i.e., a single value in matrix form. * ``arr(3) @ arr(3)`` performs the same computation as the previous, but returns the result with shape (), i.e., a single scalar value, not in matrix form. So this is the standard inner product on vectors. An infelicity of this definition for 1d vectors is that it makes ``@`` non-associative in some cases (``(Mat1 @ vec) @ Mat2`` != ``Mat1 @ (vec @ Mat2)``). But this seems to be a case where practicality beats purity: non-associativity only arises for strange expressions that would never be written in practice; if they are written anyway then there is a consistent rule for understanding what will happen (``Mat1 @ vec @ Mat2`` is parsed as ``(Mat1 @ vec) @ Mat2``, just like ``a - b - c``); and, not supporting 1d vectors would rule out many important use cases that do arise very commonly in practice. No-one wants to explain to new users why to solve the simplest linear system in the obvious way, they have to type ``(inv(A) @ b[:, np.newaxis]).flatten()`` instead of ``inv(A) @ b``, or perform an ordinary least-squares regression by typing ``solve(X.T @ X, X @ y[:, np.newaxis]).flatten()`` instead of ``solve(X.T @ X, X @ y)``. No-one wants to type ``(a[np.newaxis, :] @ b[:, np.newaxis])[0, 0]`` instead of ``a @ b`` every time they compute an inner product, or ``(a[np.newaxis, :] @ Mat @ b[:, np.newaxis])[0, 0]`` for general quadratic forms instead of ``a @ Mat @ b``. In addition, sage and sympy (see below) use these non-associative semantics with an infix matrix multiplication operator (they use ``*``), and they report that they haven't experienced any problems caused by it. * For inputs with more than 2 dimensions, we treat the last two dimensions as being the dimensions of the matrices to multiply, and 'broadcast' across the other dimensions. This provides a convenient way to quickly compute many matrix products in a single operation. For example, ``arr(10, 2, 3) @ arr(10, 3, 4)`` performs 10 separate matrix multiplies, each of which multiplies a 2x3 and a 3x4 matrix to produce a 2x4 matrix, and then returns the 10 resulting matrices together in an array with shape (10, 2, 4). The intuition here is that we treat these 3d arrays of numbers as if they were 1d arrays *of matrices*, and then apply matrix multiplication in an elementwise manner, where now each 'element' is a whole matrix. Note that broadcasting is not limited to perfectly aligned arrays; in more complicated cases, it allows several simple but powerful tricks for controlling how arrays are aligned with each other; see [#broadcasting]_ for details. (In particular, it turns out that when broadcasting is taken into account, the standard scalar * matrix product is a special case of the elementwise multiplication operator ``*``.) If one operand is >2d, and another operand is 1d, then the above rules apply unchanged, with 1d->2d promotion performed before broadcasting. E.g., ``arr(10, 2, 3) @ arr(3)`` first promotes to ``arr(10, 2, 3) @ arr(3, 1)``, then broadcasts the right argument to create the aligned operation ``arr(10, 2, 3) @ arr(10, 3, 1)``, multiplies to get an array with shape (10, 2, 1), and finally removes the added dimension, returning an array with shape (10, 2). Similarly, ``arr(2) @ arr(10, 2, 3)`` produces an intermediate array with shape (10, 1, 3), and a final array with shape (10, 3). * 0d (scalar) inputs raise an error. Scalar * matrix multiplication is a mathematically and algorithmically distinct operation from matrix @ matrix multiplication, and is already covered by the elementwise ``*`` operator. Allowing scalar @ matrix would thus both require an unnecessary special case, and violate TOOWTDI. The recommended semantics for ``@@`` are:: def __matpow__(self, n): if not isinstance(n, numbers.Integral): raise TypeError("@@ not implemented for fractional powers") if n == 0: return identity_matrix_with_shape(self.shape) elif n < 0: return inverse(self) @ (self @@ (n + 1)) else: return self @ (self @@ (n - 1)) (Of course we expect that much more efficient implementations will be used in practice.) Notice that if given an appropriate definition of ``identity_matrix_with_shape``, then this definition will automatically handle >2d arrays appropriately. Notice also that with this definition, ``vector @@ 2`` gives the squared Euclidean length of the vector, a commonly used value. Also, while it is rarely useful to explicitly compute inverses or other negative powers in standard immediate-mode dense matrix code, these computations are natural when doing symbolic or deferred-mode computations (as in e.g. sympy, theano, numba, numexpr); therefore, negative powers are fully supported. Fractional powers, though, bring in variety of `mathematical complications`_, so we leave it to individual projects to decide whether they want to try to define some reasonable semantics for fractional inputs. .. _`mathematical complications`: https://en.wikipedia.org/wiki/Square_root_of_a_matrix Adoption -------- We group existing Python projects which provide array- or matrix-like types based on what API they currently use for elementwise and matrix multiplication. **Projects which currently use * for *elementwise* multiplication, and function/method calls for *matrix* multiplication:** The developers of the following projects have expressed an intention to implement ``@`` and ``@@`` on their array-like types using the above semantics: * numpy * pandas * blaze * theano The following projects have been alerted to the existence of the PEP, but it's not yet known what they plan to do if it's accepted. We don't anticipate that they'll have any objections, though, since everything proposed here is consistent with how they already do things: * pycuda * panda3d **Projects which currently use * for *matrix* multiplication, and function/method calls for *elementwise* multiplication:** The following projects have expressed an intention, if this PEP is accepted, to migrate from their current API to the elementwise-``*``, matmul-``@`` convention (i.e., this is a list of projects whose API fragmentation will probably be eliminated if this PEP is accepted): * numpy (``numpy.matrix``) * scipy.sparse * pyoperators * pyviennacl The following projects have been alerted to the existence of the PEP, but it's not known what they plan to do if it's accepted (i.e., this is a list of projects whose API fragmentation may or may not be eliminated if this PEP is accepted): * cvxopt **Projects which currently use * for *matrix* multiplication, and which do not implement elementwise multiplication at all:** There are several projects which implement matrix types, but from a very different perspective than the numerical libraries discussed above. These projects focus on computational methods for analyzing matrices in the sense of abstract mathematical objects (i.e., linear maps over free modules over rings), rather than as big bags full of numbers that need crunching. And it turns out that from the abstract math point of view, there isn't much use for elementwise operations in the first place; as discussed in the Background section above, elementwise operations are motivated by the bag-of-numbers approach. The different goals of these projects mean that they don't encounter the basic problem that this PEP exists to address, making it mostly irrelevant to them; while they appear superficially similar, they're actually doing something quite different. They use ``*`` for matrix multiplication (and for group actions, and so forth), and if this PEP is accepted, their expressed intention is to continue doing so, while perhaps adding ``@`` and ``@@`` on matrices as aliases for ``*`` and ``**``: * sympy * sage If you know of any actively maintained Python libraries which provide an interface for working with numerical arrays or matrices, and which are not listed above, then please let the PEP author know: njs at pobox.com Rationale for specification details =================================== Choice of operator ------------------ Why ``@`` instead of some other punctuation symbol? It doesn't matter much, and there isn't any consensus across other programming languages about how this operator should be named [#matmul-other-langs]_, but ``@`` has a few advantages: * ``@`` is a friendly character that Pythoneers are already used to typing in decorators, and its use in email addresses means it is more likely to be easily accessible across keyboard layouts than some other characters (e.g. ``$`` or non-ASCII characters). * The mATrices mnemonic is cute. * It's round like ``*`` and :math:`\cdot`. * The use of a single-character token makes ``@@`` possible, which is a nice bonus. * The swirly shape is reminiscent of the simultaneous sweeps over rows and columns that define matrix multiplication; its asymmetry is evocative of its non-commutative nature. (Non)-Definitions for built-in types ------------------------------------ No ``__matmul__`` or ``__matpow__`` are defined for builtin numeric types (``float``, ``int``, etc.) or for the ``numbers.Number`` hierarchy, because these types represent scalars, and the consensus semantics for ``@`` are that it should raise an error on scalars. We do not -- for now -- define a ``__matmul__`` method on the standard ``memoryview`` or ``array.array`` objects, for several reasons. First, there is currently no way to create multidimensional memoryview objects using only the stdlib, and array objects cannot represent multidimensional data at all, which makes ``__matmul__`` much less useful. Second, providing a quality implementation of matrix multiplication is highly non-trivial. Naive nested loop implementations are very slow and providing one in CPython would just create a trap for users. But the alternative -- providing a modern, competitive matrix multiply -- would require that CPython link to a BLAS library, which brings a set of new complications. In particular, several popular BLAS libraries (including the one that ships by default on OS X) currently break the use of ``multiprocessing`` [#blas-fork]_. And finally, we'd have to add quite a bit beyond ``__matmul__`` before ``memoryview`` or ``array.array`` would be useful for numeric work -- like elementwise versions of the other arithmetic operators, just to start. Put together, these considerations mean that the cost/benefit of adding ``__matmul__`` to these types just isn't there, so for now we'll continue to delegate these problems to numpy and friends, and defer a more systematic solution to a future proposal. There are also non-numeric Python builtins which define ``__mul__`` (``str``, ``list``, ...). We do not define ``__matmul__`` for these types either, because why would we even do that. Unresolved issues ----------------- Associativity of ``@`` '''''''''''''''''''''' It's been suggested that ``@`` should be right-associative, on the grounds that for expressions like ``Mat @ Mat @ vec``, the two different evaluation orders produce the same result, but the right-associative order ``Mat @ (Mat @ vec)`` will be faster and use less memory than the left-associative order ``(Mat @ Mat) @ vec``. (Matrix-vector multiplication is much cheaper than matrix-matrix multiplication). It would be a shame if users found themselves required to use an overabundance of parentheses to achieve acceptable speed/memory usage in common situations, but, it's not currently clear whether such cases actually are common enough to override Python's general rule of left-associativity, or even whether they're more common than the symmetric cases where left-associativity would be faster (though this does seem intuitively plausible). The only way to answer this is probably to do an audit of some real-world uses and check how often the associativity matters in practice; if this PEP is accepted in principle, then we should probably do this check before finalizing it. Rejected alternatives to adding a new operator ============================================== Over the past few decades, the Python numeric community has explored a variety of ways to resolve the tension between matrix and elementwise multiplication operations. PEP 211 and PEP 225, both proposed in 2000 and last seriously discussed in 2008 [#threads-2008]_, were early attempts to add new operators to solve this problem, but suffered from serious flaws; in particular, at that time the Python numerical community had not yet reached consensus on the proper API for array objects, or on what operators might be needed or useful (e.g., PEP 225 proposes 6 new operators with unspecified semantics). Experience since then has now led to consensus that the best solution, for both numeric Python and core Python, is to add a single infix operator for matrix multiply (together with the other new operators this implies like ``@=``). We review some of the rejected alternatives here. **Use a second type that defines __mul__ as matrix multiplication:** As discussed above (`Background: What's wrong with the status quo?`_), this has been tried this for many years via the ``numpy.matrix`` type (and its predecessors in Numeric and numarray). The result is a strong consensus among both numpy developers and developers of downstream packages that ``numpy.matrix`` should essentially never be used, because of the problems caused by having conflicting duck types for arrays. (Of course one could then argue we should *only* define ``__mul__`` to be matrix multiplication, but then we'd have the same problem with elementwise multiplication.) There have been several pushes to remove ``numpy.matrix`` entirely; the only counter-arguments have come from educators who find that its problems are outweighed by the need to provide a simple and clear mapping between mathematical notation and code for novices (see `Transparent syntax is especially crucial for non-expert programmers`_). But, of course, starting out newbies with a dispreferred syntax and then expecting them to transition later causes its own problems. The two-type solution is worse than the disease. **Add lots of new operators, or add a new generic syntax for defining infix operators:** In addition to being generally un-Pythonic and repeatedly rejected by BDFL fiat, this would be using a sledgehammer to smash a fly. The scientific python community has consensus that adding one operator for matrix multiplication is enough to fix the one otherwise unfixable pain point. (In retrospect, we all think PEP 225 was a bad idea too -- or at least far more complex than it needed to be.) **Add a new @ (or whatever) operator that has some other meaning in general Python, and then overload it in numeric code:** This was the approach taken by PEP 211, which proposed defining ``@`` to be the equivalent of ``itertools.product``. The problem with this is that when taken on its own terms, adding an infix operator for ``itertools.product`` is just silly. (During discussions of this PEP, a similar suggestion was made to define ``@`` as a general purpose function composition operator, and this suffers from the same problem; ``functools.compose`` isn't even useful enough to exist.) Matrix multiplication has a uniquely strong rationale for inclusion as an infix operator. There almost certainly don't exist any other binary operations that will ever justify adding any other infix operators to Python. **Add a .dot method to array types so as to allow "pseudo-infix" A.dot(B) syntax:** This has been in numpy for some years, and in many cases it's better than dot(A, B). But it's still much less readable than real infix notation, and in particular still suffers from an extreme overabundance of parentheses. See `Why should matrix multiplication be infix?`_ above. **Use a 'with' block to toggle the meaning of * within a single code block**: E.g., numpy could define a special context object so that we'd have:: c = a * b # element-wise multiplication with numpy.mul_as_dot: c = a * b # matrix multiplication However, this has two serious problems: first, it requires that every array-like type's ``__mul__`` method know how to check some global state (``numpy.mul_is_currently_dot`` or whatever). This is fine if ``a`` and ``b`` are numpy objects, but the world contains many non-numpy array-like objects. So this either requires non-local coupling -- every numpy competitor library has to import numpy and then check ``numpy.mul_is_currently_dot`` on every operation -- or else it breaks duck-typing, with the above code doing radically different things depending on whether ``a`` and ``b`` are numpy objects or some other sort of object. Second, and worse, ``with`` blocks are dynamically scoped, not lexically scoped; i.e., any function that gets called inside the ``with`` block will suddenly find itself executing inside the mul_as_dot world, and crash and burn horribly -- if you're lucky. So this is a construct that could only be used safely in rather limited cases (no function calls), and which would make it very easy to shoot yourself in the foot without warning. **Use a language preprocessor that adds extra numerically-oriented operators and perhaps other syntax:** (As per recent BDFL suggestion: [#preprocessor]_) This suggestion seems based on the idea that numerical code needs a wide variety of syntax additions. In fact, given ``@``, most numerical users don't need any other operators or syntax; it solves the one really painful problem that cannot be solved by other means, and that causes painful reverberations through the larger ecosystem. Defining a new language (presumably with its own parser which would have to be kept in sync with Python's, etc.), just to support a single binary operator, is neither practical nor desireable. In the numerical context, Python's competition is special-purpose numerical languages (Matlab, R, IDL, etc.). Compared to these, Python's killer feature is exactly that one can mix specialized numerical code with code for XML parsing, web page generation, database access, network programming, GUI libraries, and so forth, and we also gain major benefits from the huge variety of tutorials, reference material, introductory classes, etc., which use Python. Fragmenting "numerical Python" from "real Python" would be a major source of confusion. A major motivation for this PEP is to *reduce* fragmentation. Having to set up a preprocessor would be an especially prohibitive complication for unsophisticated users. And we use Python because we like Python! We don't want almost-but-not-quite-Python. **Use overloading hacks to define a "new infix operator" like *dot*, as in a well-known Python recipe:** (See: [#infix-hack]_) Beautiful is better than ugly. This is... not beautiful. And not Pythonic. And especially unfriendly to beginners, who are just trying to wrap their heads around the idea that there's a coherent underlying system behind these magic incantations that they're learning, when along comes an evil hack like this that violates that system, creates bizarre error messages when accidentally misused, and whose underlying mechanisms can't be understood without deep knowledge of how object oriented systems work. We've considered promoting this as a general solution, and perhaps if the PEP is rejected we'll revisit this option, but so far the numeric community has mostly elected to leave this one on the shelf. References ========== .. [#preprocessor] From a comment by GvR on a G+ post by GvR; the comment itself does not seem to be directly linkable: https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u .. [#infix-hack] http://code.activestate.com/recipes/384122-infix-operators/ http://www.sagemath.org/doc/reference/misc/sage/misc/decorators.html#sage.misc.decorators.infix_operator .. [#scipy-conf] http://conference.scipy.org/past.html .. [#pydata-conf] http://pydata.org/events/ .. [#lht] In this formula, :math:`\beta` is a vector or matrix of regression coefficients, :math:`V` is the estimated variance/covariance matrix for these coefficients, and we want to test the null hypothesis that :math:`H\beta = r`; a large :math:`S` then indicates that this hypothesis is unlikely to be true. For example, in an analysis of human height, the vector :math:`\beta` might contain one value which was the the average height of the measured men, and another value which was the average height of the measured women, and then setting :math:`H = [1, -1], r = 0` would let us test whether men and women are the same height on average. Compare to eq. 2.139 in http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode17.html Example code is adapted from https://github.com/rerpy/rerpy/blob/0d274f85e14c3b1625acb22aed1efa85d122ecb7/rerpy/incremental_ls.py#L202 .. [#pycon-tutorials] Out of the 36 tutorials scheduled for PyCon 2014 (https://us.pycon.org/2014/schedule/tutorials/), we guess that the 8 below will almost certainly deal with matrices: * Dynamics and control with Python * Exploring machine learning with Scikit-learn * How to formulate a (science) problem and analyze it using Python code * Diving deeper into Machine Learning with Scikit-learn * Data Wrangling for Kaggle Data Science Competitions ? An etude * Hands-on with Pydata: how to build a minimal recommendation engine. * Python for Social Scientists * Bayesian statistics made simple In addition, the following tutorials could easily involve matrices: * Introduction to game programming * mrjob: Snakes on a Hadoop *("We'll introduce some data science concepts, such as user-user similarity, and show how to calculate these metrics...")* * Mining Social Web APIs with IPython Notebook * Beyond Defaults: Creating Polished Visualizations Using Matplotlib This gives an estimated range of 8 to 12 / 36 = 22% to 33% of tutorials dealing with matrices; saying ~20% then gives us some wiggle room in case our estimates are high. .. [#sloc-details] SLOCs were defined as physical lines which contain at least one token that is not a COMMENT, NEWLINE, ENCODING, INDENT, or DEDENT. Counts were made by using ``tokenize`` module from Python 3.2.3 to examine the tokens in all files ending ``.py`` underneath some directory. Only tokens which occur at least once in the source trees are included in the table. The counting script will be available as an auxiliary file once this PEP is submitted; until then, it can be found here: https://gist.github.com/njsmith/9157645 Matrix multiply counts were estimated by counting how often certain tokens which are used as matrix multiply function names occurred in each package. In principle this could create false positives, but as far as I know the counts are exact; it's unlikely that anyone is using ``dot`` as a variable name when it's also the name of one of the most widely-used numpy functions. All counts were made using the latest development version of each project as of 21 Feb 2014. 'stdlib' is the contents of the Lib/ directory in commit d6aa3fa646e2 to the cpython hg repository, and treats the following tokens as indicating matrix multiply: n/a. 'scikit-learn' is the contents of the sklearn/ directory in commit 69b71623273ccfc1181ea83d8fb9e05ae96f57c7 to the scikit-learn repository (https://github.com/scikit-learn/scikit-learn), and treats the following tokens as indicating matrix multiply: ``dot``, ``fast_dot``, ``safe_sparse_dot``. 'nipy' is the contents of the nipy/ directory in commit 5419911e99546401b5a13bd8ccc3ad97f0d31037 to the nipy repository (https://github.com/nipy/nipy/), and treats the following tokens as indicating matrix multiply: ``dot``. .. [#blas-fork] BLAS libraries have a habit of secretly spawning threads, even when used from single-threaded programs. And threads play very poorly with ``fork()``; the usual symptom is that attempting to perform linear algebra in a child process causes an immediate deadlock. .. [#threads-2008] http://fperez.org/py4science/numpy-pep225/numpy-pep225.html .. [#broadcasting] http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html .. [#matmul-other-langs] http://mail.scipy.org/pipermail/scipy-user/2014-February/035499.html .. [#github-details] Counts were produced by manually entering the string ``"import foo"`` or ``"from foo import"`` (with quotes) into the Github code search page, e.g.: https://github.com/search?q=%22import+numpy%22&ref=simplesearch&type=Code on 2014-04-10 at ~21:00 UTC. The reported values are the numbers given in the "Languages" box on the lower-left corner, next to "Python". This also causes some undercounting (e.g., leaving out Cython code, and possibly one should also count HTML docs and so forth), but these effects are negligible (e.g., only ~1% of numpy usage appears to occur in Cython code, and probably even less for the other modules listed). The use of this box is crucial, however, because these counts appear to be stable, while the "overall" counts listed at the top of the page ("We've found ___ code results") are highly variable even for a single search -- simply reloading the page can cause this number to vary by a factor of 2 (!!). (They do seem to settle down if one reloads the page repeatedly, but nonetheless this is spooky enough that it seemed better to avoid these numbers.) These numbers should of course be taken with multiple grains of salt; it's not clear how representative Github is of Python code in general, and limitations of the search tool make it impossible to get precise counts. AFAIK this is the best data set currently available, but it'd be nice if it were better. In particular: * Lines like ``import sys, os`` will only be counted in the ``sys`` row. * A file containing both ``import X`` and ``from X import`` will be counted twice * Imports of the form ``from X.foo import ...`` are missed. We could catch these by instead searching for "from X", but this is a common phrase in English prose, so we'd end up with false positives from comments, strings, etc. For many of the modules considered this shouldn't matter too much -- for example, the stdlib modules have flat namespaces -- but it might especially lead to undercounting of django, scipy, and twisted. Also, it's possible there exist other non-stdlib modules we didn't think to test that are even more-imported than numpy -- though we tried quite a few of the obvious suspects. If you find one, let us know! The modules tested here were chosen based on a combination of intuition and the top-100 list at pypi-ranking.info. Fortunately, it doesn't really matter if it turns out that numpy is, say, merely the *third* most-imported non-stdlib module, since the point is just that numeric programming is a common and mainstream activity. Finally, we should point out the obvious: whether a package is import**ed** is rather different from whether it's import**ant**. No-one's claiming numpy is "the most important package" or anything like that. Certainly more packages depend on distutils, e.g., then depend on numpy -- and far fewer source files import distutils than import numpy. But this is fine for our present purposes. Most source files don't import distutils because most source files don't care how they're distributed, so long as they are; these source files thus don't care about details of how distutils' API works. This PEP is in some sense about changing how numpy's and related packages' APIs work, so the relevant metric is to look at source files that are choosing to directly interact with that API, which is sort of like what we get by looking at import statements. .. [#hugunin] The first such proposal occurs in Jim Hugunin's very first email to the matrix SIG in 1995, which lays out the first draft of what became Numeric. He suggests using ``*`` for elementwise multiplication, and ``%`` for matrix multiplication: https://mail.python.org/pipermail/matrix-sig/1995-August/000002.html Copyright ========= This document has been placed in the public domain. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From njs at pobox.com Fri Mar 14 03:08:17 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 02:08:17 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote: > [...] Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really, I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't too distracting! -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From ericsnowcurrently at gmail.com Fri Mar 14 03:08:35 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 13 Mar 2014 20:08:35 -0600 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <lfrt3c$62p$1@ger.gmane.org> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> <lfrt3c$62p$1@ger.gmane.org> Message-ID: <CALFfu7A16Fo9KeVTc=RBqM2OhA9tSj9Hx3ngviVtids5=npFNg@mail.gmail.com> On Mar 13, 2014 3:18 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote: > > Le 13/03/2014 08:43, Eric Snow a ?crit : > >> 1. the existing sys module be renamed to _sys (and the stdlib be >> changed accordingly where appropriate); >> 2. a pure Python sys module be added to the stdlib that wraps the existing API; > > > I'm not sure what that brings until the need arises. It means no one needs to change 'import sys' to 'import _sys' in their code. > However, making it a pure Python sys module means it won't be immediately available at startup like sys currently is (i.e. it's a built-in module). I don't know if there are any scenarios where it may matter. As I noted, we would need to update some spots in the stdlib to import _sys rather than sys. > > >> 2.5. the new module should use the replace-itself-in-sys.modules >> technique to provide a module subclass that supports descriptors. > > > I don't think any module should be specific in that regard. Special cases generally make the language more complicated to learn. Also, you haven't explained the rationale. As others have noted, there isn't much point to making any of these changes until we have some need that they satisfy. I anticipate having that need in the mid-term and wanted to see if the proposed approach is palatable. -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/db60724e/attachment.html> From ericsnowcurrently at gmail.com Fri Mar 14 03:53:52 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Thu, 13 Mar 2014 20:53:52 -0600 Subject: [Python-ideas] a standard approach to module-replace-itself-in-sys.modules In-Reply-To: <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com> References: <CALFfu7Cs+Vyge8t-MQUxsgvRmRjEz-Vee3y4Cfp1W+JvL-udaw@mail.gmail.com> <CAP1=2W7ysdV+JmCi6OcrLbLaazK6ye_Z8BqXOGQWaLoa4Zw_wg@mail.gmail.com> Message-ID: <CALFfu7DSO9cQ=1k0yXpEjnnd3aq-1HjSDRovpXuV9w+AT1uBqA@mail.gmail.com> On Mar 13, 2014 6:54 AM, "Brett Cannon" <brett at python.org> wrote: > Do we actually want to promote this practice by codifying support for it in the stdlib? I guess I was expecting that question from you. :) I figure if the practice is officially supported then we should help folks doing it to not shoot themselves in the foot. As to how much such a tool will promote the practice, I'm not sure that it will, particularly if the helper is located out of the way. However I'm sure you've watched similar additions unfold and have a better sense of that than I do. > If you're doing this to get decorators for module attributes then you should say this is for that goal and the solution is getting the module swapping hack to be easier to use (and might call for a better solution). Otherwise this proposal feels like it's trying to codify behaviour that is rarely necessary. Agreed. Furthermore I doubt I'll push on this idea without a strong need first. More than anything I wanted to share what seemed like an elegant solution to the "problem", even if we don't need to solve it right away. If I get more serious about it I'll hit up folks that are using the current hack (Django?) and see if it improves anything for them. -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140313/0fcb4c2b/attachment.html> From abarnert at yahoo.com Fri Mar 14 05:03:57 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 13 Mar 2014 21:03:57 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> Message-ID: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> This (or various related ideas) actually has come up somewhat regularly. I think the reason it hasn't gone anywhere is that there was never enough input from the numerical community; someone says, "I'll bet this would be useful for numerics," and someone else says "Then why haven't the numarray/numeric/numpy guys asked for it since 2000?", and nobody has an answer to that. This obviously is the answer to that. One thing that always comes up is a suggestion for using Unicode. There are obvious downsides--the Unicode multiplication character isn't easy to type; even if Python and major code editors are fully Unicode friendly, code often has to go through channels that may not be; etc. But it's worth asking whether the numeric community has considered this and rejected it for these reasons, or for other reasons, or if they'd be happy with it but just didn't think Python was ready for it, or whatever. Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least. On Mar 13, 2014, at 19:08, Nathaniel Smith <njs at pobox.com> wrote: > On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote: >> [...] > > Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really, > I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't > too distracting! > > -n > > -- > Nathaniel J. Smith > Postdoctoral researcher - Informatics - University of Edinburgh > http://vorpus.org > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From alexander.belopolsky at gmail.com Fri Mar 14 05:18:48 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 14 Mar 2014 00:18:48 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> Message-ID: <CAP7h-xYNFCVCMfmZoCtUneV8XWTyuaNDE7K+oEgOf-EV+N4+rQ@mail.gmail.com> On Fri, Mar 14, 2014 at 12:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote: > One thing that always comes up is a suggestion for using Unicode. There > are obvious downsides--the Unicode multiplication character isn't easy to > type; even if Python and major code editors are fully Unicode friendly, > code often has to go through channels that may not be; etc. But it's worth > asking whether the numeric community has considered this and rejected it > for these reasons, or for other reasons, or if they'd be happy with it but > just didn't think Python was ready for it, or whatever. It did. Some 60 years ago [1]. Mostly rejected by now [2]. [1] http://en.wikipedia.org/wiki/APL_(programming_language) [2] http://www.jsoftware.com/start.htm -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/9f3cd458/attachment-0001.html> From tjreedy at udel.edu Fri Mar 14 05:33:35 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 14 Mar 2014 00:33:35 -0400 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <lfu0r5$s4i$1@ger.gmane.org> On 3/13/2014 9:59 PM, Nathaniel Smith wrote: > I'd have rejected these too! So I thought, maybe we should try the > radical tactic of writing down what we actually want, carefully > explaining why we want it, and then asking for it. Great idea ;-). Until someone comes up with reasons to reject this, I am +1. -- Terry Jan Reedy From njs at pobox.com Fri Mar 14 06:05:42 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 05:05:42 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> Message-ID: <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com> On Fri, Mar 14, 2014 at 4:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote: > This (or various related ideas) actually has come up somewhat regularly. I think the reason it hasn't gone anywhere is that there was never enough input from the numerical community; someone says, "I'll bet this would be useful for numerics," and someone else says "Then why haven't the numarray/numeric/numpy guys asked for it since 2000?", and nobody has an answer to that. This obviously is the answer to that. The numeric community has many talents, but crossing the cultural divide with upstream is not really a speciality... > One thing that always comes up is a suggestion for using Unicode. There are obvious downsides--the Unicode multiplication character isn't easy to type; even if Python and major code editors are fully Unicode friendly, code often has to go through channels that may not be; etc. But it's worth asking whether the numeric community has considered this and rejected it for these reasons, or for other reasons, or if they'd be happy with it but just didn't think Python was ready for it, or whatever. We don't really have a strong opinion on which character is used. It is nice if it's easy to type, though -- esp. because in many ways it's already beginners who suffer the brunt of the elmul/matmul issues, and beginners are exactly the people for whom figuring out how to type some stupid character is a prohibitive speed bump. (Scientific python has a very large ongoing stream of newbie programmers.) And I don't know that any of the Unicode characters are actually better. In real math the two operations are distinguished by context, so there's no existing character with the right meaning that we can steal. In some ways having an unusual character for this is better, because it indicates a special-case operation. If the two multiplication characters are * and ?, then how do you remember which is which? But if they're * and @, then it's easy to remember that * is the general-use one, and @ is the special non-commutative matrix multiplication one. It's not like new operators are being added to Python every week and we need to start scraping the bottom of the barrel for new characters. ASCII gets the job done, has some minor upsides, and no real downsides. > Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least. I'm not an expert on Haskell and C++ matrix libraries -- perhaps someone else will speak up -- but Haskell's extensible infix system and static typing do seem like they put them in a pretty different design space. And I'm skeptical that there exist Haskell libraries with anything like numpy's maturity, though I'll be delighted to be proven wrong. Eigen is maybe the most popular C++ matrix/array library right now, and AFAICT from their docs they use a system where there's one "array" type that *only* supports elementwise multiplication, and one "matrix" type that *only* supports matrix multiplication, and if you want both operations you have to cast back and forth. This seems a bit annoying to me, but not as deadly as it is in Python -- in C++ the static type system means they can at least fob off the work of keeping track of which variables are which type, and doing conversions at function boundaries, onto the compiler. In Python these conversions have to happen by hand, and the two-type solution is just not workable. In general, I doubt there's huge amounts of experience to steal from other languages, because Python is in a unique place: AFAICT it's the only general purpose language that has ever made serious inroads against the specialized numeric languages (Matlab, R, GAUSS, IDL, etc.) on their home territory. So we're breaking new ground here. (All those languages do have separate infix operators for elementwise and matrix multiplication; they all involve horrible names like ".*" or "%*%" and all kinds of weird inconsistencies.) -n > On Mar 13, 2014, at 19:08, Nathaniel Smith <njs at pobox.com> wrote: > >> On Fri, Mar 14, 2014 at 1:59 AM, Nathaniel Smith <njs at pobox.com> wrote: >>> [...] >> >> Hmm, not sure how that "Fwd:" snuck onto the subject line. (Or really, >> I do know, but am embarrassed to say.) Oh well, sorry, hope it isn't >> too distracting! >> >> -n >> >> -- >> Nathaniel J. Smith >> Postdoctoral researcher - Informatics - University of Edinburgh >> http://vorpus.org >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From taleinat at gmail.com Fri Mar 14 07:42:03 2014 From: taleinat at gmail.com (Tal Einat) Date: Fri, 14 Mar 2014 08:42:03 +0200 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <CALWZvp4x2sQgU=eLuX5T8gMsGeisOUygbwa0pxWW7nnOr7JwTQ@mail.gmail.com> On Fri, Mar 14, 2014 at 3:59 AM, Nathaniel Smith <njs at pobox.com> wrote: > > PEP: XXXX > Title: Dedicated infix operators for matrix multiplication and matrix power A wholehearted +1! - Tal Einat From storchaka at gmail.com Fri Mar 14 07:55:04 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 14 Mar 2014 08:55:04 +0200 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <20140313160353.745d9659@anarchist.wooz.org> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org> <20140313160353.745d9659@anarchist.wooz.org> Message-ID: <lfu92v$8pb$1@ger.gmane.org> 13.03.14 22:03, Barry Warsaw ???????(??): > On Mar 13, 2014, at 02:04 PM, Serhiy Storchaka wrote: >> Many sys attributes (argv, displayhook, excepthook, modules, path, >> path_hooks, path_importer_cache, ps1, ps2, stderr, stdin, stdout, >> tracebacklimit) are used implicit in builtins and in interpreter core. Even >> import system doesn't work without the sys module. So it should be >> initialized very early and finalized very late. > > If the C API PySys_* functions were updated to use _sys instead, hopefully > most stuff in the core would still work, right? Perhaps. If sys attributes will be writable properties which updates _sys. From abarnert at yahoo.com Fri Mar 14 09:33:09 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 14 Mar 2014 01:33:09 -0700 (PDT) Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAPJVwBkNL7-cD4Mdu9C2-zT42tPyqnoRWt9f8yH4mxbK38Zcmw@mail.gmail.com> <AC3D84B1-07E2-4270-AD25-7D92025C2A04@yahoo.com> <CAPJVwB=sCdGbjj5RhW86jVkKucENmk0xS7gPoFzNkFhS=2E-PA@mail.gmail.com> Message-ID: <1394785989.3468.YahooMailNeo@web181002.mail.ne1.yahoo.com> From: Nathaniel Smith <njs at pobox.com> Sent: Thursday, March 13, 2014 10:05 PM >On Fri, Mar 14, 2014 at 4:03 AM, Andrew Barnert <abarnert at yahoo.com> wrote: [snip, here and elsewhere] >The numeric community has many talents, but crossing the cultural >divide with upstream is not really a speciality... Well, you've fixed that here. >> One thing that always comes up is a suggestion for using Unicode. > >We don't really have a strong opinion on which character is used. It >is nice if it's easy to type, though -- esp. because in many ways it's >already beginners who suffer the brunt of the elmul/matmul issues, and >beginners are exactly the people for whom figuring out how to type >some stupid character is a prohibitive speed bump. (Scientific python >has a very large ongoing stream of newbie programmers.) And I don't >know that any of the Unicode characters are actually better. In real >math the two operations are distinguished by context, so there's no >existing character with the right meaning that we can steal. In some >ways having an unusual character for this is better, because it >indicates a special-case operation. If the two multiplication >characters are * and ?, then how do you remember which is which? But >if they're * and @, then it's easy to remember that * is the >general-use one, and @ is the special non-commutative matrix >multiplication one. Honestly, that sounds like a strong negative opinion, with good motivations, not no opinion, The argument for Unicode is generally "readability is more important than writability," all things being equal. The traditional counter to that is that all things aren't equal, as it's still way too hard for novices to write Unicode characters. But adding the additional rationale that there is no obvious Unicode character for this operator certainly strengthens the case for @. >It's not like new operators are being added to Python every week and >we need to start scraping the bottom of the barrel for new characters. >ASCII gets the job done, has some minor upsides, and no real >downsides. Well, we have just about two ASCII characters left, @ and ?,?so we really are scraping the bottom of the barrel. But you made a good argument why this is worthy of one of those two characters. >> Also, how do other general purpose programming languages solve this? Surely people do matrix math in Haskell and C++, at least? Do they just use separate types for matrices and arrays because they don't have to worry about duck typing (being statically-typed languages)? Do they just avoid mixing libraries together to avoid the problem? Or have people attempted to reuse % or other operators? Or (doesn't apply to C++, but does to Haskell) do they use spelled-out infix functions like `mmul` instead of trying to come up with symbolic operators? A lot of those answers wouldn't tell us anything more than "their experience isn't relevant to Python", but it would be nice to know that at least. > >I'm not an expert on Haskell and C++ matrix libraries -- perhaps >someone else will speak up -- but Haskell's extensible infix system >and static typing do seem like they put them in a pretty different >design space. And I'm skeptical that there exist Haskell libraries >with anything like numpy's maturity, though I'll be delighted to be >proven wrong. From what I can tell, it's more common to implement matrix libraries in Haskell than to use them. Probably because Haskell is more of a research and/or teaching language than a language you'd expect, say, physicists to use. But I was hoping someone would actually know that, rather than just guess? >Eigen is maybe the most popular C++ matrix/array library right now, >and AFAICT from their docs they use a system where there's one "array" >type that *only* supports elementwise multiplication, and one "matrix" >type that *only* supports matrix multiplication, and if you want both >operations you have to cast back and forth. This seems a bit annoying >to me, but not as deadly as it is in Python -- in C++ the static type >system means they can at least fob off the work of keeping track of >which variables are which type, and doing conversions at function >boundaries, onto the compiler. In Python these conversions have to >happen by hand, and the two-type solution is just not workable. Yes, that's why I suspected that C++ experience would not be applicable to Python. Assigning a matrix value to an array variable, or passing it to an array function, can be an implicit (but still obvious) conversion. And the way they'd solve the same problems Python solves with duck typing would be would generic functions, which is pretty different from duck-typed functions. And I'd expect that most other static languages are similar. But it might be worth adding to the PEP if you actually know the answer rather than just suspecting. >In general, I doubt there's huge amounts of experience to steal from >other languages, because Python is in a unique place: AFAICT it's the >only general purpose language that has ever made serious inroads >against the specialized numeric languages (Matlab, R, GAUSS, IDL, >etc.) on their home territory. So we're breaking new ground here. (All >those languages do have separate infix operators for elementwise and >matrix multiplication; they all involve horrible names like ".*" or >"%*%" and all kinds of weird inconsistencies.) You could be right. Most of the core low-level libraries are in C, and C++ programmers have a habit of just using C libraries with their C APIs rather than wrapping them in generic/OO C++ APIs? (Honestly, I think automatic C compatibility is one of the biggest problems with C++, not its biggest strength. In Python, it's easy to wrap a C library, and you have to do it, so you do it and end up with a great API; in C++, it's easy to wrap a C library, but you don't have to, so you end up using it C-style and losing all the benefits of C++.) From mal at egenix.com Fri Mar 14 11:16:34 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 14 Mar 2014 11:16:34 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <5322D702.6040907@egenix.com> On 14.03.2014 02:59, Nathaniel Smith wrote: > PEP: XXXX > Title: Dedicated infix operators for matrix multiplication and matrix power > > ... > > Specification > ============= > > Two new binary operators are added to the Python language, together > with corresponding in-place versions: > > ======= ========================= =============================== > Op Precedence/associativity Methods > ======= ========================= =============================== > ``@`` Same as ``*`` ``__matmul__``, ``__rmatmul__`` > ``@@`` Same as ``**`` ``__matpow__``, ``__rmatpow__`` > ``@=`` n/a ``__imatmul__`` > ``@@=`` n/a ``__imatpow__`` > ======= ========================= =============================== > > ... > > When working with n-dimensional arrays, there are two different ways > we might want to define multiplication. One is elementwise > multiplication:: > > [[1, 2], [[11, 12], [[1 * 11, 2 * 12], > [3, 4]] x [13, 14]] = [3 * 13, 4 * 14]] > > and the other is `matrix multiplication`_: > > .. _matrix multiplication: https://en.wikipedia.org/wiki/Matrix_multiplication I have some questions: 1. Since in math, the operator is usually spelt "?" (the center dot, or "." but that's already reserved for methods and attributes in Python), why not try to use that instead of "@" (which in Python already identifies decorators) ? 2. The PEP should include a section on how other programming languages solve this, i.e. what syntax they use for matrix multiplications. 3. Since matrix multiplication is only one type of product you find in math, albeit a very frequently used one, how would those other products fit into the picture ? Would then have to use methods again ? E.g. think of cross product, inner product, outer/tensor product. 4. Another very common operation needed in vector/matrix calculation is transposition. This is usually written as superscript "T" or "t" ("?" in Unicode). Wouldn't this operator be needed as well, to make the picture complete ? OTOH, we currently don't have postfix operators in Python, so I guess writing this as A.transpose() comes close enough ;-) Now since this is all about syntactic sugar, we also need to look at some code examples: I == A @@ -1 @ A vs. I == A ?? -1 ? A vs. I == A.inverse().dot(A) (A @ B).transpose() == A.transpose() @ B.transpose() vs. (A ? B).transpose() == A.transpose() ? B.transpose() vs. A.dot(B).transpose() == A.transpose().dot(B.transpose()) c = A @ v vs. c = A ? v vs. c = A.dot(v) Hmm, even though I'd love to see matrix operators in Python, I don't think they really add clarity to the syntax of matrix calculations - a bit disappointing, I must say :-( -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 14 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 15 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 26 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Fri Mar 14 11:24:26 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 11:24:26 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <20140314112426.7854e9f6@fsol> Hi, On Fri, 14 Mar 2014 01:59:14 +0000 Nathaniel Smith <njs at pobox.com> wrote: > > PEP 211, which instead adds an operator for itertools.product, aka, > "maybe we can sneak matrix multiply past Guido in some sort of... > large, wooden rabbit..." A chocolate rabbit might be tastier for some core developers ;-) Ok, so first I'm gonna state upfront that I don't have any personal interest in this proposal (i.e. I don't do matrix multiplications often enough for it to matter to me). That said: > This PEP proposes two new binary operators dedicated to matrix > multiplication and matrix power, spelled ``@`` and ``@@`` > respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) That sounds like a strange choice to me. There's nothing in "@" or "@@" that suggests "matrices" or "multiplication". Also, are there precedents in other languages? AFAIK, other languages sometimes use ".*" for matrix multiplication (as opposed to element-wise multiplication). Why not re-use that convention? Would that lead to parsing difficulties? That said, I'm rather sympathetic to the general idea, and if "@" is the least contentious option, then I'm ok with it too ("@@" doesn't sound like a good idea at all, though). Thanks for writing this PEP! Regards Antoine. From solipsis at pitrou.net Fri Mar 14 11:29:31 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 11:29:31 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> Message-ID: <20140314112931.4c38de75@fsol> On Fri, 14 Mar 2014 11:16:34 +0100 "M.-A. Lemburg" <mal at egenix.com> wrote: > > 4. Another very common operation needed in vector/matrix calculation > is transposition. This is usually written as superscript "T" or "t" > ("?" in Unicode). Wouldn't this operator be needed as well, to > make the picture complete ? OTOH, we currently don't have postfix > operators in Python, so I guess writing this as A.transpose() > comes close enough ;-) Or simply implement __invert__, so you can write it "~ A". (__invert__ doesn't really invert a number, it takes the bitwise complement :-)) > Hmm, even though I'd love to see matrix operators in Python, > I don't think they really add clarity to the syntax of matrix > calculations - a bit disappointing, I must say :-( I think they do when you have several of these operations combined in a single formula, with parentheses and the like. Regards Antoine. From robert.kern at gmail.com Fri Mar 14 12:25:27 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 11:25:27 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <5322D702.6040907@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> Message-ID: <lfuouo$v25$1@ger.gmane.org> On 2014-03-14 10:16, M.-A. Lemburg wrote: > I have some questions: > > 1. Since in math, the operator is usually spelt "?" (the center dot, > or "." but that's already reserved for methods and attributes in > Python), why not try to use that instead of "@" (which in Python > already identifies decorators) ? I think the current feeling of the Python core team is against including non-ASCII characters in the language's keywords or operators. Even if that were not so, I would still recommend against it because it would be quite difficult to type. I don't know off-hand the key combination to do it on my native system, and it would change from system to system. > 2. The PEP should include a section on how other programming languages > solve this, i.e. what syntax they use for matrix multiplications. > > 3. Since matrix multiplication is only one type of product you find > in math, albeit a very frequently used one, how would those other > products fit into the picture ? Would then have to use methods > again ? E.g. think of cross product, inner product, outer/tensor > product. Our experience is that these have come up much less regularly than matrix multiplication. The two products in common use in our code are the Hadamard product (elementwise multiplication, currently assigned to * in numpy) and matrix multiplication (currently done with the function numpy.dot()). > 4. Another very common operation needed in vector/matrix calculation > is transposition. This is usually written as superscript "T" or "t" > ("?" in Unicode). Wouldn't this operator be needed as well, to > make the picture complete ? OTOH, we currently don't have postfix > operators in Python, so I guess writing this as A.transpose() > comes close enough ;-) Indeed. Numpy already uses a .T property for this. > Now since this is all about syntactic sugar, we also need to look at > some code examples: > > > I == A @@ -1 @ A > vs. > I == A ?? -1 ? A > vs. > I == A.inverse().dot(A) > > > (A @ B).transpose() == A.transpose() @ B.transpose() > vs. > (A ? B).transpose() == A.transpose() ? B.transpose() > vs. > A.dot(B).transpose() == A.transpose().dot(B.transpose()) (A @ B).T == B.T @ A.T (A ? B).T == B.T ? A.T A.dot(B).T == B.T.dot(A.T) (FWIW, I didn't notice the math error until I wrote out the @ version.) > c = A @ v > vs. > c = A ? v > vs. > c = A.dot(v) > > > Hmm, even though I'd love to see matrix operators in Python, > I don't think they really add clarity to the syntax of matrix > calculations - a bit disappointing, I must say :-( Some more from real code: RSR = R.dot(var_beta.dot(R.T)) RSR = R @ var_beta @ R.T xx_inv.dot(xeps.dot(xx_inv)) xx_inv @ xeps @ xx_inv dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T) dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T) dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T)) (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T) torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From mal at egenix.com Fri Mar 14 12:48:13 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 14 Mar 2014 12:48:13 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfuouo$v25$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> Message-ID: <5322EC7D.1020908@egenix.com> On 14.03.2014 12:25, Robert Kern wrote: > On 2014-03-14 10:16, M.-A. Lemburg wrote: >> 4. Another very common operation needed in vector/matrix calculation >> is transposition. This is usually written as superscript "T" or "t" >> ("?" in Unicode). Wouldn't this operator be needed as well, to >> make the picture complete ? OTOH, we currently don't have postfix >> operators in Python, so I guess writing this as A.transpose() >> comes close enough ;-) > > Indeed. Numpy already uses a .T property for this. Ah, good trick :-) >> Now since this is all about syntactic sugar, we also need to look at >> some code examples: >> >> >> I == A @@ -1 @ A >> vs. >> I == A ?? -1 ? A >> vs. >> I == A.inverse().dot(A) >> >> >> (A @ B).transpose() == B.transpose() @ A.transpose() >> vs. >> (A ? B).transpose() == B.transpose() ? A.transpose() >> vs. >> A.dot(B).transpose() == B.transpose().dot(A.transpose()) > > (A @ B).T == B.T @ A.T > (A ? B).T == B.T ? A.T > A.dot(B).T == B.T.dot(A.T) > > (FWIW, I didn't notice the math error until I wrote out the @ version.) Thanks; I should have proofread the email before hitting the send button. I've correct the quoted version above to have the comparisons return True for all A and B instead of just for a select few :-) >> c = A @ v >> vs. >> c = A ? v >> vs. >> c = A.dot(v) >> >> >> Hmm, even though I'd love to see matrix operators in Python, >> I don't think they really add clarity to the syntax of matrix >> calculations - a bit disappointing, I must say :-( > > Some more from real code: > > RSR = R.dot(var_beta.dot(R.T)) > RSR = R @ var_beta @ R.T > > xx_inv.dot(xeps.dot(xx_inv)) > xx_inv @ xeps @ xx_inv > > dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T) > dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T) > > dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T)) > (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T) > > torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) > (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] This doesn't look very readable to me - the operator saves you a few parens in some situations, but as in the last example, it can also require adding new ones. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 14 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 15 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 26 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Fri Mar 14 12:54:47 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 12:54:47 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> Message-ID: <20140314125447.3486c23a@fsol> On Fri, 14 Mar 2014 12:48:13 +0100 "M.-A. Lemburg" <mal at egenix.com> wrote: > > > > Some more from real code: > > > > RSR = R.dot(var_beta.dot(R.T)) > > RSR = R @ var_beta @ R.T > > > > xx_inv.dot(xeps.dot(xx_inv)) > > xx_inv @ xeps @ xx_inv > > > > dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T) > > dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T) > > > > dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T)) > > (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T) > > > > torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) > > (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] > > This doesn't look very readable to me - the operator saves you > a few parens in some situations, but as in the last example, it can > also require adding new ones. The parentheses mirror those necessary in the equivalent mathematical formula, though, so they are "natural" in a sense. I do find the "@" examples much more readable myself - except that I don't understand what they are about, of course :-) Regards Antoine. From steve at pearwood.info Fri Mar 14 13:32:39 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Mar 2014 23:32:39 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314112426.7854e9f6@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> Message-ID: <20140314123239.GP12595@ando> On Fri, Mar 14, 2014 at 11:24:26AM +0100, Antoine Pitrou wrote: > > This PEP proposes two new binary operators dedicated to matrix > > multiplication and matrix power, spelled ``@`` and ``@@`` > > respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) > > That sounds like a strange choice to me. There's nothing in "@" or "@@" > that suggests "matrices" or "multiplication". The PEP even gives a mnemonic for it: @ is for mATrix multiplication. There's nothing in @ that suggests "decorator", nor is there anything about * that suggests scalar multiplication. Out of the common symbols used in Python, & is possibly the only operator symbol with a non-arbitrary connection to its meaning. & was originally a ligature of "et", the Latin word for "and". Everything else is pretty much arbitrary, and it's only habit and long use that makes them seem normal. > AFAIK, other languages sometimes use ".*" for matrix multiplication (as > opposed to element-wise multiplication). Why not re-use that > convention? Would that lead to parsing difficulties? "Syntax should not look like grit on Tim Peters' monitor." .* is, in my opinion, pretty ugly, and leads to even ugly extensions: a .* b # matrix multiplication a .*= b # in place matrix multiplication a .** b # matrix exponentiation a .**= b # in-place matrix exponentiation And it suffers from exactly the same lack of connection to matrix multiplication as @ does: there is nothing about . that spells "matrix". It also has the problem that it could cause confusion with vector dot product. Read out A .* B aloud and you get something that sounds like it might mean dot product, "A dot times B". > That said, I'm rather sympathetic to the general idea, and if "@" is > the least contentious option, then I'm ok with it too ("@@" doesn't > sound like a good idea at all, though). I don't see why. Since Python uses * and ** for scalar multiplication and exponentiation, and given that @ is accepted as matrix multiplication, then it is completely logical to follow the same pattern, which gives us @@ for matrix exponentiation. That's no sillier than using ** for exponentiation, instead of ^ as the Gods intended :-) -- Steven From solipsis at pitrou.net Fri Mar 14 13:57:40 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 13:57:40 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> Message-ID: <20140314135740.050bcd7f@fsol> On Fri, 14 Mar 2014 23:32:39 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > On Fri, Mar 14, 2014 at 11:24:26AM +0100, Antoine Pitrou wrote: > > > > This PEP proposes two new binary operators dedicated to matrix > > > multiplication and matrix power, spelled ``@`` and ``@@`` > > > respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) > > > > That sounds like a strange choice to me. There's nothing in "@" or "@@" > > that suggests "matrices" or "multiplication". > > The PEP even gives a mnemonic for it: @ is for mATrix multiplication. Well, "@" is not pronounced "at" in every country, so your mnemonic will only work for a subset of Python users. > There's nothing in @ that suggests "decorator", nor is there anything > about * that suggests scalar multiplication. Apart from the fact that "*" is commonly used for multiplication in most programming languages, you mean? ("@" either spells "decorator" or "in reply to" or "e-mail address" to me, depending on the context) > It also has the problem that it could cause confusion with vector dot > product. Read out A .* B aloud and you get something that sounds like it > might mean dot product, "A dot times B". That is true, and I realize I might have got the convention reversed :-) But, again, "dot product" is an English term which has no direct equivalent in other languages. In French, "dot product" doesn't exist, we only say "scalar product" and therefore the confusion doesn't exist. > > That said, I'm rather sympathetic to the general idea, and if "@" is > > the least contentious option, then I'm ok with it too ("@@" doesn't > > sound like a good idea at all, though). > > I don't see why. Since Python uses * and ** for scalar multiplication > and exponentiation, and given that @ is accepted as matrix > multiplication, then it is completely logical to follow the same > pattern, which gives us @@ for matrix exponentiation. It's logical, but it's much less useful, and it's also starting to look really weird: since the operator space is scarce, it makes sense to only provide those that have common use. Regards Antoine. From donald at stufft.io Fri Mar 14 14:01:38 2014 From: donald at stufft.io (Donald Stufft) Date: Fri, 14 Mar 2014 09:01:38 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314135740.050bcd7f@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> <20140314135740.050bcd7f@fsol> Message-ID: <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io> The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. > On Mar 14, 2014, at 8:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > > ("@" either spells "decorator" or "in reply to" or "e-mail address" to > me, depending on the context) From solipsis at pitrou.net Fri Mar 14 14:19:56 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 14:19:56 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> <20140314135740.050bcd7f@fsol> <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io> Message-ID: <20140314141956.25a5b920@fsol> On Fri, 14 Mar 2014 09:01:38 -0400 Donald Stufft <donald at stufft.io> wrote: > The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. It's not about "handling it", it's about finding the best candidate. Regards Antoine. From mal at egenix.com Fri Mar 14 14:20:33 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 14 Mar 2014 14:20:33 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfuouo$v25$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> Message-ID: <53230221.1030501@egenix.com> On 14.03.2014 12:25, Robert Kern wrote: > On 2014-03-14 10:16, M.-A. Lemburg wrote: > >> I have some questions: >> >> 1. Since in math, the operator is usually spelt "?" (the center dot, >> or "." but that's already reserved for methods and attributes in >> Python), why not try to use that instead of "@" (which in Python >> already identifies decorators) ? > > I think the current feeling of the Python core team is against including non-ASCII characters in the > language's keywords or operators. Even if that were not so, I would still recommend against it > because it would be quite difficult to type. I don't know off-hand the key combination to do it on > my native system, and it would change from system to system. That's a fair argument. How about using the degree symbol instead: "?" ? (A ? B).T == B.T ? A.T -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 14 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 15 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 26 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From robert.kern at gmail.com Fri Mar 14 14:33:26 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 13:33:26 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53230221.1030501@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> Message-ID: <lfv0em$r68$1@ger.gmane.org> On 2014-03-14 13:20, M.-A. Lemburg wrote: > On 14.03.2014 12:25, Robert Kern wrote: >> On 2014-03-14 10:16, M.-A. Lemburg wrote: >> >>> I have some questions: >>> >>> 1. Since in math, the operator is usually spelt "?" (the center dot, >>> or "." but that's already reserved for methods and attributes in >>> Python), why not try to use that instead of "@" (which in Python >>> already identifies decorators) ? >> >> I think the current feeling of the Python core team is against including non-ASCII characters in the >> language's keywords or operators. Even if that were not so, I would still recommend against it >> because it would be quite difficult to type. I don't know off-hand the key combination to do it on >> my native system, and it would change from system to system. > > That's a fair argument. How about using the degree symbol instead: "?" ? > > (A ? B).T == B.T ? A.T That's still not ASCII, and I still don't know how to type it off the top of my head (US keyboard, OS X), though experimentation shows that Alt-0 does it easily enough. I don't know if it's universally that easy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From masklinn at masklinn.net Fri Mar 14 14:35:15 2014 From: masklinn at masklinn.net (Masklinn) Date: Fri, 14 Mar 2014 14:35:15 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> <20140314135740.050bcd7f@fsol> <DAEC2082-3A34-4DA3-85B1-374D0190B196@stufft.io> Message-ID: <599A7E12-670C-47E9-9DAD-F6DD3E0B1263@masklinn.net> On 2014-03-14, at 14:01 , Donald Stufft <donald at stufft.io> wrote: > The reason @ is used for reply is because the menomic Is "at soandso". Probably if people can handle that they can handle it here too. People don't "handle" anything, they learn the semantics of the @ symbol in-context, whatever name they give it. That does not mean non-english-speakers associate it with the sound "at" (they don't, in my experience), even ignoring the dodginess of using "at" as a mnemonic for "matrix", and @ as a shortcut for that (doubly so in Python where @ already means "decorator") From ethan at stoneleaf.us Fri Mar 14 15:11:26 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 14 Mar 2014 07:11:26 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfuouo$v25$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> Message-ID: <53230E0E.9030707@stoneleaf.us> On 03/14/2014 04:25 AM, Robert Kern wrote: > > Some more from real code: > > RSR = R.dot(var_beta.dot(R.T)) > RSR = R @ var_beta @ R.T > > xx_inv.dot(xeps.dot(xx_inv)) > xx_inv @ xeps @ xx_inv > > dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T) > dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T) > > dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T)) > (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T) > > torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) > (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] Arrgggghhhhhhh!! Uncle! Uncle! +1! -- ~Ethan~ From ethan at stoneleaf.us Fri Mar 14 15:13:27 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 14 Mar 2014 07:13:27 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <5322EC7D.1020908@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> Message-ID: <53230E87.7070503@stoneleaf.us> On 03/14/2014 04:48 AM, M.-A. Lemburg wrote: > On 14.03.2014 12:25, Robert Kern wrote: >> >> RSR = R.dot(var_beta.dot(R.T)) >> RSR = R @ var_beta @ R.T >> >> xx_inv.dot(xeps.dot(xx_inv)) >> xx_inv @ xeps @ xx_inv >> >> dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - 4/period*F2lower.dot(F2lower.T) >> dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ F2lower.T) >> >> dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + FX.dot(Gi.dot(dFX2_dper.T)) >> (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ dFX2_dper.T) >> >> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) >> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] > > This doesn't look very readable to me - the operator saves you > a few parens in some situations, but as in the last example, it can > also require adding new ones. The difference being that grouping parens are easier to read and are less visual clutter than calling parens. -- ~Ethan~ From ericsnowcurrently at gmail.com Fri Mar 14 15:42:39 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 14 Mar 2014 08:42:39 -0600 Subject: [Python-ideas] re-organizing the sys module In-Reply-To: <lfu92v$8pb$1@ger.gmane.org> References: <CALFfu7Cfc9u-xNAXnNJJFBRRtBO6RVpr8g0n3ghC2yQe0pgweQ@mail.gmail.com> <lfrt3c$62p$1@ger.gmane.org> <lfs6qo$tvh$1@ger.gmane.org> <20140313160353.745d9659@anarchist.wooz.org> <lfu92v$8pb$1@ger.gmane.org> Message-ID: <CALFfu7DYonXKqUdJww59g-taADS7xNB15-M48jjMcs1eiGtHmw@mail.gmail.com> On Mar 14, 2014 12:55 AM, "Serhiy Storchaka" <storchaka at gmail.com> wrote: > Perhaps. If sys attributes will be writable properties which updates _sys. Precisely! -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/24f2aca0/attachment.html> From robert.kern at gmail.com Fri Mar 14 15:44:02 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 14:44:02 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53230221.1030501@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> Message-ID: <lfv4j2$hcd$1@ger.gmane.org> On 2014-03-14 13:20, M.-A. Lemburg wrote: > On 14.03.2014 12:25, Robert Kern wrote: >> On 2014-03-14 10:16, M.-A. Lemburg wrote: >> >>> I have some questions: >>> >>> 1. Since in math, the operator is usually spelt "?" (the center dot, >>> or "." but that's already reserved for methods and attributes in >>> Python), why not try to use that instead of "@" (which in Python >>> already identifies decorators) ? >> >> I think the current feeling of the Python core team is against including non-ASCII characters in the >> language's keywords or operators. Even if that were not so, I would still recommend against it >> because it would be quite difficult to type. I don't know off-hand the key combination to do it on >> my native system, and it would change from system to system. > > That's a fair argument. How about using the degree symbol instead: "?" ? > > (A ? B).T == B.T ? A.T Your point is taken, though. I do find these smaller symbols more readable and similar to standard mathematical notation than an @ sign, which is as big or bigger than most uppercase characters. Unfortunately, ASCII leaves us few single-character options. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From guido at python.org Fri Mar 14 15:49:40 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2014 07:49:40 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfv4j2$hcd$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> Message-ID: <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> Where on earth is the degree sign on my keyboard? (Don't answer. It's a rhetorical question.) On Fri, Mar 14, 2014 at 7:44 AM, Robert Kern <robert.kern at gmail.com> wrote: > On 2014-03-14 13:20, M.-A. Lemburg wrote: > >> On 14.03.2014 12:25, Robert Kern wrote: >> >>> On 2014-03-14 10:16, M.-A. Lemburg wrote: >>> >>> I have some questions: >>>> >>>> 1. Since in math, the operator is usually spelt "?" (the center dot, >>>> or "." but that's already reserved for methods and attributes in >>>> Python), why not try to use that instead of "@" (which in Python >>>> already identifies decorators) ? >>>> >>> >>> I think the current feeling of the Python core team is against including >>> non-ASCII characters in the >>> language's keywords or operators. Even if that were not so, I would >>> still recommend against it >>> because it would be quite difficult to type. I don't know off-hand the >>> key combination to do it on >>> my native system, and it would change from system to system. >>> >> >> That's a fair argument. How about using the degree symbol instead: "?" ? >> >> (A ? B).T == B.T ? A.T >> > > Your point is taken, though. I do find these smaller symbols more readable > and similar to standard mathematical notation than an @ sign, which is as > big or bigger than most uppercase characters. Unfortunately, ASCII leaves > us few single-character options. > > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/0971552f/attachment.html> From p.f.moore at gmail.com Fri Mar 14 15:53:08 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 14 Mar 2014 14:53:08 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53230E87.7070503@stoneleaf.us> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> Message-ID: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> On 14 March 2014 14:13, Ethan Furman <ethan at stoneleaf.us> wrote: >>> >>> torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) >>> (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] >> >> >> This doesn't look very readable to me - the operator saves you >> a few parens in some situations, but as in the last example, it can >> also require adding new ones. > > > The difference being that grouping parens are easier to read and are less > visual clutter than calling parens. Personally, my biggest problem with all of these is that the @ sign is a bit too big and bulky, so it's visually jarring. But: 1. As the PEP mentions, there aren't many options available. 2. I trust the scientific community to choose something that they are comfortable with (I'm only an interested outsider). 3. Ultimately, this is just bikeshedding. One genuine question though - when the PEP was developed, were multi-character operators like .* or <*> considered? A "rejected alternative operator symbols" would be a useful addition to the PEP (although it'd rob all us non-experts of the opportunity to bikeshed :-)) On a related note, the @@ operator is visually dreadful (far too heavy). While I see the */** analogy, and I appreciate that there's few good options, I'd definitely like to see some evidence that it's "the best of a bad lot" in the PEP. Paul. From njs at pobox.com Fri Mar 14 15:55:06 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 14:55:06 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53230221.1030501@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> Message-ID: <CAPJVwBnSPYstfU+NhJAiCvNSXBK8HjxZ2zYV7=e0qDz3EY73=A@mail.gmail.com> On Fri, Mar 14, 2014 at 1:20 PM, M.-A. Lemburg <mal at egenix.com> wrote: > On 14.03.2014 12:25, Robert Kern wrote: >> On 2014-03-14 10:16, M.-A. Lemburg wrote: >> >>> I have some questions: >>> >>> 1. Since in math, the operator is usually spelt "?" (the center dot, >>> or "." but that's already reserved for methods and attributes in >>> Python), why not try to use that instead of "@" (which in Python >>> already identifies decorators) ? >> >> I think the current feeling of the Python core team is against including non-ASCII characters in the >> language's keywords or operators. Even if that were not so, I would still recommend against it >> because it would be quite difficult to type. I don't know off-hand the key combination to do it on >> my native system, and it would change from system to system. > > That's a fair argument. How about using the degree symbol instead: "?" ? > > (A ? B).T == B.T ? A.T Well, obviously we can bikeshed this all day :-). For reasons the draft PEP goes into in more detail, what we need is a symbol that means "matrix rather than scalar/elementwise multiplication", and there is no existing conventional symbol (either in math or programming) that has that meaning -- \cdot, \degree, etc. don't have that meaning any more than @ does. So we have to make something up. My feeling is that @ is the least-bad option, and -- as the person who's maybe been staring at these code samples the longest -- I found that I got used to it pretty quickly. But I don't think debating is going to lead to any "obviously best" answer, at some point we'll just have to pick something. So maybe it's more useful to focus more on the other parts of the proposal? :-) -n From robert.kern at gmail.com Fri Mar 14 15:56:31 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 14:56:31 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> Message-ID: <lfv5af$roe$1@ger.gmane.org> On 2014-03-14 14:49, Guido van Rossum wrote: > Where on earth is the degree sign on my keyboard? (Don't answer. It's a > rhetorical question.) To be fair to Marc-Andre, there is a key for this right on his keyboard. It is we benighted Americans who lack it. http://en.wikipedia.org/wiki/German_keyboard_layout -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From joseph.martinot-lagarde at m4x.org Fri Mar 14 15:54:02 2014 From: joseph.martinot-lagarde at m4x.org (Joseph Martinot-Lagarde) Date: Fri, 14 Mar 2014 14:54:02 +0000 (UTC) Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> Message-ID: <loom.20140314T154850-185@post.gmane.org> Robert Kern <robert.kern at ...> writes: > > On 2014-03-14 13:20, M.-A. Lemburg wrote: > > On 14.03.2014 12:25, Robert Kern wrote: > >> On 2014-03-14 10:16, M.-A. Lemburg wrote: > >> > >>> I have some questions: > >>> > >>> 1. Since in math, the operator is usually spelt "?" (the center dot, > >>> or "." but that's already reserved for methods and attributes in > >>> Python), why not try to use that instead of " <at> " (which in Python > >>> already identifies decorators) ? > >> > >> I think the current feeling of the Python core team is against including non-ASCII characters in the > >> language's keywords or operators. Even if that were not so, I would still recommend against it > >> because it would be quite difficult to type. I don't know off-hand the key combination to do it on > >> my native system, and it would change from system to system. > > > > That's a fair argument. How about using the degree symbol instead: "?" ? > > > > (A ? B).T == B.T ? A.T > > Your point is taken, though. I do find these smaller symbols more readable and > similar to standard mathematical notation than an <at> sign, which is as big or > bigger than most uppercase characters. Unfortunately, ASCII leaves us few > single-character options. > Putting aside tha ascii problem, ? is easily written using a AZERTY keyboard. It is smaller and less convoluted than @ and looks like the mathematical notation for function composition, which is similar to matrix multiplication. Still, not ascii and not displayed on every keyboard... From njs at pobox.com Fri Mar 14 16:00:02 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 15:00:02 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfuouo$v25$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> Message-ID: <CAPJVwBmRhWp_4KSuXL==cVAqp1-tQb1fw8u7GFjRtez3kuSP2w@mail.gmail.com> On Fri, Mar 14, 2014 at 11:25 AM, Robert Kern <robert.kern at gmail.com> wrote: > On 2014-03-14 10:16, M.-A. Lemburg wrote: >> Now since this is all about syntactic sugar, we also need to look at >> some code examples: [...] >> c = A @ v >> vs. >> c = A ? v >> vs. >> c = A.dot(v) >> >> Hmm, even though I'd love to see matrix operators in Python, >> I don't think they really add clarity to the syntax of matrix >> calculations - a bit disappointing, I must say :-( > > Some more from real code: > > RSR = R.dot(var_beta.dot(R.T)) > RSR = R @ var_beta @ R.T > > xx_inv.dot(xeps.dot(xx_inv)) > xx_inv @ xeps @ xx_inv > > dF2lower_dper.dot(F2lower.T) + F2lower.dot(dF2lower_dper.T) - > 4/period*F2lower.dot(F2lower.T) > dF2lower_dper @ F2lower.T + F2lower @ dF2lower_dper.T - 4/period*(F2lower @ > F2lower.T) > > dFX_dper.dot(Gi.dot(FX2.T)) - FX.dot(Gi.dot(dG_dper.dot(Gi.dot(FX2.T)))) + > FX.dot(Gi.dot(dFX2_dper.T)) > (dFX_dper @ Gi @ FX2.T) - (FX @ Gi @ dG_dper @ Gi @ FX2.T) + (FX @ G @ > dFX2_dper.T) > > torient_inv.dot(tdof).dot(torient).dot(self.vertices[parent].meta['key'])) > (((torient_inv @ tdof) @ torient) @ self.vertices[parent].meta['key'] For those skimming along, note that there's also a more detailed example in the draft PEP, looking at a few different aspects of code usability: https://github.com/numpy/numpy/pull/4351/files#diff-dd521035cec59cd5fb2e040f0703742bR230 -n From mal at egenix.com Fri Mar 14 16:16:11 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 14 Mar 2014 16:16:11 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfv5af$roe$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> <lfv5af$roe$1@ger.gmane.org> Message-ID: <53231D3B.6070908@egenix.com> On 14.03.2014 15:56, Robert Kern wrote: > On 2014-03-14 14:49, Guido van Rossum wrote: >> Where on earth is the degree sign on my keyboard? (Don't answer. It's a >> rhetorical question.) > > To be fair to Marc-Andre, there is a key for this right on his keyboard. It is we benighted > Americans who lack it. > > http://en.wikipedia.org/wiki/German_keyboard_layout The keyboard is indeed where I got the idea from :-) I didn't know that it's not available on US keyboards; only German and French keyboard appear to have a key for it: http://en.wikipedia.org/wiki/Degree_symbol#Keyboard_entry Ok, enough bike shedding for today :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 14 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 15 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 26 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From flying-sheep at web.de Fri Mar 14 16:29:02 2014 From: flying-sheep at web.de (Philipp A.) Date: Fri, 14 Mar 2014 16:29:02 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53231D3B.6070908@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> <lfv5af$roe$1@ger.gmane.org> <53231D3B.6070908@egenix.com> Message-ID: <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com> well, on linux, everyone has ???. on american layouts, it?s AltGr+Shift+0, I think. also i think it?s one reason why you americans love your funny imperial units so much: you can?t even *type* ?23?C? if you?re not on linux. 2014-03-14 16:16 GMT+01:00 M.-A. Lemburg <mal at egenix.com>: > On 14.03.2014 15:56, Robert Kern wrote: > > On 2014-03-14 14:49, Guido van Rossum wrote: > >> Where on earth is the degree sign on my keyboard? (Don't answer. It's a > >> rhetorical question.) > > > > To be fair to Marc-Andre, there is a key for this right on his keyboard. > It is we benighted > > Americans who lack it. > > > > http://en.wikipedia.org/wiki/German_keyboard_layout > > The keyboard is indeed where I got the idea from :-) > > I didn't know that it's not available on US keyboards; only German > and French keyboard appear to have a key for it: > > http://en.wikipedia.org/wiki/Degree_symbol#Keyboard_entry > > Ok, enough bike shedding for today :-) > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 14 2014) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-03-29: PythonCamp 2014, Cologne, Germany ... 15 days to go > 2014-04-09: PyCon 2014, Montreal, Canada ... 26 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/eae3fb81/attachment-0001.html> From phd at phdru.name Fri Mar 14 16:36:43 2014 From: phd at phdru.name (Oleg Broytman) Date: Fri, 14 Mar 2014 16:36:43 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> <lfv5af$roe$1@ger.gmane.org> <53231D3B.6070908@egenix.com> <CAN8d9gkLcoXHp__2wCstu-R3J0t2GRXkTrMf3QYD9C8=gVNaaQ@mail.gmail.com> Message-ID: <20140314153643.GA14506@phdru.name> On Fri, Mar 14, 2014 at 04:29:02PM +0100, "Philipp A." <flying-sheep at web.de> wrote: > well, on linux, everyone has ???????. Well, not on my linux. PS. I am -10**6 on any non-ascii character. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From ron3200 at gmail.com Fri Mar 14 16:41:18 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 14 Mar 2014 10:41:18 -0500 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314123239.GP12595@ando> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> Message-ID: <lfv7ug$tuv$1@ger.gmane.org> On 03/14/2014 07:32 AM, Steven D'Aprano wrote: >> >AFAIK, other languages sometimes use ".*" for matrix multiplication (as >> >opposed to element-wise multiplication). Why not re-use that >> >convention? Would that lead to parsing difficulties? > "Syntax should not look like grit on Tim Peters' monitor." > > .* is, in my opinion, pretty ugly, and leads to even ugly > extensions: > > a .* b # matrix multiplication > a .*= b # in place matrix multiplication > a .** b # matrix exponentiation > a .**= b # in-place matrix exponentiation > > And it suffers from exactly the same lack of connection to matrix > multiplication as @ does: there is nothing about . that spells "matrix". > > It also has the problem that it could cause confusion with vector dot > product. Read out A .* B aloud and you get something that sounds like it > might mean dot product, "A dot times B". The best I can come up with is to use '^' instead of dot. a ^* b # matrix multiplication a ^^ b # matrix exponentiation Also two ^ together looks like a M. It's not a strong association, but it may help some people remember what it's for. Or the second one could be spelled... ^**, but I think the ^^ is cleaner. Question? Is it possible for python to tell these apart at compile time? >>> a = 6 >>> b = 4 >>> c = 8 >>> a^b 2 >>> a ^b c File "<stdin>", line 1 a ^b c ^ SyntaxError: invalid syntax So that this later example does... a.__use__(self, "b", c) The ^b is unrelated to the name b in the second case. Or is this just not possible? If it could be done, then ... a ^* b --> a.__use__(self, "*", b) a ^^ b --> a.__use__(self, "^", b) Where the __use__ method is defined on a module level object. (and not defined on builtin's as a general rule.) def __use__(self, s, other): if s == "*": ... # do matrix multiply return result elif s == "^": ... # do matrix exponentiation return result raise TypeError Anyway to make this work nicely? Cheers, Ron From zachary.ware+pyideas at gmail.com Fri Mar 14 16:46:25 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Fri, 14 Mar 2014 10:46:25 -0500 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> Message-ID: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> Disclaimer: I am not a numpy (or similar) user and have only skimmed this thread. If I've simply missed something big, I can be safely ignored :) On Fri, Mar 14, 2014 at 9:53 AM, Paul Moore <p.f.moore at gmail.com> wrote: > Personally, my biggest problem with all of these is that the @ sign is > a bit too big and bulky, so it's visually jarring. > > But: > > 1. As the PEP mentions, there aren't many options available. > 2. I trust the scientific community to choose something that they are > comfortable with (I'm only an interested outsider). > 3. Ultimately, this is just bikeshedding. > > One genuine question though - when the PEP was developed, were > multi-character operators like .* or <*> considered? A "rejected > alternative operator symbols" would be a useful addition to the PEP > (although it'd rob all us non-experts of the opportunity to bikeshed > :-)) > > On a related note, the @@ operator is visually dreadful (far too > heavy). While I see the */** analogy, and I appreciate that there's > few good options, I'd definitely like to see some evidence that it's > "the best of a bad lot" in the PEP. I agree with Paul, @/@@ just look scary as operators. Here's a few multi-character options I've come up with that I would like to see shot down in flames before @/@@ are added to Python: >< (for multiplication, not sure about exponentiation. I only like it because it's the shortest thing I've come up with that looks somewhat like multiplication) [*] / [**] ([] make me think 'matrix', but this might be confusing to the parser) |*| / |**| (pretty close to [], shouldn't confuse the parser) The downside is that the inplace version of matrix exponentiation would be a 5 character operator ([**]=), which I will freely admit is not appealing, but it *looks* quite a lot nicer to me than "@@=" does. -- Zach From njs at pobox.com Fri Mar 14 17:20:38 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 16:20:38 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> Message-ID: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com> On Fri, Mar 14, 2014 at 3:46 PM, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote: > I agree with Paul, @/@@ just look scary as operators. Here's a few > multi-character options I've come up with that I would like to see > shot down in flames before @/@@ are added to Python: > >>< (for multiplication, not sure about exponentiation. I only like it because it's the shortest thing I've come up with that looks somewhat like multiplication) > [*] / [**] ([] make me think 'matrix', but this might be confusing to > the parser) > |*| / |**| (pretty close to [], shouldn't confuse the parser) I really liked the [*] or |*| "footnote operator" idea, and got all excited, until I tried it :-). Using the PEP's example of a linear hypothesis test, compare: S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) versus S = (H [*] beta - r).T [*] inv(H [*] V [*] H.T) [*] (H [*] beta - r) S = (H |*| beta - r).T |*| inv(H |*| V |*| H.T) |*| (H |*| beta - r) A big part of the motivation for wanting an infix operator for this kind of expression is that when using function syntax, the visual clutter from all the required parentheses makes expressions hard to parse by eye. [*] has this same problem of introducing visual nesting and making it hard to pick out which vertical shapes are actually grouping parens, and which are the operators. E.g., try counting how many sub-expressions there are in each of these expressions -- I find it much easier to pick out the 3 groups in the top example than in the bottom two. Sort of a variant of leaning toothpick syndrome... -n From p.f.moore at gmail.com Fri Mar 14 17:16:12 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 14 Mar 2014 16:16:12 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> Message-ID: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> On 14 March 2014 15:46, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote: > I agree with Paul, @/@@ just look scary as operators. Here's a few > multi-character options I've come up with that I would like to see > shot down in flames before @/@@ are added to Python: Just as a contrasting point, I've been reading this thread on gmail with a proportional font. I went and looked at the PEP in a fixed width font earlier, and the @ sign doesn't look anywhere near as bad there. Paul From nathan at cmu.edu Fri Mar 14 17:39:21 2014 From: nathan at cmu.edu (Nathan Schneider) Date: Fri, 14 Mar 2014 12:39:21 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> Message-ID: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> On Fri, Mar 14, 2014 at 12:16 PM, Paul Moore <p.f.moore at gmail.com> wrote: > On 14 March 2014 15:46, Zachary Ware <zachary.ware+pyideas at gmail.com> > wrote: > > I agree with Paul, @/@@ just look scary as operators. Here's a few > > multi-character options I've come up with that I would like to see > > shot down in flames before @/@@ are added to Python: > > Just as a contrasting point, I've been reading this thread on gmail > with a proportional font. I went and looked at the PEP in a fixed > width font earlier, and the @ sign doesn't look anywhere near as bad > there. > > In Courier New: S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) Still looks kind of bulky to me, because @ is the height and width of a capital letter. How about prefixing * with an innocuous backtick? S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r) That way no part of the operator extends to the baseline, so identifiers and parentheses/brackets are visually well-separated from this as they are with most other binary operators. Nathan > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/88d86d95/attachment-0001.html> From alexander.belopolsky at gmail.com Fri Mar 14 17:40:15 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 14 Mar 2014 12:40:15 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com> Message-ID: <CAP7h-xaTjbTQ+gctt=pj87id7EeTUdYptWTb9_NN-5jn383z6g@mail.gmail.com> On Fri, Mar 14, 2014 at 12:20 PM, Nathaniel Smith <njs at pobox.com> wrote: > >>< (for multiplication, not sure about exponentiation. I only like it > because it's the shortest thing I've come up with that looks somewhat like > multiplication) > > [*] / [**] ([] make me think 'matrix', but this might be confusing to > > the parser) > > |*| / |**| (pretty close to [], shouldn't confuse the parser) > > I really liked the [*] or |*| "footnote operator" idea, and got all > excited, until I tried it :-). +1 Consider this: A[*](B+C) vs. A[0]*(B+C) the former looks like that later fat-fingered. Given that in numpy it is no uncommon to have ":", ":,:", "...", or even "()" to go inside [], [*] is more likely to be interpreted by those who first see it as some new form of fancy indexing rather than matrix multiplication. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/fecf6b1d/attachment.html> From njs at pobox.com Fri Mar 14 17:41:36 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 16:41:36 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> Message-ID: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> On Fri, Mar 14, 2014 at 2:53 PM, Paul Moore <p.f.moore at gmail.com> wrote: > One genuine question though - when the PEP was developed, were > multi-character operators like .* or <*> considered? A "rejected > alternative operator symbols" would be a useful addition to the PEP > (although it'd rob all us non-experts of the opportunity to bikeshed > :-)) > > On a related note, the @@ operator is visually dreadful (far too > heavy). While I see the */** analogy, and I appreciate that there's > few good options, I'd definitely like to see some evidence that it's > "the best of a bad lot" in the PEP. It's worth noting that @@ will probably see marginal use -- matrix power and matrix inversion are not common operations in number crunching code. Matrix inversion is very common in math formulas, but on a computer you almost always want to use a fused invert+multiply operation instead of inversion itself. My original draft didn't even allow '@@ -1'; I added it at the request of the symbolic math guys. In practice the main use case for @@ will probably be as a crutch for beginners before they learn about better tools like numpy.linalg.solve (which implements fused invert+multiply). Anyway, I wrote some more text for the PEP, see what you think: Rationale for specification details =================================== Choice of operator ------------------ Why ``@`` instead of some other spelling? There isn't any consensus across other programming languages about how this operator should be named [#matmul-other-langs]_; here we discuss the various options. Restricting ourselves only to symbols present on US English keyboards, the punctuation characters that don't already have a meaning in Python expression context are: ``@``, backtick, ``$``, ``!``, and ``?``. Of these options, ``@`` is clearly the best; ``!`` and ``?`` are already heavily freighted with inapplicable meanings in the programming context, backtick has been banned from Python by BDFL pronouncement (see PEP 3099), and ``$`` is uglier, even more dissimilar to ``*`` and :math:`\cdot`, and has Perl/PHP baggage. ``$`` is probably the second-best option of these, though. Symbols which are not present on US English keyboards start at a significant disadvantage (having to spend 5 minutes at the beginning of every numeric Python tutorial just going over keyboard layouts is not a hassle anyone really wants). Plus, even if we somehow overcame the typing problem, it's not clear there are any that are actually better than ``@``. Some options that have been suggested include: * U+00D7 MULTIPLICATION SIGN: ``A ? B`` * U+22C5 DOT OPERATOR: ``A ? B`` * U+2297 CIRCLED TIMES: ``A ? B`` * U+00B0 DEGREE: ``A ? B`` What we need, though, is an operator that means "matrix multiplication, as opposed to scalar/elementwise multiplication". There is no conventional symbol for these in mathematics or programming, where these operations are usually distinguished by context. (And U+2297 CIRCLED TIMES is actually used conventionally to mean exactly the opposite: elementwise multiplication -- the "Hadamard product" -- as opposed to matrix multiplication). ``@`` at least has the virtue that it *looks* like a funny non-commutative operator; a naive user who knows maths but not programming couldn't look at ``A * B`` versus ``A ? B``, or ``A * B`` versus ``A ? B``, or ``A * B`` versus ``A ? B`` and guess which one is the usual multiplication, and which one is the special case. Finally, there is the option of using multi-character tokens. Some options: * Matlab uses a ``.*`` operator. Aside from being visually confusable with ``*``, this would be a terrible choice for us because in Matlab, ``*`` means matrix multiplication and ``.*`` means elementwise multiplication, so using ``.*`` for matrix multiplication would make us exactly backwards from what Matlab users expect. * APL apparently used ``+.?``, which by combining a multi-character token, confusing attribute-access-like . syntax, and a unicode character, ranks somewhere below U+2603 SNOWMAN on our candidate list. If we like the idea of combining addition and multiplication operators as being evocative of how matrix multiplication actually works, then something like ``+*`` could be used -- though this may be too easy to confuse with ``*+``, which is just multiplication combined with the unary ``+`` operator. * PEP 211 suggested ``~*`` and ``~**``. This has the downside that it sort of suggests that there is a unary ``*`` operator that is being combined with unary ``~``, but it could work. * R uses ``%*%`` for matrix multiplication. In R this forms part of a general extensible infix system in which all tokens of the form ``%foo%`` are user-defined binary operators. We could steal the token without stealing the system. * Some other plausible candidates that have been suggested: ``><`` (= ascii drawing of the multiplication sign ?); the footnote operators ``[*]`` and ``[**]`` or ``|*|`` and ``|**|`` (but when used in context, the use of vertical grouping symbols tends to recreate the nested parentheses visual clutter that was noted as one of the major downsides of the function syntax we're trying to get away from); ``^*`` and ``^^``. So, it doesn't matter much, but ``@`` seems as good or better than any of the alternatives: * It's a friendly character that Pythoneers are already used to typing in decorators, but the decorator usage and the math expression usage are sufficiently dissimilar that it would be hard to confuse them in practice. * It's widely accessible across keyboard layouts (and thanks to its use in email addresses, this is true even of weird keyboards like those in phones). * It's round like ``*`` and :math:`\cdot`. * The mATrices mnemonic is cute. * The use of a single-character token reduces the line-noise effect, and makes ``@@`` possible, which is a nice bonus. * The swirly shape is reminiscent of the simultaneous sweeps over rows and columns that define matrix multiplication * Its asymmetry is evocative of its non-commutative nature. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From robert.kern at gmail.com Fri Mar 14 17:46:27 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 16:46:27 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> Message-ID: <lfvboj$eab$1@ger.gmane.org> On 2014-03-14 16:39, Nathan Schneider wrote: > In Courier New: > > S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) > > Still looks kind of bulky to me, because @ is the height and width of a capital > letter. How about prefixing * with an innocuous backtick? > > S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r) > > That way no part of the operator extends to the baseline, so identifiers and > parentheses/brackets are visually well-separated from this as they are with most > other binary operators. Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From p.f.moore at gmail.com Fri Mar 14 17:47:39 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 14 Mar 2014 16:47:39 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> Message-ID: <CACac1F_hDzrp6vOi3RMqrVtR=sKGqBmTZGNqd5WsU=6Zead-Nw@mail.gmail.com> On 14 March 2014 16:41, Nathaniel Smith <njs at pobox.com> wrote: > Anyway, I wrote some more text for the PEP, see what you think: >From my POV, this pretty much covers it. Thanks :-) Paul From guido at python.org Fri Mar 14 17:47:52 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2014 09:47:52 -0700 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> On Thu, Mar 13, 2014 at 6:59 PM, Nathaniel Smith <njs at pobox.com> wrote: [...] > PEP: XXXX > Title: Dedicated infix operators for matrix multiplication and matrix power > Version: $Revision$ > Last-Modified: $Date$ > Author: Nathaniel J. Smith <njs at pobox.com> > Status: Draft > Type: Standards Track > Python-Version: 3.5 > Content-Type: text/x-rst > Created: 20-Feb-2014 > Post-History: > I still have to read this but I've assigned a PEP number. Henceforth this will be known as PEP 465. So far I really like what I've read. On the bikeshedding, if someone wants to start introducing Unicode (non-ASCII) characters as operators they would have to propose a separate PEP arguing the benefits of Unicode -- I am heavily opposed to doing that ATM, for reasons that have already been brought up in the bikeshed. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/8e02f4ae/attachment-0001.html> From alexander.belopolsky at gmail.com Fri Mar 14 17:49:54 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 14 Mar 2014 12:49:54 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> Message-ID: <CAP7h-xbmavvO0BATr8prT0UtFYoeAoHGea0wvFy5Y0zonaAq9w@mail.gmail.com> On Fri, Mar 14, 2014 at 12:41 PM, Nathaniel Smith <njs at pobox.com> wrote: > (And U+2297 CIRCLED TIMES is actually used conventionally to > mean exactly the opposite: elementwise multiplication -- the "Hadamard > product" -- as opposed to matrix multiplication). > It's actually worse: CIRCLED TIMES is commonly used for tensor or outer product that is opposite to the proposed inner product meaning of vec @ vec. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/ee23b8ca/attachment.html> From robert.kern at gmail.com Fri Mar 14 17:54:55 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 16:54:55 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAPJVwBng6Ow+A5mmOCo=-s6BWrB9o0E_wrY-TO_Lt0OwYyy+VA@mail.gmail.com> Message-ID: <lfvc8f$luc$1@ger.gmane.org> On 2014-03-14 16:41, Nathaniel Smith wrote: > * The mATrices mnemonic is cute. I recommend dropping this mnemonic from the PEP. It has been pointed out that the mnemonic does not cross languages[1], and even to my Anglo ears is not especially evocative. [1] Python, like most programming languages, is unabashedly Anglocentric in its keywords, standard library and documentation, so some amount of Anglocentrism in new language features is unavoidable. My point is just that an Anglocentric mnemonic isn't really helping the argument for the PEP. If the PEP needs a mnemonic to survive, this is not going to cut it. If the PEP doesn't need a mnemonic, as I believe, it's best left out. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From zachary.ware+pyideas at gmail.com Fri Mar 14 18:04:24 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Fri, 14 Mar 2014 12:04:24 -0500 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CAPJVwBkuwRwowmq1-9FDYNLfVQydeNuv5gTnY0dncY5drrBpvg@mail.gmail.com> Message-ID: <CAKJDb-M-pWbsHBka=3NTgC7u7S2_L6+v4p-e+fNXBAO-hReGbA@mail.gmail.com> On Fri, Mar 14, 2014 at 11:20 AM, Nathaniel Smith <njs at pobox.com> wrote: > On Fri, Mar 14, 2014 at 3:46 PM, Zachary Ware > <zachary.ware+pyideas at gmail.com> wrote: >> Here's a few >> multi-character options I've come up with that I would like to see >> shot down in flames before @/@@ are added to Python: <snip> > A big part of the motivation for wanting an infix operator for this > kind of expression is that when using function syntax, the visual > clutter from all the required parentheses makes expressions hard to > parse by eye. [*] has this same problem of introducing visual nesting > and making it hard to pick out which vertical shapes are actually > grouping parens, and which are the operators. E.g., try counting how > many sub-expressions there are in each of these expressions -- I find > it much easier to pick out the 3 groups in the top example than in the > bottom two. Sort of a variant of leaning toothpick syndrome... This, plus added text for the PEP in a later message, is enough of a fire for me :). Thank you. I'll also admit that, again like Paul, I was reading in a proportional font. @/@@ is significantly less bad in fixed width, but I still don't love it. I certainly won't stand in the way if it's determined to be the least bad option, though. -- Zach From ron3200 at gmail.com Fri Mar 14 18:05:43 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 14 Mar 2014 12:05:43 -0500 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfv7ug$tuv$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> <lfv7ug$tuv$1@ger.gmane.org> Message-ID: <lfvcsp$t4s$1@ger.gmane.org> On 03/14/2014 10:41 AM, Ron Adam wrote: > > > On 03/14/2014 07:32 AM, Steven D'Aprano wrote: >>> >AFAIK, other languages sometimes use ".*" for matrix multiplication (as >>> >opposed to element-wise multiplication). Why not re-use that >>> >convention? Would that lead to parsing difficulties? >> "Syntax should not look like grit on Tim Peters' monitor." >> >> .* is, in my opinion, pretty ugly, and leads to even ugly >> extensions: >> >> a .* b # matrix multiplication >> a .*= b # in place matrix multiplication >> a .** b # matrix exponentiation >> a .**= b # in-place matrix exponentiation >> >> And it suffers from exactly the same lack of connection to matrix >> multiplication as @ does: there is nothing about . that spells "matrix". >> >> It also has the problem that it could cause confusion with vector dot >> product. Read out A .* B aloud and you get something that sounds like it >> might mean dot product, "A dot times B". > > The best I can come up with is to use '^' instead of dot. > > a ^* b # matrix multiplication > a ^^ b # matrix exponentiation > > Also two ^ together looks like a M. It's not a strong association, but it > may help some people remember what it's for. Or the second one could be > spelled... ^**, but I think the ^^ is cleaner. > > > Question? Is it possible for python to tell these apart at compile time? > > >>> a = 6 > >>> b = 4 > >>> c = 8 > >>> a^b > 2 > >>> a ^b c > File "<stdin>", line 1 > a ^b c > ^ > SyntaxError: invalid syntax > > So that this later example does... > > a.__use__(self, "b", c) > > > The ^b is unrelated to the name b in the second case. Or is this just not > possible? One more comment, if this idea is limited to only allow symbols that can't be seen in identifiers, then the conflict goes away also. That's probably not an unreasonable restriction. > If it could be done, then ... > > a ^* b --> a.__use__(self, "*", b) > a ^^ b --> a.__use__(self, "^", b) > > > Where the __use__ method is defined on a module level object. (and not > defined on builtin's as a general rule.) > > def __use__(self, s, other): > if s == "*": > ... # do matrix multiply > return result > elif s == "^": > ... # do matrix exponentiation > return result > raise TypeError > > > > Anyway to make this work nicely? I would of liked it to be more general, but this could work for just symbols. It could also be extended to allow ^'string', at some later date. Cheers, Ron From solipsis at pitrou.net Fri Mar 14 18:08:11 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 18:08:11 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <lfvboj$eab$1@ger.gmane.org> Message-ID: <20140314180811.35d7cdf6@fsol> On Fri, 14 Mar 2014 16:46:27 +0000 Robert Kern <robert.kern at gmail.com> wrote: > On 2014-03-14 16:39, Nathan Schneider wrote: > > > In Courier New: > > > > S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) > > > > Still looks kind of bulky to me, because @ is the height and width of a capital > > letter. How about prefixing * with an innocuous backtick? > > > > S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r) > > > > That way no part of the operator extends to the baseline, so identifiers and > > parentheses/brackets are visually well-separated from this as they are with most > > other binary operators. > > Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test. Not only grit, but the problem with the backtick is that it can look very close to a straight apostrophe. I am personally not fond of @, but I find it ok in that it is distinctive enough without being terribly ugly. Regards Antoine. From njs at pobox.com Fri Mar 14 18:12:12 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 17:12:12 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314180811.35d7cdf6@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <lfvboj$eab$1@ger.gmane.org> <20140314180811.35d7cdf6@fsol> Message-ID: <CAPJVwBmQOALxWgsGiEdZJ7W3BoK4+K9mrHt5e0ZTq7Ln5aRU+Q@mail.gmail.com> On Fri, Mar 14, 2014 at 5:08 PM, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Fri, 14 Mar 2014 16:46:27 +0000 > Robert Kern <robert.kern at gmail.com> wrote: >> On 2014-03-14 16:39, Nathan Schneider wrote: >> >> > In Courier New: >> > >> > S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) >> > >> > Still looks kind of bulky to me, because @ is the height and width of a capital >> > letter. How about prefixing * with an innocuous backtick? >> > >> > S = (H `* beta - r).T `* inv(H `* V `* H.T) `* (H `* beta - r) >> > >> > That way no part of the operator extends to the baseline, so identifiers and >> > parentheses/brackets are visually well-separated from this as they are with most >> > other binary operators. >> >> Fails the grit-on-Tim's-monitor test, or at least the grit-on-Robert's-monitor test. > > Not only grit, but the problem with the backtick is that it can look > very close to a straight apostrophe. Backtick has in fact been formally banned from py3 by BDFL pronouncement (for this reason): https://mail.python.org/pipermail/python-ideas/2007-January/000054.html http://legacy.python.org/dev/peps/pep-3099/ -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From mertz at gnosis.cx Fri Mar 14 18:08:57 2014 From: mertz at gnosis.cx (David Mertz) Date: Fri, 14 Mar 2014 10:08:57 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <loom.20140314T154850-185@post.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <loom.20140314T154850-185@post.gmane.org> Message-ID: <CAEbHw4YbhnV+8roPKsO7zhBtv3GyYnP1Zjwe4TDM=Qg1y0LMTA@mail.gmail.com> I think the fundamental issue with the degree sign is quite simply that it is *not-ASCII*. If we are willing to expand the syntax of Python to include other characters, then we should just use Unicode Character 'DOT OPERATOR' (U+22C5), which is actually the *exact* correct thing, not just "something that looks a bit similar." If we are worried about "stuff that's easy to enter on the keyboard", there's no reason AZERTY is necessarily more relevant than Dubeolsik or JCUKEN. And if we aim for "something that looks similar" we probably have lots of options on some keyboard layout in the world. On Fri, Mar 14, 2014 at 7:54 AM, Joseph Martinot-Lagarde < joseph.martinot-lagarde at m4x.org> wrote: > Robert Kern <robert.kern at ...> writes: > > > > > On 2014-03-14 13:20, M.-A. Lemburg wrote: > > > On 14.03.2014 12:25, Robert Kern wrote: > > >> On 2014-03-14 10:16, M.-A. Lemburg wrote: > > >> > > >>> I have some questions: > > >>> > > >>> 1. Since in math, the operator is usually spelt "?" (the center dot, > > >>> or "." but that's already reserved for methods and attributes in > > >>> Python), why not try to use that instead of " <at> " (which in > Python > > >>> already identifies decorators) ? > > >> > > >> I think the current feeling of the Python core team is against > including non-ASCII characters in the > > >> language's keywords or operators. Even if that were not so, I would > still recommend against it > > >> because it would be quite difficult to type. I don't know off-hand the > key combination to do it on > > >> my native system, and it would change from system to system. > > > > > > That's a fair argument. How about using the degree symbol instead: "?" > ? > > > > > > (A ? B).T == B.T ? A.T > > > > Your point is taken, though. I do find these smaller symbols more > readable > and > > similar to standard mathematical notation than an <at> sign, which is > as > big or > > bigger than most uppercase characters. Unfortunately, ASCII leaves us few > > single-character options. > > > > Putting aside tha ascii problem, ? is easily written using a AZERTY > keyboard. It is smaller and less convoluted than @ and looks like the > mathematical notation for function composition, which is similar to matrix > multiplication. > Still, not ascii and not displayed on every keyboard... > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/b0793869/attachment.html> From solipsis at pitrou.net Fri Mar 14 18:10:31 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 18:10:31 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <CAP7+vJKsWWe-YB+zsDAph9gzYQE1zFLqtRbSGKtQuFCJ+QQMbA@mail.gmail.com> <lfv5af$roe$1@ger.gmane.org> Message-ID: <20140314181031.507fb158@fsol> On Fri, 14 Mar 2014 14:56:31 +0000 Robert Kern <robert.kern at gmail.com> wrote: > On 2014-03-14 14:49, Guido van Rossum wrote: > > Where on earth is the degree sign on my keyboard? (Don't answer. It's a > > rhetorical question.) > > To be fair to Marc-Andre, there is a key for this right on his keyboard. It is > we benighted Americans who lack it. > > http://en.wikipedia.org/wiki/German_keyboard_layout Just for the record, on a French keyboard the combo is Shift+). Regards Antoine. From solipsis at pitrou.net Fri Mar 14 18:11:33 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 14 Mar 2014 18:11:33 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <53230221.1030501@egenix.com> <lfv4j2$hcd$1@ger.gmane.org> <loom.20140314T154850-185@post.gmane.org> Message-ID: <20140314181133.6be48954@fsol> On Fri, 14 Mar 2014 14:54:02 +0000 (UTC) Joseph Martinot-Lagarde <joseph.martinot-lagarde at m4x.org> wrote: > > Putting aside tha ascii problem, ? is easily written using a AZERTY > keyboard. It is smaller and less convoluted than @ and looks like the > mathematical notation for function composition, which is similar to matrix > multiplication. Perhaps we should keep it for the temperature literals that will appear in Python 4, though :-) Regards Antoine. From guido at python.org Fri Mar 14 18:53:01 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2014 10:53:01 -0700 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> Message-ID: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> I have now read the PEP, and I think it's good. I think it's a waste of time to keep bikeshedding on the choice of operator -- @ is the best compromise. I do have a few specific notes: - Right associativity is not unheard of in Python. E.g. **. If you think that for other reasons @ should be right associative, don't let Python's tradition stop you. But then you need to decide which of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or (a*b)@c? And if you choose the latter, it follows that a at b*c means a@(b*c) -- is that okay? (And similar examples exist for the other choice.) - Did you consider a duck-typing (is that the word?) attribute? E.g. a*b is elementwise multiplication; a.M*b must be used for matrix multiplication. (Your use of .T as "transpose" made me think of this.) Of course the question is, can you get those packages that currently use * for matrix multiply to comply? (I don't consider this a serious counter-proposal. But you list a bunch of rejected alternatives; this could be in that list. - Is @@ really necessary? It seems you are adding it mostly because it's cute and because of the parallel with **, not because it is actually important enough to add new syntax. And then later you use it as an argument for @, which seems a bit circular. Also, if we were to make @ right-associative, the parallel with ** is already imperfect. - For better counts of usages, perhaps Sourcegraph.com might help? It is a source code query engine that has a Python parser and (limited) type inference built in (also separately available as pysonar on github IIRC). To be clear, I don't need more numbers to be convinced. Once we've decided on associativity and @@, I'm ready to accept. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c888ea8b/attachment-0001.html> From ronan.lamy at gmail.com Fri Mar 14 19:40:49 2014 From: ronan.lamy at gmail.com (Ronan Lamy) Date: Fri, 14 Mar 2014 18:40:49 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <53234D31.20300@gmail.com> I have a few remarks, in random order: * The PEP is not just about the multiplication of matrices. The matrix-vector and vector-vector semantics are required in order to get nice expressions. This should probably be mentioned more prominently. * The complicated semantics for higher-dimensional arrays are effectively incompatible with the matrix-vector and vector-vector cases: for instance, writing the product of a list of matrices by a list of vectors requires something like "mats @ vecs[..., np.newaxis]". In other words, A @ B is AFAICT nearly equivalent to numpy.dot(A, B), therefore people who cannot use numpy.dot() today will not be able to use "@" either. * A big part of the problem with np.matrix is that it subclasses np.ndarray but has a completely different __mul__(). IMHO, the problems with it are a good argument for the importance of the Liskov substitution principle, but say rather little about the viability of a correctly implemented matrix type. From antony.lee at berkeley.edu Fri Mar 14 19:41:11 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Fri, 14 Mar 2014 11:41:11 -0700 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> Message-ID: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> A simple suggestion: what about defining the special methods as __at__, __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)? This make the operator more "neutral" -- it make it sound less "wrong" to overload it for something else that matrix multiplication (which is certainly an important topic, but even though I use numpy quite a lot, I don't actually use it at all for linear algebra (in fact I use elementwise multiplications much more often) -- so counting imports of numpy is a somewhat biaised metric for counting users of matrix multiplication). Antony 2014-03-14 10:53 GMT-07:00 Guido van Rossum <guido at python.org>: > I have now read the PEP, and I think it's good. I think it's a waste of > time to keep bikeshedding on the choice of operator -- @ is the best > compromise. I do have a few specific notes: > > - Right associativity is not unheard of in Python. E.g. **. If you > think that for other reasons @ should be right associative, don't > let Python's tradition stop you. But then you need to decide which > of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or > (a*b)@c? And if you choose the latter, it follows that a at b*c means > a@(b*c) -- is that okay? (And similar examples exist for the other > choice.) > > - Did you consider a duck-typing (is that the word?) attribute? > E.g. a*b is elementwise multiplication; a.M*b must be used for > matrix multiplication. (Your use of .T as "transpose" made me think > of this.) Of course the question is, can you get those packages > that currently use * for matrix multiply to comply? (I don't consider > this a serious counter-proposal. But you list a bunch of rejected > alternatives; this could be in that list. > > - Is @@ really necessary? It seems you are adding it mostly because > it's cute and because of the parallel with **, not because it is > actually important enough to add new syntax. And then later you use > it as an argument for @, which seems a bit circular. Also, if we > were to make @ right-associative, the parallel with ** is already > imperfect. > > - For better counts of usages, perhaps Sourcegraph.com might help? It > is a source code query engine that has a Python parser and (limited) > type inference built in (also separately available as pysonar on > github IIRC). To be clear, I don't need more numbers to be convinced. > > Once we've decided on associativity and @@, I'm ready to accept. > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/ea723753/attachment.html> From guido at python.org Fri Mar 14 19:53:47 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2014 11:53:47 -0700 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> Message-ID: <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com> On Fri, Mar 14, 2014 at 11:41 AM, Antony Lee <antony.lee at berkeley.edu>wrote: > A simple suggestion: what about defining the special methods as __at__, > __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)? This make > the operator more "neutral" -- it make it sound less "wrong" to overload it > for something else that matrix multiplication (which is certainly an > important topic, but even though I use numpy quite a lot, I don't actually > use it at all for linear algebra (in fact I use elementwise multiplications > much more often) -- so counting imports of numpy is a somewhat biaised > metric for counting users of matrix multiplication). > Ratatatatatat! :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/7ceda45b/attachment.html> From phd at phdru.name Fri Mar 14 20:00:09 2014 From: phd at phdru.name (Oleg Broytman) Date: Fri, 14 Mar 2014 20:00:09 +0100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com> Message-ID: <20140314190009.GA19829@phdru.name> On Fri, Mar 14, 2014 at 11:53:47AM -0700, Guido van Rossum <guido at python.org> wrote: > On Fri, Mar 14, 2014 at 11:41 AM, Antony Lee <antony.lee at berkeley.edu>wrote: > > A simple suggestion: what about defining the special methods as __at__, > > __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)? This make > > the operator more "neutral" -- it make it sound less "wrong" to overload it > > for something else that matrix multiplication (which is certainly an > > important topic, but even though I use numpy quite a lot, I don't actually > > use it at all for linear algebra (in fact I use elementwise multiplications > > much more often) -- so counting imports of numpy is a somewhat biaised > > metric for counting users of matrix multiplication). > > Ratatatatatat! :-) Is this the sound made by matrices being multiplied? ;-) Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From robert.kern at gmail.com Fri Mar 14 20:41:10 2014 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 14 Mar 2014 19:41:10 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> Message-ID: <lfvm08$bjg$1@ger.gmane.org> On 2014-03-14 17:53, Guido van Rossum wrote: > I have now read the PEP, and I think it's good. I think it's a waste of time to > keep bikeshedding on the choice of operator -- @ is the best compromise. I do > have a few specific notes: > > - Right associativity is not unheard of in Python. E.g. **. If you > think that for other reasons @ should be right associative, don't > let Python's tradition stop you. But then you need to decide which > of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or > (a*b)@c? And if you choose the latter, it follows that a at b*c means > a@(b*c) -- is that okay? (And similar examples exist for the other > choice.) I *think* either works out fine in practice, but I have a preference for @ binding tighter than *. `scalar * matrix @ vector` does fewer flops that way, and most current expressions are written with this binding anyways: `scalar * np.dot(matrix, vector)`. It just feels right to me. > - Did you consider a duck-typing (is that the word?) attribute? Facade? > E.g. a*b is elementwise multiplication; a.M*b must be used for > matrix multiplication. (Your use of .T as "transpose" made me think > of this.) Of course the question is, can you get those packages > that currently use * for matrix multiply to comply? (I don't consider > this a serious counter-proposal. But you list a bunch of rejected > alternatives; this could be in that list. It seems to me that the left-associativity of * makes this less useful than a dedicated operator if we consider chains of matrix multiplications. A.M * B.M * C == (A.M * B.M) * C We probably could make the rule that if the right-operand is one of these matmul-facades, then the result should also be a matmul-facade, but otherwise would be a plain numpy array. That would take care of this case, but it's not obvious to me that it's unambiguously the right thing to do in all cases. > - Is @@ really necessary? It seems you are adding it mostly because > it's cute and because of the parallel with **, not because it is > actually important enough to add new syntax. And then later you use > it as an argument for @, which seems a bit circular. Also, if we > were to make @ right-associative, the parallel with ** is already > imperfect. In my personal experience, I would use a matrix power operator much less than a matrix multiplication operator. I, at least, am content to continue to use a function call for that. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From skip at pobox.com Fri Mar 14 20:42:26 2014 From: skip at pobox.com (Skip Montanaro) Date: Fri, 14 Mar 2014 14:42:26 -0500 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote: > You'll notice that this draft is rather more developed than the > average first-round PEP posting Has this acquired a PEP number yet? It seems far enough along that it really ought to be more broadly visible. Skip From phd at phdru.name Fri Mar 14 20:53:41 2014 From: phd at phdru.name (Oleg Broytman) Date: Fri, 14 Mar 2014 20:53:41 +0100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> Message-ID: <20140314195341.GB19829@phdru.name> On Fri, Mar 14, 2014 at 02:42:26PM -0500, Skip Montanaro <skip at pobox.com> wrote: > On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote: > > You'll notice that this draft is rather more developed than the > > average first-round PEP posting > > Has this acquired a PEP number yet? It seems far enough along that it > really ought to be more broadly visible. https://mail.python.org/pipermail/python-ideas/2014-March/027099.html Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From zachary.ware+pyideas at gmail.com Fri Mar 14 20:54:11 2014 From: zachary.ware+pyideas at gmail.com (Zachary Ware) Date: Fri, 14 Mar 2014 14:54:11 -0500 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> Message-ID: <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com> On Fri, Mar 14, 2014 at 2:42 PM, Skip Montanaro <skip at pobox.com> wrote: > On Thu, Mar 13, 2014 at 8:59 PM, Nathaniel Smith <njs at pobox.com> wrote: >> You'll notice that this draft is rather more developed than the >> average first-round PEP posting > > Has this acquired a PEP number yet? It seems far enough along that it > really ought to be more broadly visible. Guido committed it to peps earlier: http://hg.python.org/peps/file/tip/pep-0465.txt It should be available at http://www.python.org/dev/peps/pep-0465/ soon. -- Zach From skip at pobox.com Fri Mar 14 21:04:11 2014 From: skip at pobox.com (Skip Montanaro) Date: Fri, 14 Mar 2014 15:04:11 -0500 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CANc-5UwgU-w45N9kC+srT5hceRx1xN0BYvix+a6Ou9zDPobX-w@mail.gmail.com> <CAKJDb-OTFV1yGKavdvL6A4Q4698mhCctvrsT1vYEj0N6mF_9+A@mail.gmail.com> Message-ID: <CANc-5Uy4GvtsduvYxWmniHiqRueLHpz8esn1quzHJYveH9JKWA@mail.gmail.com> On Fri, Mar 14, 2014 at 2:54 PM, Zachary Ware <zachary.ware+pyideas at gmail.com> wrote: > Guido committed it to peps earlier: > http://hg.python.org/peps/file/tip/pep-0465.txt > > It should be available at http://www.python.org/dev/peps/pep-0465/ soon. Thanks. I was just searching the PEP 0 page and didn't see it. Skip From ncoghlan at gmail.com Fri Mar 14 21:15:51 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Mar 2014 06:15:51 +1000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> Message-ID: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> On 15 Mar 2014 04:42, "Antony Lee" <antony.lee at berkeley.edu> wrote: > > A simple suggestion: what about defining the special methods as __at__, __rat__, __iat__ (@) and __atat__, __ratat__, __iatat__ (@@)? This make the operator more "neutral" -- it make it sound less "wrong" to overload it for something else that matrix multiplication (which is certainly an important topic, but even though I use numpy quite a lot, I don't actually use it at all for linear algebra (in fact I use elementwise multiplications much more often) -- so counting imports of numpy is a somewhat biaised metric for counting users of matrix multiplication). The method name for "*" is "__mul__" rather than "__star__", and so on for the other types. Naming the magic methods for their intended semantics rather than their syntax is an established pattern. A few other miscellaneous comments: - nice work on the PEP Nathaniel! - as with others, "@" as the operator doesn't thrill me, but I also think it crosses the threshold of "good enough given the constraints" - the PEP should probably recommend adding an "operator.matmul" function, a "PyObject_MatrixMultiply" C API and consider whether or not the new special method should be given a C level type slot. Cheers, Nick. > > Antony > > > 2014-03-14 10:53 GMT-07:00 Guido van Rossum <guido at python.org>: >> >> I have now read the PEP, and I think it's good. I think it's a waste of time to keep bikeshedding on the choice of operator -- @ is the best compromise. I do have a few specific notes: >> >> - Right associativity is not unheard of in Python. E.g. **. If you >> think that for other reasons @ should be right associative, don't >> let Python's tradition stop you. But then you need to decide which >> of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or >> (a*b)@c? And if you choose the latter, it follows that a at b*c means >> a@(b*c) -- is that okay? (And similar examples exist for the other >> choice.) >> >> - Did you consider a duck-typing (is that the word?) attribute? >> E.g. a*b is elementwise multiplication; a.M*b must be used for >> matrix multiplication. (Your use of .T as "transpose" made me think >> of this.) Of course the question is, can you get those packages >> that currently use * for matrix multiply to comply? (I don't consider >> this a serious counter-proposal. But you list a bunch of rejected >> alternatives; this could be in that list. >> >> - Is @@ really necessary? It seems you are adding it mostly because >> it's cute and because of the parallel with **, not because it is >> actually important enough to add new syntax. And then later you use >> it as an argument for @, which seems a bit circular. Also, if we >> were to make @ right-associative, the parallel with ** is already >> imperfect. >> >> - For better counts of usages, perhaps Sourcegraph.com might help? It >> is a source code query engine that has a Python parser and (limited) >> type inference built in (also separately available as pysonar on >> github IIRC). To be clear, I don't need more numbers to be convinced. >> >> Once we've decided on associativity and @@, I'm ready to accept. >> >> -- >> --Guido van Rossum (python.org/~guido) >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/317338b9/attachment-0001.html> From storchaka at gmail.com Fri Mar 14 21:17:58 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 14 Mar 2014 22:17:58 +0200 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <lfvo4j$43j$1@ger.gmane.org> 14.03.14 03:59, Nathaniel Smith ???????(??): > PEP: XXXX > Title: Dedicated infix operators for matrix multiplication and matrix power This is about matrix multiplication and matrix power. But what about matrix division? It is needed to distinguish elementwise division from left and right matrix divisions. And what about concatenation? Python lists and tuples use ``+`` for this, but NumPy arrayse use ``+`` for bitwise addition. Matrix multiplication is only defined on 2d arrays (matrices). But why not on arbitrary tensors (except scalars)? From ncoghlan at gmail.com Fri Mar 14 21:46:26 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Mar 2014 06:46:26 +1000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfvo4j$43j$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <lfvo4j$43j$1@ger.gmane.org> Message-ID: <CADiSq7dLVn8AJeLFLw8iTHZaitBJ_v7BbYnV7yJDUr9ngyXnxA@mail.gmail.com> On 15 Mar 2014 06:18, "Serhiy Storchaka" <storchaka at gmail.com> wrote: > > 14.03.14 03:59, Nathaniel Smith ???????(??): > >> PEP: XXXX >> Title: Dedicated infix operators for matrix multiplication and matrix power > > > This is about matrix multiplication and matrix power. But what about matrix division? It is needed to distinguish elementwise division from left and right matrix divisions. And what about concatenation? Python lists and tuples use ``+`` for this, but NumPy arrayse use ``+`` for bitwise addition. > > Matrix multiplication is only defined on 2d arrays (matrices). But why not on arbitrary tensors (except scalars)? These are covered in the PEP - the relatively simple "just a new matrix multiplication operator" proposal is due to the fact that, after long experience, matrix multiplication is the only operator the numeric community really feel they miss in Python compared to working in more special purpose languages like Matlab or R. Everything else can continue to use appropriately named methods and functions, just as it does today. That does raise a question for me though: does Julia have syntax for matrix multiplication? If so, what does that look like? Cheers, Nick. > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/7a0768f0/attachment.html> From skip at pobox.com Fri Mar 14 21:51:10 2014 From: skip at pobox.com (Skip Montanaro) Date: Fri, 14 Mar 2014 15:51:10 -0500 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> Message-ID: <CANc-5Uw+piJx9hFj4VU0h2wxNAm3DiCWWhKFjzk=RNRiKp3XYQ@mail.gmail.com> On Fri, Mar 14, 2014 at 3:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > The method name for "*" is "__mul__" rather than "__star__" Yes, but ?just think. W ith a couple extra "@"s we could ?start defining __ratatatat__<http://www.merriam-webster.com/audio.php?file=ratata02&word=rat-a-tat-tat&text=%5C%CB%8Cra-t%C9%99-%CB%8Cta(t)-%CB%88tat%5C> ? methods.? ?? :-) Skip -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/d12ac5b4/attachment.html> From anthony at xtfx.me Fri Mar 14 22:03:17 2014 From: anthony at xtfx.me (C Anthony Risinger) Date: Fri, 14 Mar 2014 16:03:17 -0500 Subject: [Python-ideas] Function-like Modules (better control/introspection) Message-ID: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> hello, could we make modules work more like this... https://gist.github.com/xtfxme/9556806 tl;dr: __code__ object available (potentially a __self__ object), reload by calling, makes working with modules more natural/safe! ENCLOSED: modules-next.py -- C Anthony #----------------------------------------------------------------------------------------( modules-next.py ) # encoding: utf-8 # # modules-next # # C Anthony Risinger import sys m_name = 'modules_next' m_source = r"""\ if __name__ == '{0}': from . import __call__ as reexec from . import __code__ as code from . import __dict__ as ns print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) __name__ = 'modules_next_reloaded' reexec() print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) else: print('!!! __name__ == {{0}}\n'.format(__name__)) def throw(): raise Exception(__name__) """.format(m_name) m_code = compile(m_source, m_name, 'exec') m_dict = { '__file__': __file__ + '.next', '__name__': m_name, '__path__': list(), '__doc__': None, } #...bind namespace/functions module = eval('lambda: None', m_dict) #...automatically expose as attributes module.__dict__ = module.__globals__ #...must be set twice, __code__ sees __dict__, but we see descriptor module.__name__ = m_name #...redefine the function body! module.__code__ = m_code #...yay! we haz legitimate module! sys.modules[m_name] = module print('--* importing...') import modules_next print('>>> {0}: {1}\n{2}\n'.format( __name__, modules_next.__code__, modules_next.__dict__.keys(), )) print('--* importing (for real this time)...') #...import/reload by calling our func-y-module module() print('>>> {0}: {1}\n{2}\n'.format( __name__, modules_next.__code__, modules_next.__dict__.keys(), )) print('--* throwing...') modules_next.throw() -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c8c8bb6b/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: modules-next.py Type: text/x-python Size: 1521 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/c8c8bb6b/attachment-0001.py> From njs at pobox.com Fri Mar 14 22:21:12 2014 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 14 Mar 2014 21:21:12 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> Message-ID: <CAPJVwBmm+WDCpsRDAVOZ=YgRX5u7HfWZ+Cs2NWdHev9_v8_PsQ@mail.gmail.com> On Fri, Mar 14, 2014 at 5:53 PM, Guido van Rossum <guido at python.org> wrote: > I have now read the PEP, and I think it's good. I think it's a waste of time > to keep bikeshedding on the choice of operator -- @ is the best compromise. > I do have a few specific notes: > > - Right associativity is not unheard of in Python. E.g. **. If you > think that for other reasons @ should be right associative, don't > let Python's tradition stop you. But then you need to decide which > of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or > (a*b)@c? And if you choose the latter, it follows that a at b*c means > a@(b*c) -- is that okay? (And similar examples exist for the other > choice.) Like Robert I have the suspicion that the best option is to make @ right-associative and place it just above (more tightly binding) than *. But I'll poll numpy-discussion and friends and see if anyone has ideas for more objective measures. > - Did you consider a duck-typing (is that the word?) attribute? > E.g. a*b is elementwise multiplication; a.M*b must be used for > matrix multiplication. (Your use of .T as "transpose" made me think > of this.) Of course the question is, can you get those packages > that currently use * for matrix multiply to comply? (I don't consider > this a serious counter-proposal. But you list a bunch of rejected > alternatives; this could be in that list. This is an interesting suggestion! I think it hasn't received full consideration before because the tangle between numpy.ndarray and numpy.matrix means that the most obvious implementation for an ndarray.M attribute would be to return a full-fledged numpy.matrix object... and no-one wants to encourage proliferation of numpy.matrix objects. But returning a tiny special-purpose class that only implements __mul__ would avoid that objection. At that point it's basically a nicer version of the "user-defined infix" '*dot*' operator idea. It has similar problems with needing an unaesthetically magical implementation, producing a proliferation of special classes (one extra one for each array type), requiring an allocation on every call, etc., but this might well be the next-best thing to a real operator. All in all, I think we'd rather have a real operator, so if you're happy to go with @ then I won't put lots of effort into finding horrible problems that force us to reject .M ;-), but I'll certainly add it to the PEP in any case. > - Is @@ really necessary? It seems you are adding it mostly because > it's cute and because of the parallel with **, not because it is > actually important enough to add new syntax. And then later you use > it as an argument for @, which seems a bit circular. Also, if we > were to make @ right-associative, the parallel with ** is already > imperfect. @@ hasn't received as much attention as the rest of the proposal, so I'll check in with numpy-discussion etc. on this as well. But my personal feeling is +0 on @@ -- all else being equal, it's nicer to have it than not. Is all else equal? Given the existence of @, the increase in language complexity is small (or arguably negative, since people who know *, **, @ may be surprised to find @@ missing, and have to memorize its non-existence as an extra rule); the opportunity cost is low (given the existence of @ I can't imagine we'll want to use the token @@ for something else later); and @@ will be used in real life, if not as often as @ itself -- 'vec @@ 2' for squared Euclidean length probably won't be too uncommon, 'matrix @@ n' gets used for things like simulating random walks on graphs or other markov chains, and the 'matrix @@ -1' notation will probably be a nice win for beginners and other less-sophisticated programmers who just want to get some computation done in a way that doesn't require them to keep a more complicated mapping between math and code notation in their head or caring about numerical details. I think of @@ being like, say, '%=' -- nothing one would ever add syntax for in isolation, but if designing an overall system of operators then it makes more sense. Probably in the end it just comes down to your aesthetic judgement :-). Numeric folk will be fine either way. (And I wouldn't consider @@ a *strong* argument for spelling '@' as '@'; it's just mentioned in that section because there aren't really *any* strong arguments for preferring one spelling versus another, we have to make a decision, and so a weak argument is better than nothing. Two useful operators for the complexity cost of one is a good deal? :shrug: Obviously if @@ gets dropped I'll drop that bullet point as well.) > - For better counts of usages, perhaps Sourcegraph.com might help? It > is a source code query engine that has a Python parser and (limited) > type inference built in (also separately available as pysonar on > github IIRC). To be clear, I don't need more numbers to be convinced. Oo, that is shiny, thanks for the tip. > Once we've decided on associativity and @@, I'm ready to accept. Wonderful to hear! Thanks for giving this so much time and attention (on no warning)! -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From ronan.lamy at gmail.com Fri Mar 14 23:01:54 2014 From: ronan.lamy at gmail.com (Ronan Lamy) Date: Fri, 14 Mar 2014 22:01:54 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfvo4j$43j$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <lfvo4j$43j$1@ger.gmane.org> Message-ID: <53237C52.80600@gmail.com> Le 14/03/14 20:17, Serhiy Storchaka a ?crit : > Matrix multiplication is only defined on 2d arrays (matrices). But why > not on arbitrary tensors (except scalars)? The PEP does include a (complicated) definition of "matrix multiplication" for arrays of all ranks. In particular, @-multiplication of rank-1 arrays is the dot product. From greg.ewing at canterbury.ac.nz Fri Mar 14 22:46:33 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 10:46:33 +1300 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <532378B9.2040707@canterbury.ac.nz> +1000! This all makes total sense. -- Greg From steve at pearwood.info Fri Mar 14 23:48:12 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Mar 2014 09:48:12 +1100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> Message-ID: <20140314224812.GR12595@ando> On Fri, Mar 14, 2014 at 01:59:14AM +0000, Nathaniel Smith wrote: > This PEP proposes two new binary operators dedicated to matrix > multiplication and matrix power, spelled ``@`` and ``@@`` > respectively. (Mnemonic: ``@`` is ``*`` for mATrices.) When I first started reading the PEP, I was rather dubious about the choice of @ as operator, but it surprised me at how quickly I got used to it. I suppose because unconsciously I associated it with the common shorthand implying (scalar) multiplication: 5kg of apples @ $2 per kg costs $10.00 so it didn't take me very long to warm to it. +1 -- Steven From greg.ewing at canterbury.ac.nz Sat Mar 15 00:16:25 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:16:25 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314112931.4c38de75@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <20140314112931.4c38de75@fsol> Message-ID: <53238DC9.1010908@canterbury.ac.nz> Antoine Pitrou wrote: > Or simply implement __invert__, so you can write it "~ A". > (__invert__ doesn't really invert a number, it takes the bitwise > complement :-)) But that's already taken to mean elementwise bitwise complement. -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 15 00:27:11 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:27:11 +1300 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> Message-ID: <5323904F.1000008@canterbury.ac.nz> Antony Lee wrote: > A simple suggestion: what about defining the special methods as __at__, > __rat__, __iat__ Please, no -- the last thing we need in Python is iyats! http://www.paravia.com/wiki/index.php5?title=Iyat -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 15 00:29:24 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:29:24 +1300 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314190009.GA19829@phdru.name> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CAP7+vJ+++US+rQS-2kMHWkgjY8uN6rR3jgzkNRCAg2XQx1Hu=g@mail.gmail.com> <20140314190009.GA19829@phdru.name> Message-ID: <532390D4.2000508@canterbury.ac.nz> Oleg Broytman wrote: > On Fri, Mar 14, 2014 at 11:53:47AM -0700, Guido van Rossum <guido at python.org> wrote: > >>Ratatatatatat! :-) > > Is this the sound made by matrices being multiplied? ;-) I think it's the sound of a PEP idea being lined up against the wall and shot. :-( -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 15 00:41:23 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:41:23 +1300 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314224812.GR12595@ando> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> Message-ID: <532393A3.9010005@canterbury.ac.nz> Steven D'Aprano wrote: > I suppose because unconsciously I associated it with the common > shorthand implying (scalar) multiplication: > > 5kg of apples @ $2 per kg costs $10.00 That analogy actually extends to the matrix case as well. E.g. if you have a vector q of quantities and a vector p of prices, then q @ p is the total price. -- Greg From solipsis at pitrou.net Sat Mar 15 00:50:32 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 00:50:32 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> Message-ID: <20140315005032.773acc95@fsol> On Sat, 15 Mar 2014 12:41:23 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Steven D'Aprano wrote: > > I suppose because unconsciously I associated it with the common > > shorthand implying (scalar) multiplication: > > > > 5kg of apples @ $2 per kg costs $10.00 > > That analogy actually extends to the matrix case as well. > E.g. if you have a vector q of quantities and a vector > p of prices, then q @ p is the total price. It depends how the vectors are layed out (horizontal @ vertical or vertical @ horizontal). Regards Antoine. From greg.ewing at canterbury.ac.nz Sat Mar 15 00:55:38 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:55:38 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315005032.773acc95@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> Message-ID: <532396FA.7010601@canterbury.ac.nz> Antoine Pitrou wrote: > It depends how the vectors are layed out (horizontal @ vertical or > vertical @ horizontal). The definition of @ in the proposal is such that two 1D arrays is interpreted as horizontal @ vertical. -- Greg From ncoghlan at gmail.com Sat Mar 15 01:04:46 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 15 Mar 2014 10:04:46 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <532396FA.7010601@canterbury.ac.nz> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> Message-ID: <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com> On 15 Mar 2014 09:56, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote: > > Antoine Pitrou wrote: > >> It depends how the vectors are layed out (horizontal @ vertical or >> vertical @ horizontal). > > > The definition of @ in the proposal is such that two > 1D arrays is interpreted as horizontal @ vertical. Oh, nice! Maybe the PEP should include implementing that for lists and our other 1D sequence types? Also, I just remembered another minor error in the PEP - as of Python 3.3, memoryview.cast() allows the creation of multidimensional views, so the PEP is incorrect in saying that can't be done with the builtins/stdlib. Cheers, Nick. > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/9f6609b4/attachment.html> From solipsis at pitrou.net Sat Mar 15 01:09:44 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 01:09:44 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> Message-ID: <20140315010944.6b8244e1@fsol> On Sat, 15 Mar 2014 12:55:38 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Antoine Pitrou wrote: > > > It depends how the vectors are layed out (horizontal @ vertical or > > vertical @ horizontal). > > The definition of @ in the proposal is such that two > 1D arrays is interpreted as horizontal @ vertical. Really? That should be up to the third-party library implementing the @ operator for its types, not to the language itself: Python _suggests_ an use case for @, it doesn't mandate it (especially as there's no appropriate data type in the stdlib). Regards Antoine. From njs at pobox.com Sat Mar 15 01:25:07 2014 From: njs at pobox.com (Nathaniel Smith) Date: Sat, 15 Mar 2014 00:25:07 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315010944.6b8244e1@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> Message-ID: <CAPJVwB=YZeM-kYr-7w=pMWaScCzz2NGNMwAAfLcO8=OzBHnKvA@mail.gmail.com> On Sat, Mar 15, 2014 at 12:09 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sat, 15 Mar 2014 12:55:38 +1300 > Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> Antoine Pitrou wrote: >> >> > It depends how the vectors are layed out (horizontal @ vertical or >> > vertical @ horizontal). >> >> The definition of @ in the proposal is such that two >> 1D arrays is interpreted as horizontal @ vertical. > > Really? That should be up to the third-party library implementing the @ > operator for its types, not to the language itself: Python _suggests_ > an use case for @, it doesn't mandate it (especially as there's no > appropriate data type in the stdlib). See: http://legacy.python.org/dev/peps/pep-0465/#intended-usage-details Which begins: "This section is informative, rather than normative -- it documents the consensus of a number of libraries that provide array- or matrix-like objects on how the @ and @@ operators will be implemented. ..." -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From greg.ewing at canterbury.ac.nz Sat Mar 15 01:27:22 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 13:27:22 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315010944.6b8244e1@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> Message-ID: <53239E6A.6020205@canterbury.ac.nz> Antoine Pitrou wrote: > On Sat, 15 Mar 2014 12:55:38 +1300 > Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > >>The definition of @ in the proposal is such that two >>1D arrays is interpreted as horizontal @ vertical. > > Really? That should be up to the third-party library implementing the @ > operator for its types, It is. It's described as "recommended semantics" in the PEP, not something defined by the language. -- Greg From asmeurer at gmail.com Sat Mar 15 01:48:46 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 14 Mar 2014 17:48:46 -0700 (PDT) Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <3bea8fbc-64b5-42c1-93b6-2d6faf7562c5@googlegroups.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAPJVwBmm+WDCpsRDAVOZ=YgRX5u7HfWZ+Cs2NWdHev9_v8_PsQ@mail.gmail.com> <3bea8fbc-64b5-42c1-93b6-2d6faf7562c5@googlegroups.com> Message-ID: <9e2519ad-0940-478c-bff0-39f47ab1e826@googlegroups.com> On Friday, March 14, 2014 7:46:11 PM UTC-5, Aaron Meurer wrote: On Friday, March 14, 2014 4:21:12 PM UTC-5, Nathaniel Smith wrote: > On Fri, Mar 14, 2014 at 5:53 PM, Guido van Rossum <gu... at python.org> > wrote: > >> > I have now read the PEP, and I think it's good. I think it's a waste of > time > >> > to keep bikeshedding on the choice of operator -- @ is the best > compromise. > >> > I do have a few specific notes: > >> > > >> > - Right associativity is not unheard of in Python. E.g. **. If you > >> > think that for other reasons @ should be right associative, don't > >> > let Python's tradition stop you. But then you need to decide which > >> > of * and @ binds more tightly -- e.g. does a*b at c mean a*(b at c) or > >> > (a*b)@c? And if you choose the latter, it follows that a at b*c means > >> > a@(b*c) -- is that okay? (And similar examples exist for the other > >> > choice.) > >> > Like Robert I have the suspicion that the best option is to make @ > >> right-associative and place it just above (more tightly binding) than > >> *. But I'll poll numpy-discussion and friends and see if anyone has > ideas for more objective measures. > I guess a common case would be a*B at x, where a is a scalar. Is it more efficient or numerically stable to evaluate that one way or the other? Aaron Meurer > > - Did you consider a duck-typing (is that the word?) attribute? > >> > E.g. a*b is elementwise multiplication; a.M*b must be used for > >> > matrix multiplication. (Your use of .T as "transpose" made me think > >> > of this.) Of course the question is, can you get those packages > >> > that currently use * for matrix multiply to comply? (I don't consider > >> > this a serious counter-proposal. But you list a bunch of rejected > >> > alternatives; this could be in that list. > >> > This is an interesting suggestion! I think it hasn't received full > >> consideration before because the tangle between numpy.ndarray and > >> numpy.matrix means that the most obvious implementation for an > >> ndarray.M attribute would be to return a full-fledged numpy.matrix > >> object... and no-one wants to encourage proliferation of numpy.matrix > >> objects. But returning a tiny special-purpose class that only > >> implements __mul__ would avoid that objection. At that point it's > >> basically a nicer version of the "user-defined infix" '*dot*' operator > >> idea. It has similar problems with needing an unaesthetically magical > >> implementation, producing a proliferation of special classes (one > >> extra one for each array type), requiring an allocation on every call, > >> etc., but this might well be the next-best thing to a real operator. > >> > All in all, I think we'd rather have a real operator, so if you're > >> happy to go with @ then I won't put lots of effort into finding > >> horrible problems that force us to reject .M ;-), but I'll certainly > >> add it to the PEP in any case. > >> > > - Is @@ really necessary? It seems you are adding it mostly because > >> > it's cute and because of the parallel with **, not because it is > >> > actually important enough to add new syntax. And then later you use > >> > it as an argument for @, which seems a bit circular. Also, if we > >> > were to make @ right-associative, the parallel with ** is already > >> > imperfect. > >> > @@ hasn't received as much attention as the rest of the proposal, so > >> I'll check in with numpy-discussion etc. on this as well. But my > >> personal feeling is +0 on @@ -- all else being equal, it's nicer to > >> have it than not. Is all else equal? Given the existence of @, the > >> increase in language complexity is small (or arguably negative, since > >> people who know *, **, @ may be surprised to find @@ missing, and have > >> to memorize its non-existence as an extra rule); the opportunity cost > >> is low (given the existence of @ I can't imagine we'll want to use the > >> token @@ for something else later); and @@ will be used in real life, > >> if not as often as @ itself -- 'vec @@ 2' for squared Euclidean length > >> probably won't be too uncommon, 'matrix @@ n' gets used for things > >> like simulating random walks on graphs or other markov chains, and the > >> 'matrix @@ -1' notation will probably be a nice win for beginners and > >> other less-sophisticated programmers who just want to get some > >> computation done in a way that doesn't require them to keep a more > >> complicated mapping between math and code notation in their head or > >> caring about numerical details. I think of @@ being like, say, '%=' -- > >> nothing one would ever add syntax for in isolation, but if designing > >> an overall system of operators then it makes more sense. > >> > Probably in the end it just comes down to your aesthetic judgement > >> :-). Numeric folk will be fine either way. > >> > (And I wouldn't consider @@ a *strong* argument for spelling '@' as > >> '@'; it's just mentioned in that section because there aren't really > >> *any* strong arguments for preferring one spelling versus another, we > >> have to make a decision, and so a weak argument is better than > >> nothing. Two useful operators for the complexity cost of one is a good > >> deal? :shrug: Obviously if @@ gets dropped I'll drop that bullet point > >> as well.) > >> > > - For better counts of usages, perhaps Sourcegraph.com might help? It > >> > is a source code query engine that has a Python parser and (limited) > >> > type inference built in (also separately available as pysonar on > >> > github IIRC). To be clear, I don't need more numbers to be convinced. > >> > Oo, that is shiny, thanks for the tip. > >> > > Once we've decided on associativity and @@, I'm ready to accept. > >> > Wonderful to hear! Thanks for giving this so much time and attention > >> (on no warning)! > >> > -n > >> > -- > >> Nathaniel J. Smith > >> Postdoctoral researcher - Informatics - University of Edinburgh > >> http://vorpus.org > >> _______________________________________________ > >> Python-ideas mailing list > >> Python... at python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/161d878b/attachment-0001.html> From greg.ewing at canterbury.ac.nz Sat Mar 15 00:16:01 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 15 Mar 2014 12:16:01 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> Message-ID: <53238DB1.7060608@canterbury.ac.nz> Nathan Schneider wrote: > How about prefixing * with an innocuous backtick? Just one little backtick... it's wafer-thin... -- Greg From cjwelborn at live.com Sat Mar 15 03:07:18 2014 From: cjwelborn at live.com (Christopher Welborn) Date: Fri, 14 Mar 2014 21:07:18 -0500 Subject: [Python-ideas] for-loop-if like list comps have? Message-ID: <lg0ck8$nnh$1@ger.gmane.org> mylist = ['apple', 'banana', 'orange', 'grapefruit'] # Why is this okay... for item in [x for x in mylist if 'p' in x]: print(item) # But this isn't? for item in mylist if 'p' in item: SyntaxError: invalid syntax # Instead you have to do another nesting level.. for item in mylist: if 'p' in item: print(item) # Or another way. for item in mylist: if 'p' not in item: continue print(item) ...And there is 'filter', 'lambdas', and whatever else to achieve the same result. But the list comprehension is already closely related to a for-loop, and it seems natural to go from: [x for x in mylist if x] to: for x in mylist if x: Just an idea, sorry if it has been mentioned before. -- \?\ /?/\ \ \/??\/ / / Christopher Welborn (cj) \__/\__/ / cjwelborn at live?com \__/\__/ http://welbornprod.com From steve at pearwood.info Sat Mar 15 03:37:48 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Mar 2014 13:37:48 +1100 Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <lg0ck8$nnh$1@ger.gmane.org> References: <lg0ck8$nnh$1@ger.gmane.org> Message-ID: <20140315023747.GA16526@ando> On Fri, Mar 14, 2014 at 09:07:18PM -0500, Christopher Welborn wrote: > mylist = ['apple', 'banana', 'orange', 'grapefruit'] > > # Why is this okay... > for item in [x for x in mylist if 'p' in x]: > print(item) > > # But this isn't? > for item in mylist if 'p' in item: > SyntaxError: invalid syntax This suggestion has been made, oh, a million times -- not that I would ever exaggerate... *wink* -- and has been rejected each time. Personally, I don't think it gains you too much, all you save is one line and one nesting level, and if you're nested so deeply that this becomes important, your code is probably in desperate need of refactoring. The downside includes additional complication. It's one more thing for people to learn. Not only do you have to learn that there are two independent features, the for-statement and the if-statement, which being independent can be combined however you like: # 1 for x in seq: if cond: ... #2 if cond: for x in seq: ... but now you also have to learn a *third* form: for x in seq if cond: ... and learn that it is the same as #1 above rather than #2 above. And that there *isn't* a fourth form: if cond for x in seq: ... or other combinations like: while x if cond: if cond while x: if cond try: for x try: for x if y try: etc. The way statements work in Python is that you combine them by using a new line and starting a new block, not by having special syntactic forms for each combination. List comprehensions are different, because they don't create a new block, and they are an expression not a statement. -- Steven From abarnert at yahoo.com Sat Mar 15 03:43:32 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 14 Mar 2014 19:43:32 -0700 Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <lg0ck8$nnh$1@ger.gmane.org> References: <lg0ck8$nnh$1@ger.gmane.org> Message-ID: <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com> On Mar 14, 2014, at 19:07, Christopher Welborn <cjwelborn at live.com> wrote: > mylist = ['apple', 'banana', 'orange', 'grapefruit'] > > # Why is this okay... > for item in [x for x in mylist if 'p' in x]: > print(item) > > # But this isn't? > for item in mylist if 'p' in item: > SyntaxError: invalid syntax > > # Instead you have to do another nesting level.. > for item in mylist: > if 'p' in item: > print(item) > > # Or another way. > for item in mylist: > if 'p' not in item: > continue > print(item) > > ...And there is 'filter', 'lambdas', and whatever else > to achieve the same result. But the list comprehension > is already closely related to a for-loop, and it seems > natural to go from: > [x for x in mylist if x] > to: > for x in mylist if x: > > Just an idea, sorry if it has been mentioned before. In a listcomp, each for or if is a separate clause, and they can be nested any way you want (except for the restriction that the first clause has to be a for). So, you can do this: [x for xs in xss for x in xs] [x for x in xs if x if x-1] [x for xs in xss if xs for x in xs if x] Should you be able to write all of those as one-liner statements? And this isn't just a weird quirk of syntax; each clause is conceptually exactly the same as a nested statement (except for some minor differences with the first one); that's how comprehensions are defined in the language reference. Not all languages do comprehensions this way; Clojure, Racket, etc. have only for loop clauses in their comprehensions, but every for loop clause has an optional if phrase. Which would make your translation obviously right, instead of wrong for a hard to notice reason. (And it would make comprehensions slightly easier to explain. And it would also make it easier to add things like a while phrase, which people keep asking for.) But that would be a pretty big change for Python at this point. From bruce at leapyear.org Sat Mar 15 05:09:02 2014 From: bruce at leapyear.org (Bruce Leban) Date: Fri, 14 Mar 2014 21:09:02 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <53238DB1.7060608@canterbury.ac.nz> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> Message-ID: <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> I think making @ right associative would make this less suitable for other uses. Example someclassobject @ somevalue @ somevalue won't work with right-associativity. Plus precedence of mixing right and left can be confusing. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140314/7594b8b5/attachment.html> From steve at pearwood.info Sat Mar 15 08:35:09 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Mar 2014 18:35:09 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> Message-ID: <20140315073508.GB16526@ando> On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote: > I think making @ right associative would make this less suitable for other > uses. Example > > someclassobject @ somevalue @ somevalue > > won't work with right-associativity. Why not? It works with the other right-associative operator: x**y**z "Works" depends on what you expect it to do. Unless you tell us what these "other uses" are, how can we know that right-associativity won't work? > Plus precedence of mixing right and left can be confusing. Perhaps. Perhaps not. This works as expected: 3*2**5 Are there any other right-associative operators that are confusing? -- Steven From solipsis at pitrou.net Sat Mar 15 12:27:33 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 12:27:33 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> Message-ID: <20140315122733.35ed7909@fsol> On Sat, 15 Mar 2014 18:35:09 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote: > > I think making @ right associative would make this less suitable for other > > uses. Example > > > > someclassobject @ somevalue @ somevalue > > > > won't work with right-associativity. > > Why not? It works with the other right-associative operator: > > x**y**z > > "Works" depends on what you expect it to do. The real question is why @ would be right-associative. "**" is very rarely used in a chained manner as the above, so its associativity isn't really important (I don't think I have ever written "x**y**z"). @ will be used routinely in a chained manner, so the question is more important here. The possible reason given in the PEP is very weak and amounts to premature optimization: """It's been suggested that @ should be right-associative, on the grounds that for expressions like Mat @ Mat @ vec, the two different evaluation orders produce the same result, but the right-associative order Mat @ (Mat @ vec) will be faster and use less memory than the left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication is much cheaper than matrix-matrix multiplication).""" If that's the only reason, then I'd like @ to be left-associative. Regards Antoine. From robert.kern at gmail.com Sat Mar 15 12:47:48 2014 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 15 Mar 2014 11:47:48 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfvo4j$43j$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <lfvo4j$43j$1@ger.gmane.org> Message-ID: <lg1ekl$11j$1@ger.gmane.org> On 2014-03-14 20:17, Serhiy Storchaka wrote: > 14.03.14 03:59, Nathaniel Smith ???????(??): >> PEP: XXXX >> Title: Dedicated infix operators for matrix multiplication and matrix power > > This is about matrix multiplication and matrix power. But what about matrix > division? It is needed to distinguish elementwise division from left and right > matrix divisions. In our experience, matrix division comes up rather more rarely. For numerical problems, these should typically be implemented with a linear solve of some type. The best kind of solve varies based on the problem, so that's a good reason to continue to implement it with a function call rather than an operator. For symbolic problems (a la Sympy), both forms can be spelled reasonably well given matrix multiplication and matrix power. > And what about concatenation? Python lists and tuples use > ``+`` for this, but NumPy arrayse use ``+`` for bitwise addition. There are *lots* of ways to concatenate an n-dimensional array, and we have a whole complement of functions to do that, vstack(), hstack(), dstack(), concatenate(), r_[], c_[]. A single operator would not suffice. > Matrix multiplication is only defined on 2d arrays (matrices). But why not on > arbitrary tensors (except scalars)? The PEP defines how numpy intends to interpret n-dimensional operands and the reasons why. For tensors qua tensors you tend to want to do arbitrary Einstein-summation, which is hard to do in a single operator. numpy.einsum() handles that for us. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From greg.ewing at canterbury.ac.nz Sat Mar 15 12:55:09 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Mar 2014 00:55:09 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315122733.35ed7909@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> Message-ID: <53243F9D.1060802@canterbury.ac.nz> Antoine Pitrou wrote: > The possible reason given in the PEP is very weak and amounts to > premature optimization: I don't think it's just a matter of optimization. Often, matrix @ vector represents a linear operator acting on an element of a vector space. When you chain them, A @ B @ C @ v conceptually represents acting on v with C, then B, then A. -- Greg From oscar.j.benjamin at gmail.com Sat Mar 15 13:03:10 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 15 Mar 2014 12:03:10 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <CADiSq7ewkK8FeXfVxtfgvxgPbKD94m7UVqWua6YQ+sRpJVo5+Q@mail.gmail.com> Message-ID: <CAHVvXxQ4JbriLpn0NxHZnJgbNF6dWZY-wuemUD_DQZS1s=uuWQ@mail.gmail.com> On 15 March 2014 00:04, Nick Coghlan <ncoghlan at gmail.com> wrote: > > On 15 Mar 2014 09:56, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote: >> >> Antoine Pitrou wrote: >> >>> It depends how the vectors are layed out (horizontal @ vertical or >>> vertical @ horizontal). >> >> >> The definition of @ in the proposal is such that two >> 1D arrays is interpreted as horizontal @ vertical. > > Oh, nice! Maybe the PEP should include implementing that for lists and our > other 1D sequence types? I don't think that that should be included in the PEP. It's unlikely that many users will want that and if they do it will be subject to bike-shedding of issues that aren't directly relevant to the problem that this PEP tries to solve. It'll get brought up on python-ideas some time in the next couple of years and then interested parties can bike-shed over it separately (with the benefit of an existing implementation of the operators in numpy for comparison). > Also, I just remembered another minor error in the PEP - as of Python 3.3, > memoryview.cast() allows the creation of multidimensional views, so the PEP > is incorrect in saying that can't be done with the builtins/stdlib. I'm getting a bit of d?j? vu here: https://mail.python.org/pipermail/python-dev/2013-September/128463.html Has this changed since then? I'd like to know how to use memoryview to create a 2D matrix of integers or floats. Also even if it is possible to create a 2D view in the stdlib I don't think that the stdlib should try so support matrix multiplication unless it is prepared to try and do so efficiently. Numpy delegates this task to the underlying BLAS library which will usually be not just asymptotically more efficient than a na?ve algorithm but also heavily micro-optimised. I doubt that the Python core wants to grow a dependency on having a BLAS library so unless someone wants to reimplement this part of one it's best to leave it out. A big +1 on the PEP from me. I think this brings numpy on a par with Matlab for linear algebra. Matlab also has other matrix related operators such as matrix left and right divide with \ and / but I think that these are triviallising what should be understood as non-trivial operations and are also overloaded to mean too many different things (given different array shapes). Similarly Matlab has made a big mistake in having the default multiplication operator be matrix multiplication which creates a significant pitfall for new users and becomes a common source of problems that people have to work through every time they start debugging their code (it's just a time waster really). Some people have complained that the @ symbol looks clunky. Personally I think it's important that this operator should be very distinctive. Using an innocent-looking operator for matrix multiplication has been tried before and based on ~7 years of teaching Matlab I would say that it's a bad idea. Oscar From solipsis at pitrou.net Sat Mar 15 13:06:31 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 13:06:31 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> Message-ID: <20140315130631.6f97ec16@fsol> On Sun, 16 Mar 2014 00:55:09 +1300 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Antoine Pitrou wrote: > > > The possible reason given in the PEP is very weak and amounts to > > premature optimization: > > I don't think it's just a matter of optimization. Often, > matrix @ vector represents a linear operator acting on an > element of a vector space. When you chain them, > > A @ B @ C @ v > > conceptually represents acting on v with C, then B, > then A. It can just as well represent "acting" on v with (A @ B @ C). Of course, mathematically it shouldn't make a difference, but in computer programming right-associative operators are always a special case, and therefore an additional cognitive burden. Regards Antoine. From daoust.mj at gmail.com Sat Mar 15 13:20:59 2014 From: daoust.mj at gmail.com (Mark Daoust) Date: Sat, 15 Mar 2014 08:20:59 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power Message-ID: <CAFg8gYz1Z27WPkCzqfy+=cCiH1q4KGy=RX1Fg3Rs1=xqkkZNzg@mail.gmail.com> *>Aaron Meurer* asmeurer at gmail.com <python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20Fwd%3A%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%0A%20for%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3C9e2519ad-0940-478c-bff0-39f47ab1e826%40googlegroups.com%3E> > *>Sat Mar 15 01:48:46 CET 2014* > > > Like Robert I have the suspicion that the best option is to make @ > > right-associative and place it just above (more tightly binding) than > > > > *. But I'll poll numpy-discussion and friends and see if anyone has > > ideas for more objective measures. > > > > I guess a common case would be a*B at x, where a is a scalar. Is it more > efficient or numerically stable to evaluate that one way or the other? I'm not a numerical linear algebra expert, but I disagree. If your matrix code needs to be optimized, there's no way around thinking about your specific situation. Optimizing matrix expressions is a tricky problem. <http://en.wikipedia.org/wiki/Matrix_chain_multiplication> http://en.wikipedia.org/wiki/Matrix_chain_multiplication http://www.youtube.com/watch?v=nVt24G_2VC0 If you want a different order use parenthesis, or build a symbolic expression. I don't think either choice of priority, or associativity will be the right often enough be worth an affront to the "principal of least surprise". Things are nice and clear the way they are. >From the docs http://docs.python.org/3.4/reference/expressions.html#binary-arithmetic-operations : """The binary arithmetic operations have the conventional priority levels... Apart from the power operator[s], there are only two levels, one for multiplicative operators and one for additive operators""" let's keep it like this: Vote P/E/MD/AS not P/E/M/MD/AS. I think that if there is a reason to change is Greg is on the right track: *>Greg Ewing* greg.ewing at canterbury.ac.nz <python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%20for%0A%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3C53243F9D.1060802%40canterbury.ac.nz%3E> *>Sat Mar 15 12:55:09 CET 2014* >Often,matrix @ vector represents a linear operator acting on an element of a vector space. When you chain them, > A @ B @ C @ v >conceptually represents acting on v with C, then B, then A. PS: >*Nathaniel Smith* njs at pobox.com <python-ideas%40python.org?Subject=Re%3A%20%5BPython-ideas%5D%20Fwd%3A%20%5BRFC%5D%20draft%20PEP%3A%20Dedicated%20infix%20operators%0A%20for%20matrix%20multiplication%20and%20matrix%20power&In-Reply-To=%3CCAPJVwBmm%2BWDCpsRDAVOZ%3DYgRX5u7HfWZ%2BCs2NWdHev9_v8_PsQ%40mail.gmail.com%3E> >*Fri Mar 14 22:21:12 CET 2014* > >The 'matrix @@ -1' notation will probably be a nice win for >beginners and other less-sophisticated programmers ... There's a spot where I like the Facade approach: return an object that only has one method, numpy.solve as __mul__. Make it hard to do badly. >[x.M * y] This is an interesting suggestion! I think it hasn't received full consideration before ... >but this might well be the next-best thing to a real operator. Maybe It is a real operator, it's just spelled ".M*" instead of "@" ;) Thanks for all the work on this Nathaniel. Mark Daoust -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/96bcf9e6/attachment-0001.html> From oscar.j.benjamin at gmail.com Sat Mar 15 13:20:42 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 15 Mar 2014 12:20:42 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315130631.6f97ec16@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> Message-ID: <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> On 15 March 2014 12:06, Antoine Pitrou <solipsis at pitrou.net> wrote: >> > The possible reason given in the PEP is very weak and amounts to >> > premature optimization: >> >> I don't think it's just a matter of optimization. Often, >> matrix @ vector represents a linear operator acting on an >> element of a vector space. When you chain them, >> >> A @ B @ C @ v >> >> conceptually represents acting on v with C, then B, >> then A. > > It can just as well represent "acting" on v with (A @ B @ C). > > Of course, mathematically it shouldn't make a difference, but in > computer programming right-associative operators are always a special > case, and therefore an additional cognitive burden. I don't think it's a premature optimisation. It's a significant algorithmic optimisation. A @ (B @ (C @ v))) # 3 matrix-vector multiplies (((A @ B) @ C) @ v) # 2 matrix-matrix multiplies and one matrix-vector multiply If the matrices are NxN and the vector of length N then the matrix-vector multiplications can be performed with the asymptotically optimal N**2 operations but the matrix-matrix multiplications require something like N**2.5 or worse. It is possible but unusual for people to write this the other way round (in which case the optimisation would favour left-associativity). In any case many of the users of these operators will not know the difference between left and right associativity and will either use brackets or just write it out and hope that Python/numpy know what to do. Oscar From robert.kern at gmail.com Sat Mar 15 13:22:22 2014 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 15 Mar 2014 12:22:22 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315130631.6f97ec16@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> Message-ID: <lg1glg$mc4$1@ger.gmane.org> On 2014-03-15 12:06, Antoine Pitrou wrote: > On Sun, 16 Mar 2014 00:55:09 +1300 > Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> Antoine Pitrou wrote: >> >>> The possible reason given in the PEP is very weak and amounts to >>> premature optimization: >> >> I don't think it's just a matter of optimization. Often, >> matrix @ vector represents a linear operator acting on an >> element of a vector space. When you chain them, >> >> A @ B @ C @ v >> >> conceptually represents acting on v with C, then B, >> then A. > > It can just as well represent "acting" on v with (A @ B @ C). > > Of course, mathematically it shouldn't make a difference, but in > computer programming right-associative operators are always a special > case, and therefore an additional cognitive burden. I think his point was that people doing linear algebra tend to read these expressions "right-to-left" anyways because of that conceptual model. I'm not entirely sure how supportable that is in the general population, but it's certainly the way I think about these expressions. For me, left-associative causes an additional cognitive burden, but it's a minor one, either way. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From solipsis at pitrou.net Sat Mar 15 13:28:11 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 13:28:11 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> Message-ID: <20140315132811.0445e543@fsol> On Sat, 15 Mar 2014 12:20:42 +0000 Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > > I don't think it's a premature optimisation. It's a significant > algorithmic optimisation. You could also make the multiplications lazy (and enforce the optimal order when computing the final result) rather than enforce that optimization at the language parsing level. It would be more flexible, and would also avoid potentially pessimizing other use cases. Regards Antoine. From t_glaessle at gmx.de Sat Mar 15 13:33:02 2014 From: t_glaessle at gmx.de (=?ISO-8859-1?Q?Thomas_Gl=E4=DFle?=) Date: Sat, 15 Mar 2014 13:33:02 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> Message-ID: <5324487E.2060505@gmx.de> > I don't think it's a premature optimisation. It's a significant > algorithmic optimisation. Just in idea, any library could optimize itself by delaying the evaluation of the expression until the result is used or a vector is right multiplied. So the optimization wouldn't be a real issue here. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1031 bytes Desc: OpenPGP digital signature URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/6f339e14/attachment.sig> From oscar.j.benjamin at gmail.com Sat Mar 15 13:36:06 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 15 Mar 2014 12:36:06 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315132811.0445e543@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> Message-ID: <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com> On 15 March 2014 12:28, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sat, 15 Mar 2014 12:20:42 +0000 > Oscar Benjamin > <oscar.j.benjamin at gmail.com> wrote: >> >> I don't think it's a premature optimisation. It's a significant >> algorithmic optimisation. > > You could also make the multiplications lazy (and enforce the optimal > order when computing the final result) rather than enforce that > optimization at the language parsing level. It would be more flexible, > and would also avoid potentially pessimizing other use cases. That's true. I believe both blaze and numpypy intend to introduce optimisations in this style. Oscar From oscar.j.benjamin at gmail.com Sat Mar 15 13:42:51 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 15 Mar 2014 12:42:51 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com> Message-ID: <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com> On 15 March 2014 12:36, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote: > On 15 March 2014 12:28, Antoine Pitrou <solipsis at pitrou.net> wrote: >> On Sat, 15 Mar 2014 12:20:42 +0000 >> Oscar Benjamin >> <oscar.j.benjamin at gmail.com> wrote: >>> >>> I don't think it's a premature optimisation. It's a significant >>> algorithmic optimisation. >> >> You could also make the multiplications lazy (and enforce the optimal >> order when computing the final result) rather than enforce that >> optimization at the language parsing level. It would be more flexible, >> and would also avoid potentially pessimizing other use cases. > > That's true. I believe both blaze and numpypy intend to introduce > optimisations in this style. Just to add to that: I personally would almost always use brackets rather than rely on left- or right- associativity for something like this. A similar way that it can come up is with scalar-scalar vs scalar-array multiplication e.g.: 2 * pi * x / L * A # A is a big array I would rewrite that as (2 * pi * x / L) * A rather than rely on precedence/associativity. Oscar From steve at pearwood.info Sat Mar 15 14:47:37 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Mar 2014 00:47:37 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com> References: <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com> <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com> Message-ID: <20140315134736.GC16526@ando> On Sat, Mar 15, 2014 at 12:42:51PM +0000, Oscar Benjamin wrote: > Just to add to that: I personally would almost always use brackets > rather than rely on left- or right- associativity for something like > this. > A similar way that it can come up is with scalar-scalar vs > scalar-array multiplication e.g.: > > 2 * pi * x / L * A # A is a big array > > I would rewrite that as > > (2 * pi * x / L) * A > > rather than rely on precedence/associativity. It seems to me that you actually are relying on precedence/ associativity, otherwise you would have written it in fully-bracketed form like this: (((2 * pi) * x) / L) * A It's your choice to include redundant brackets in an expression, but I try hard to avoid the extra visual noise. -- Steven From steve at pearwood.info Sat Mar 15 15:04:51 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Mar 2014 01:04:51 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315132811.0445e543@fsol> References: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> Message-ID: <20140315140451.GD16526@ando> On Sat, Mar 15, 2014 at 01:28:11PM +0100, Antoine Pitrou wrote: > On Sat, 15 Mar 2014 12:20:42 +0000 > Oscar Benjamin > <oscar.j.benjamin at gmail.com> wrote: > > > > I don't think it's a premature optimisation. It's a significant > > algorithmic optimisation. > > You could also make the multiplications lazy (and enforce the optimal > order when computing the final result) rather than enforce that > optimization at the language parsing level. It would be more flexible, > and would also avoid potentially pessimizing other use cases. That sounds to me that you are suggesting that every single implementation of a matrix type that supports matrix multiplication needs to spend the considerable extra effort to make the multiplication lazy, in order to "potentially" avoid hurting hypothetical use-cases which don't exist now and may never exist. That's as clear a case of YAGNI as I've seen for a long time. We have one solid use-case for a matrix multiplication operator, and that's matrix multiplication. If matrix multiplication is most usefully treated as right-associative, then we ought to make it right- associative, and not burden the matrix multiplication operator with restrictions for the sake of hypothetical non-matrix-mult uses. In the same way that there is one good use-case for the numeric exponentiation operator ** , namely numeric exponentiation, and consequently it is right-associative because that's how the operator is most usefully treated. -- Steven From oscar.j.benjamin at gmail.com Sat Mar 15 15:25:10 2014 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Sat, 15 Mar 2014 14:25:10 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315134736.GC16526@ando> References: <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> <CAHVvXxSc15Fns9d2aZ1Ur6qXFXwkipg9U88ck4Af1pPwYdghmw@mail.gmail.com> <CAHVvXxR6jteUW0_7HZFMEymnqRuvuoBbxaSu0eSxGyPUn-m7qg@mail.gmail.com> <20140315134736.GC16526@ando> Message-ID: <CAHVvXxTR4-dcUTc7PpWdTB7z3xeHTm7O1P32CxaZv_ym2A0qVQ@mail.gmail.com> On 15 March 2014 13:47, Steven D'Aprano <steve at pearwood.info> wrote: > On Sat, Mar 15, 2014 at 12:42:51PM +0000, Oscar Benjamin wrote: > >> Just to add to that: I personally would almost always use brackets >> rather than rely on left- or right- associativity for something like >> this. > >> A similar way that it can come up is with scalar-scalar vs >> scalar-array multiplication e.g.: >> >> 2 * pi * x / L * A # A is a big array >> >> I would rewrite that as >> >> (2 * pi * x / L) * A >> >> rather than rely on precedence/associativity. > > It seems to me that you actually are relying on precedence/ > associativity, otherwise you would have written it in fully-bracketed > form like this: > > (((2 * pi) * x) / L) * A The point is that I don't care about the order of evaluation for the scalar part. Wherever you put the brackets works for me: (2 * pi) * (x / L) ((2 * pi) * x) / L (2 * (pi * x)) / L 2 * ((pi * x) / L) 2 * (pi * (x / L)) The reason I care about it with A is because A is a big array. Every operation on A involves an expensive pass over all the elements of the array as well as a large temporary allocation. > It's your choice to include redundant brackets in an expression, but I > try hard to avoid the extra visual noise. I don't see those brackets as noise. To me it clearly shows that I'm only doing one pass over the array which is useful information. Oscar From solipsis at pitrou.net Sat Mar 15 15:31:54 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 15:31:54 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <CAHVvXxTC+ysu2=K8MhAn-9Bb7-Wpyj564dOSxvNbR6zphBNA+w@mail.gmail.com> <20140315132811.0445e543@fsol> <20140315140451.GD16526@ando> Message-ID: <20140315153154.140b44ee@fsol> On Sun, 16 Mar 2014 01:04:51 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > > That sounds to me that you are suggesting that every single > implementation of a matrix type that supports matrix multiplication No, only implementations that are actually concerned with that particular optimization. For an analogy, bytes objects don't implement cheap slicing: if you want that, you need to use a different type (memoryview). Regards Antoine. From steve at pearwood.info Sat Mar 15 16:11:43 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Mar 2014 02:11:43 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315122733.35ed7909@fsol> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> Message-ID: <20140315151143.GE16526@ando> On Sat, Mar 15, 2014 at 12:27:33PM +0100, Antoine Pitrou wrote: > The real question is why @ would be right-associative. That's a good question, but I don't think that should be up to us to decide. Guido clearly stated that if it is more useful to define @ as right-associative, we shouldn't let the fact that most operators are left-associative get in the way of doing the right thing here. The only people who are in a position to decide that are the users of matrix multiplication. [...] > The possible reason given in the PEP is very weak and amounts to > premature optimization: I don't think it's premature optimization. Sometimes we do know ahead of time that a calculation done one way will be faster than doing it another way: you don't have to "try it and see" to realise that repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for using str.join(). [Aside: if you do try it, the string-concat reference-counting optimization of CPython may fool you into thinking that concatenation is generally fast. It isn't.] Likewise there is nothing premature about the fact that "Matrix-vector multiplication is much cheaper than matrix-matrix multiplication". The only question is whether it is more common to write: Matrix @ Matrix @ Column_Vector or Row_Vector @ Matrix @ Matrix I'll leave it to those who do matrix maths to decide which they use more often, but personally I've never come across the second case except in schoolbook exercises. > """It's been suggested that @ should be right-associative, on the > grounds that for expressions like Mat @ Mat @ vec, the two different > evaluation orders produce the same result, but the right-associative > order Mat @ (Mat @ vec) will be faster and use less memory than the > left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication > is much cheaper than matrix-matrix multiplication).""" > > If that's the only reason, then I'd like @ to be left-associative. I'm not sure that premature pessimation is much of an improvement over premature optimization. *wink* -- Steven From solipsis at pitrou.net Sat Mar 15 16:22:33 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 16:22:33 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> Message-ID: <20140315162233.4d01afd5@fsol> On Sun, 16 Mar 2014 02:11:43 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > > The possible reason given in the PEP is very weak and amounts to > > premature optimization: > > I don't think it's premature optimization. Sometimes we do know ahead of > time that a calculation done one way will be faster than doing it > another way: you don't have to "try it and see" to realise that > repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for > using str.join(). It's premature optimization because the PEP is proposing to enforce it at the language level. We didn't change *the language* so that "str +=" allows for O(N) repeated concatenation; instead we tell people that "".join() should be used for repeated concatenation. Why would our course of action be different for matrix multiplication? Regards Antoine. From ncoghlan at gmail.com Sat Mar 15 16:48:44 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2014 01:48:44 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315162233.4d01afd5@fsol> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol> Message-ID: <CADiSq7eVnViqjLyD6pyJLNFpRKit6aL2axOzD+rOeiTgWTTtTw@mail.gmail.com> On 16 March 2014 01:22, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sun, 16 Mar 2014 02:11:43 +1100 > Steven D'Aprano <steve at pearwood.info> wrote: >> > The possible reason given in the PEP is very weak and amounts to >> > premature optimization: >> >> I don't think it's premature optimization. Sometimes we do know ahead of >> time that a calculation done one way will be faster than doing it >> another way: you don't have to "try it and see" to realise that >> repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for >> using str.join(). > > It's premature optimization because the PEP is proposing to enforce it > at the language level. We didn't change *the language* so that "str +=" > allows for O(N) repeated concatenation; instead we tell people that > "".join() should be used for repeated concatenation. Why would our > course of action be different for matrix multiplication? That's why Guido's request was for the numeric community to go back and try to come up with a better rationale one way or the other for that question - he wanted to make it clear to them that right associativity *was* potentially acceptable, so they should seriously consider the question and make the case for right associativity if they decided they really wanted it, rather than assume that we would reject the idea out of hand. Left associativity still remains the default, and this isn't the right list to argue about the choice (as most of the relevant people aren't here). If the numeric computing community decide they *do* want right associativity and add that to the PEP, *then* Guido will need to make the call as to whether or not he considers their rationale for requesting it convincing (and we will have the opportunity to chime in with our own perspectives to help him make up his mind). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Sat Mar 15 17:26:49 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Mar 2014 03:26:49 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315162233.4d01afd5@fsol> References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol> Message-ID: <20140315162649.GF16526@ando> On Sat, Mar 15, 2014 at 04:22:33PM +0100, Antoine Pitrou wrote: > On Sun, 16 Mar 2014 02:11:43 +1100 > Steven D'Aprano <steve at pearwood.info> wrote: > > > The possible reason given in the PEP is very weak and amounts to > > > premature optimization: > > > > I don't think it's premature optimization. Sometimes we do know ahead of > > time that a calculation done one way will be faster than doing it > > another way: you don't have to "try it and see" to realise that > > repeatedly adding strings in a for-loop can be O(N**2) versus O(N) for > > using str.join(). > > It's premature optimization because the PEP is proposing to enforce it > at the language level. The PEP leaves the question of left- versus right-associativity open. It has to be decided one way or the other. What evidence would you want to see before saying "It's not premature optimization, it's a justified optimization"? > We didn't change *the language* so that "str +=" > allows for O(N) repeated concatenation; instead we tell people that > "".join() should be used for repeated concatenation. Why would our > course of action be different for matrix multiplication? Exactly the same argument applies to left-associativity. Why should the language enforce optimizing the first case over the second? row_vector @ matrix @ matrix vs. matrix @ matrix @ column_vector Since infix operators have to have an associativity, the language implicitly has to optimize one case or the other. We can't sit on the fence and refuse to favour one over the other. If the matrix-using community says that both cases are equally common, (or uncommon, perhaps) and they are indifferent between left- and right-associativity, then it makes sense to stick to what the majority of other operators do. I agree with that. But if the matrix-using community comes down in favour of right- associativity, because Practicality Beats Purity and the second case is sufficiently more common than the first case, then I think it is a *broken design* to force left-associativity on them just to satisfy the expectations of people who aren't going to use the @ operator. I hope I've made my position clear here: Guido has given his blessing to the possibility of making @ right-associative, and I think that the only people who should care what the associativity of @ ends up being is the community of matrix-multiplication users. They should make that decision, based on what it useful to them. I don't care what colour this bikeshed ends up having, I just want to see that it is the users of that shed that make the decision. -- Steven From cjwelborn at live.com Sat Mar 15 18:13:16 2014 From: cjwelborn at live.com (Christopher Welborn) Date: Sat, 15 Mar 2014 12:13:16 -0500 Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com> References: <lg0ck8$nnh$1@ger.gmane.org> <64794D68-6878-4C6C-9D0B-8A3A626A1ABE@yahoo.com> Message-ID: <lg21mu$7fj$1@ger.gmane.org> On 03/14/2014 09:43 PM, Andrew Barnert wrote: > [x for xs in xss for x in xs] > [x for x in xs if x if x-1] > [x for xs in xss if xs for x in xs if x] > > Should you be able to write all of those as one-liner statements? Fair enough. No I wouldn't like any of those, but thanks for making things clearer for me. -- \?\ /?/\ \ \/??\/ / / Christopher Welborn (cj) \__/\__/ / cjwelborn at live?com \__/\__/ http://welbornprod.com From asmeurer at gmail.com Sat Mar 15 18:14:58 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Sat, 15 Mar 2014 12:14:58 -0500 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315122733.35ed7909@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> Message-ID: <CAKgW=6KNoVjJfT2SNyBLCZ0YxFmduwQV-tmjxae6duda9PPmWg@mail.gmail.com> On Sat, Mar 15, 2014 at 6:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sat, 15 Mar 2014 18:35:09 +1100 > Steven D'Aprano <steve at pearwood.info> wrote: >> On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote: >> > I think making @ right associative would make this less suitable for other >> > uses. Example >> > >> > someclassobject @ somevalue @ somevalue >> > >> > won't work with right-associativity. >> >> Why not? It works with the other right-associative operator: >> >> x**y**z >> >> "Works" depends on what you expect it to do. > > The real question is why @ would be right-associative. "**" is very > rarely used in a chained manner as the above, so its associativity > isn't really important (I don't think I have ever written "x**y**z"). > @ will be used routinely in a chained manner, so the question is more > important here. I write that all the time when working with SymPy. And the associativity of ** is *very* important. If ** were left associative it would not be useful, because, at least for positive x, y, and z, (x**y)**z is the same as x**(y*z). If you really want the stacked power it has to be right associative. Aaron Meurer > > The possible reason given in the PEP is very weak and amounts to > premature optimization: > > """It's been suggested that @ should be right-associative, on the > grounds that for expressions like Mat @ Mat @ vec, the two different > evaluation orders produce the same result, but the right-associative > order Mat @ (Mat @ vec) will be faster and use less memory than the > left-associative order (Mat @ Mat) @ vec. (Matrix-vector multiplication > is much cheaper than matrix-matrix multiplication).""" > > If that's the only reason, then I'd like @ to be left-associative. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > -- > > --- > You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group. > To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/aHVlL6BADLY/unsubscribe. > To unsubscribe from this group and all its topics, send an email to python-ideas+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. From solipsis at pitrou.net Sat Mar 15 18:15:21 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Mar 2014 18:15:21 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <20140315162233.4d01afd5@fsol> <20140315162649.GF16526@ando> Message-ID: <20140315181521.33cf22ca@fsol> On Sun, 16 Mar 2014 03:26:49 +1100 Steven D'Aprano <steve at pearwood.info> wrote: > Exactly the same argument applies to left-associativity. Why should the > language enforce optimizing the first case over the second? It has nothing to do with optimizing, it's just that left-associativity is the common expectation wrt. operators. Left-associativity doesn't need any particular justification. Regards Antoine. From cjwelborn at live.com Sat Mar 15 18:22:37 2014 From: cjwelborn at live.com (Christopher Welborn) Date: Sat, 15 Mar 2014 12:22:37 -0500 Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <20140315023747.GA16526@ando> References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando> Message-ID: <lg228f$d92$1@ger.gmane.org> On 03/14/2014 09:37 PM, Steven D'Aprano wrote: > This suggestion has been made, oh, a million times -- not that I would > ever exaggerate... *wink* -- and has been rejected each time. Oops, I wouldn't have mentioned it if I had known. > or other combinations like: > > while x if cond: > if cond while x: > if cond try: > for x try: > for x if y try: > Yeh I could see things getting out of hand if all of those were added. You both have made good points. Thank you. -- \?\ /?/\ \ \/??\/ / / Christopher Welborn (cj) \__/\__/ / cjwelborn at live?com \__/\__/ http://welbornprod.com From alexander.belopolsky at gmail.com Sat Mar 15 21:50:28 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 15 Mar 2014 16:50:28 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315151143.GE16526@ando> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> Message-ID: <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> On Sat, Mar 15, 2014 at 11:11 AM, Steven D'Aprano <steve at pearwood.info>wrote: > The only question is whether it is more common to write: > > Matrix @ Matrix @ Column_Vector > > or > > Row_Vector @ Matrix @ Matrix > > > I'll leave it to those who do matrix maths to decide which they use more > often, but personally I've never come across the second case except in > schoolbook exercises. > Abstractly, 1-dimensional arrays are neither columns nor rows, but Python's horizontal notation makes them more row-like than column-like. In 2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a column-vector. Which one is more "natural"? When you have a matrix A = [[1, 2], [3, 4]] A[1] is [3, 4], which is a row. To get a column, [2, 4], one has to write A[:,1] in numpy. When it comes to matrix - vector multiplication, [1, 2] @ [[1, 2], [3, 4]] -> [7, 10] has a text-book appearance, while [[1, 2], [3, 4]] @ [1, 2] -> [5, 11] has to be mentally cast into ([[1, 2], [3, 4]] @ [[1], [2]])[0] -> [5, 11] While it is more common in math literature to see Mat @ vec than vec @ Mat, I don't think anyone who has completed an introductory linear algebra course would have trouble understanding what [1, 2, 3] @ Mat means. On the other hand, novice programmers may find it puzzling why Mat @ [Mat1, Mat2] is the same as [Mat @ Mat1, Mat @ Mat2], but [Mat @ [vec1, vec2]] is not [Mat @ vec1, Mat @ vec2]. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/67daccb1/attachment.html> From greg.ewing at canterbury.ac.nz Sat Mar 15 23:31:49 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Mar 2014 11:31:49 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315130631.6f97ec16@fsol> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> Message-ID: <5324D4D5.5090106@canterbury.ac.nz> Antoine Pitrou wrote: > It can just as well represent "acting" on v with (A @ B @ C). Yes, but if you ask what A @ B @ C means as an operator, it means "the same thing as acting with C, then B, then A." The fact that evaluating it right-to-left also happens to be more efficient in some cases means that there are both conceptual *and* practical reasons for making @ right associative. I was just trying to point out that efficiency is not the *only* reason to consider this. There's a counter-argument as well -- it's only more efficient if the rightmost operand is just a single vector or a small enough array of vectors. Otherwise, it's more efficient to calculate a combined operator and then apply that. So the question would be whether the small case is common enough in practice to be worth having an unusual associativity. A data point from another language, for what it's worth: In APL, *all* operators are right-associative. I'm not sure what the reason for that choice was, but it may result from a similar line of reasoning around how mathematicians think about things. -- Greg From alexander.belopolsky at gmail.com Sat Mar 15 23:44:50 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 15 Mar 2014 18:44:50 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <5324D4D5.5090106@canterbury.ac.nz> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <5324D4D5.5090106@canterbury.ac.nz> Message-ID: <CAP7h-xZXh2f3PAeeOVNy3fJuug86QjRPdz2kuUC0iTguk_KkLA@mail.gmail.com> On Sat, Mar 15, 2014 at 6:31 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote: > In APL, *all* operators are right-associative. I'm not > sure what the reason for that choice was, but it may > result from a similar line of reasoning around how > mathematicians think about things. > In APL, there is no operator precedence, so right associativity is the only way to make expressions that mix unary and binary operation look natural. For example, a + -b. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/432d96d7/attachment.html> From greg.ewing at canterbury.ac.nz Sat Mar 15 23:56:55 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Mar 2014 11:56:55 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lg1glg$mc4$1@ger.gmane.org> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <53243F9D.1060802@canterbury.ac.nz> <20140315130631.6f97ec16@fsol> <lg1glg$mc4$1@ger.gmane.org> Message-ID: <5324DAB7.8010709@canterbury.ac.nz> Robert Kern wrote: > For me, left-associative causes an additional cognitive burden, but it's > a minor one, either way. Wild idea -- provide a way to spell both left and right associative versions. It's disappointing that there isn't a character that's the mirror image of an @. Failing that: A @ B left associative A @: B right associative (Ducks hail of tomatoes from the no-colons-in-expressions crowd...) -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 15 23:56:58 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Mar 2014 11:56:58 +1300 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315151143.GE16526@ando> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> Message-ID: <5324DABA.9020808@canterbury.ac.nz> Steven D'Aprano wrote: > or > > Row_Vector @ Matrix @ Matrix > > personally I've never come across the second case except in > schoolbook exercises. It turns up sometimes in quantum mechanics, at least when doing the algebra. I don't know whether people who do QM calculations numerically ever do it that way, though. Flipping it around would be easy enough if it were more efficient. -- Greg From ncoghlan at gmail.com Sun Mar 16 03:57:06 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2014 12:57:06 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> Message-ID: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> On 16 March 2014 06:50, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > > Abstractly, 1-dimensional arrays are neither columns nor rows, but Python's > horizontal notation makes them more row-like than column-like. In > 2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a > column-vector. Which one is more "natural"? Folks, please stop trying to argue this one in the abstract. The decision will likely be made primarily based on the feedback of folks that have *actually been using and teaching* Python as a tool for matrix manipulation (etc) for more than a decade. Guido has asked them to have that discussion and summarise their conclusions in the PEP - we can lob our armchair opinions into the mix after the experts have spoken. This won't be the first time features have been added to the language core specifically for the benefit of the numeric community - the ellipsis notation and extended slicing in general were added for their benefit years ago (I started getting involved in Python core development around Python 2.3, just as the core sequence types were being updated to support extended slicing, and being able to use "s[::-1]" with lists, strings, etc was a novel concept). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From guido at python.org Sun Mar 16 03:59:17 2014 From: guido at python.org (Guido van Rossum) Date: Sat, 15 Mar 2014 19:59:17 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> Message-ID: <CAP7+vJLsGP8cAT5yN+0nQuKdk=Agsaw35bRYyfYr16MWUpZY0g@mail.gmail.com> Also complex numbers were Jim Hugunin's request for numeric work. On Saturday, March 15, 2014, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 16 March 2014 06:50, Alexander Belopolsky > <alexander.belopolsky at gmail.com <javascript:;>> wrote: > > > > Abstractly, 1-dimensional arrays are neither columns nor rows, but > Python's > > horizontal notation makes them more row-like than column-like. In > > 2-dimensional case, [[1,2]] is a row-vector and [[1],[2]] is a > > column-vector. Which one is more "natural"? > > Folks, please stop trying to argue this one in the abstract. The > decision will likely be made primarily based on the feedback of folks > that have *actually been using and teaching* Python as a tool for > matrix manipulation (etc) for more than a decade. > > Guido has asked them to have that discussion and summarise their > conclusions in the PEP - we can lob our armchair opinions into the mix > after the experts have spoken. > > This won't be the first time features have been added to the language > core specifically for the benefit of the numeric community - the > ellipsis notation and extended slicing in general were added for their > benefit years ago (I started getting involved in Python core > development around Python 2.3, just as the core sequence types were > being updated to support extended slicing, and being able to use > "s[::-1]" with lists, strings, etc was a novel concept). > > Regards, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com <javascript:;> | Brisbane, > Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org <javascript:;> > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (on iPad) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/a34412ba/attachment.html> From alexander.belopolsky at gmail.com Sun Mar 16 04:12:52 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 15 Mar 2014 23:12:52 -0400 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> Message-ID: <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com> On Sat, Mar 15, 2014 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > Folks, please stop trying to argue this one in the abstract. The > decision will likely be made primarily based on the feedback of folks > that have *actually been using and teaching* Python as a tool for > matrix manipulation (etc) for more than a decade. > I would think I qualify having been using Numeric/NumPy since 2003. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140315/5bd40381/attachment.html> From ncoghlan at gmail.com Sun Mar 16 04:32:15 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2014 13:32:15 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> <CAP7h-xYYvL9V_zjvimAHzjrHyx3XE3uc+DGpcx7vgxhABX6yKw@mail.gmail.com> Message-ID: <CADiSq7feYKs0zK8_xr=B-3eTwe-0zbyBabuSB2pcFmBk+RHnsA@mail.gmail.com> On 16 March 2014 13:12, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote: > > On Sat, Mar 15, 2014 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: >> >> Folks, please stop trying to argue this one in the abstract. The >> decision will likely be made primarily based on the feedback of folks >> that have *actually been using and teaching* Python as a tool for >> matrix manipulation (etc) for more than a decade. > > > I would think I qualify having been using Numeric/NumPy since 2003. OK, thanks for the additional context - I didn't know that. However, I still suggest finding the relevant discussion on the NumPy lists and contributing there would be a better way to go, rather than getting Nathaniel to track the discussion in two places at once. We'll get another go around here and on python-dev after the PEP has been updated with answers to Guido's questions :) Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From solipsis at pitrou.net Sun Mar 16 13:26:34 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 16 Mar 2014 13:26:34 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> Message-ID: <20140316132634.13d6d589@fsol> On Sun, 16 Mar 2014 12:57:06 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote: > > Guido has asked them to have that discussion and summarise their > conclusions in the PEP - we can lob our armchair opinions into the mix > after the experts have spoken. Debunking armchair opinions is one of the points of the PEP process :-) Regards Antoine. From ncoghlan at gmail.com Sun Mar 16 14:05:15 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2014 23:05:15 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140316132634.13d6d589@fsol> References: <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <20140315122733.35ed7909@fsol> <20140315151143.GE16526@ando> <CAP7h-xa2n+b1XqOVEKKtKnCuZpUZVf10F8Q7N3mh6mCaBwBqAA@mail.gmail.com> <CADiSq7f6FNY==DUHLT=OWXrrSq0_gbttOJvOJztkq_4kH_L0EQ@mail.gmail.com> <20140316132634.13d6d589@fsol> Message-ID: <CADiSq7cEPUjj2FmpcjretgzbNnWLFWCjYJqJ44Xxc1A5u4QbgQ@mail.gmail.com> On 16 March 2014 22:26, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Sun, 16 Mar 2014 12:57:06 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: >> >> Guido has asked them to have that discussion and summarise their >> conclusions in the PEP - we can lob our armchair opinions into the mix >> after the experts have spoken. > > Debunking armchair opinions is one of the points of the PEP process :-) Aye, but at the moment it makes sense to wait and see if there is even an argument to be had - Nathaniel may decide that even after reviewing the question seriously, he doesn't want to propose a right associative operator :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From bruce at leapyear.org Mon Mar 17 04:46:08 2014 From: bruce at leapyear.org (Bruce Leban) Date: Sun, 16 Mar 2014 20:46:08 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315073508.GB16526@ando> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> Message-ID: <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> On Sat, Mar 15, 2014 at 12:35 AM, Steven D'Aprano <steve at pearwood.info>wrote: > On Fri, Mar 14, 2014 at 09:09:02PM -0700, Bruce Leban wrote: > > I think making @ right associative would make this less suitable for > other > > uses. Example > > > > someclassobject @ somevalue @ somevalue > > > > won't work with right-associativity. > > Why not? It works with the other right-associative operator: > > x**y**z > It won't work because right-associativity means that the two values on the right are combined together first and they won't be combined properly. given that the type of someclassobject is not consulted. > > "Works" depends on what you expect it to do. Unless you tell us what > these "other uses" are, how can we know that right-associativity won't > work? > Here's a simple example to illustrate the concept. Suppose I have a class that is roughly equivalent to a dict mapping values to lists of values, e.g., a = { 1: [10, 11, 12] } b = { 1: [11, 13] } I might have a + b = { 1: [10, 11, 12, 11, 13] # appends } a @ b = { 1: [10, 11, 12, 13] # appends non-duplicate values } but I also want to be able to merge in standard dicts. If @ is right-associative, a @ x @ y where x and y are dicts, will try to compute x @ y first and built-in dict doesn't supporr the @ operator. Even if it did, it's not going to do what I'd want here. I also want to give a larger example to illustrate the kind of thing I think this operator would be useful for. SQLAlchemy has query objects where you can write something like some_query.filter(...).order_by(...).join(...) I image that this could use the @ operator: some_query @ filter(...) @ order_by(...) @ join(...) The benefit here is that I have filter objects, order_by objects, etc. that can be passed around rather than only having API calls. In the current API, it's clumsy to support that as I either need both filter objects and a query.filter method or I need a clumsy query.apply method. Having an @operator I think works really well. Left vs. right associativity is useful if I wanted to support other kinds of objects on the right hand side of the @, e.g., if: some_query @ { column1: value1, column2: value2, ... } as another spelling of some_query @ filter(column1=value1, column2=value2, ... } --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140316/7fb4c9a5/attachment.html> From ncoghlan at gmail.com Mon Mar 17 06:03:19 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 17 Mar 2014 15:03:19 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> Message-ID: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> On 17 March 2014 13:46, Bruce Leban <bruce at leapyear.org> wrote: > I also want to give a larger example to illustrate the kind of thing I think > this operator would be useful for. SQLAlchemy has query objects where you > can write something like > > some_query.filter(...).order_by(...).join(...) > > > I image that this could use the @ operator: > > some_query @ filter(...) @ order_by(...) @ join(...) There are plenty of existing left-associative operators that could be used for that if Mike Bayer wanted (or could be convinced) to do so, so I don't see how that observation is relevant to the question of whether or not this particular proposal should be for a right-associative operator. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From stephen at xemacs.org Mon Mar 17 06:50:32 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 17 Mar 2014 14:50:32 +0900 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lfv7ug$tuv$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314112426.7854e9f6@fsol> <20140314123239.GP12595@ando> <lfv7ug$tuv$1@ger.gmane.org> Message-ID: <874n2xl6fr.fsf@uwakimon.sk.tsukuba.ac.jp> Ron Adam writes: > The best I can come up with is to use '^' instead of dot. This symbol is used to denote the "wedge product" of vectors, so it's pretty clearly inappropriate. I forget the exact definition, but it's related to the area (volume) of the parallelpiped spanned by the vectors. From stephen at xemacs.org Mon Mar 17 06:59:25 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 17 Mar 2014 14:59:25 +0900 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140314180811.35d7cdf6@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <lfvboj$eab$1@ger.gmane.org> <20140314180811.35d7cdf6@fsol> Message-ID: <8738ihl60y.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > I am personally not fond of @, but I find it ok in that it is > distinctive enough without being terribly ugly. "Not terribly ugly" is the best we're going to do, since Guido has already ruled out non-ASCII characters. Well, without another PEP, and I don't think we should delay PEP 465 for a "Unicode operators" PEP. From sturla.molden at gmail.com Mon Mar 17 09:54:34 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Mon, 17 Mar 2014 08:54:34 +0000 (UTC) Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> Message-ID: <585287815416738714.901387sturla.molden-gmail.com@news.gmane.org> "M.-A. Lemburg" <mal at egenix.com> wrote: >> Indeed. Numpy already uses a .T property for this. > > Ah, good trick :-) It is a better trick than you would expect. Matlab and Fortran matrix transpose has complexity O(M*N) for a rank M x N matrix. NumPy's .T attribute has complexity O(1) With Numpy, we can put .T into matrix expressions with impunity. Now THAT is a good trick :-) I am not aware of any other system where memory access is so flexible that matrix transposition can be done in O(1) time. Regards, Sturla From robert.kern at gmail.com Mon Mar 17 11:35:37 2014 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 17 Mar 2014 10:35:37 +0000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> Message-ID: <lg6j55$7g1$1@ger.gmane.org> On 2014-03-17 05:03, Nick Coghlan wrote: > On 17 March 2014 13:46, Bruce Leban <bruce at leapyear.org> wrote: >> I also want to give a larger example to illustrate the kind of thing I think >> this operator would be useful for. SQLAlchemy has query objects where you >> can write something like >> >> some_query.filter(...).order_by(...).join(...) >> >> >> I image that this could use the @ operator: >> >> some_query @ filter(...) @ order_by(...) @ join(...) > > There are plenty of existing left-associative operators that could be > used for that if Mike Bayer wanted (or could be convinced) to do so, > so I don't see how that observation is relevant to the question of > whether or not this particular proposal should be for a > right-associative operator. Although this does suggest that having a right-associative operator might provide some useful variety in the operator-overloading-as-DSL ecosystem. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From anthony at xtfx.me Mon Mar 17 16:58:20 2014 From: anthony at xtfx.me (C Anthony Risinger) Date: Mon, 17 Mar 2014 10:58:20 -0500 Subject: [Python-ideas] Function-like Modules (better control/introspection) In-Reply-To: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> Message-ID: <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> ok, let me try this again, sans distractions: could we consider adding __code__ and __call__ to module objects? __code__ would contain the code executed in the module __dict__, and __call__ would reload the module. thoughts? -- C Anthony On Fri, Mar 14, 2014 at 4:03 PM, C Anthony Risinger <anthony at xtfx.me> wrote: > hello, > > could we make modules work more like this... > > https://gist.github.com/xtfxme/9556806 > > tl;dr: __code__ object available (potentially a __self__ object), reload > by calling, makes working with modules more natural/safe! > > ENCLOSED: modules-next.py > > -- > > C Anthony > > #----------------------------------------------------------------------------------------( > modules-next.py ) > # encoding: utf-8 > # > # modules-next > # > # C Anthony Risinger > > > import sys > > > m_name = 'modules_next' > m_source = r"""\ > if __name__ == '{0}': > from . import __call__ as reexec > from . import __code__ as code > from . import __dict__ as ns > print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) > __name__ = 'modules_next_reloaded' > reexec() > print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) > else: > print('!!! __name__ == {{0}}\n'.format(__name__)) > > def throw(): > raise Exception(__name__) > """.format(m_name) > m_code = compile(m_source, m_name, 'exec') > m_dict = { > '__file__': __file__ + '.next', > '__name__': m_name, > '__path__': list(), > '__doc__': None, > } > #...bind namespace/functions > module = eval('lambda: None', m_dict) > #...automatically expose as attributes > module.__dict__ = module.__globals__ > #...must be set twice, __code__ sees __dict__, but we see descriptor > module.__name__ = m_name > #...redefine the function body! > module.__code__ = m_code > #...yay! we haz legitimate module! > sys.modules[m_name] = module > > > print('--* importing...') > import modules_next > print('>>> {0}: {1}\n{2}\n'.format( > __name__, > modules_next.__code__, > modules_next.__dict__.keys(), > )) > > print('--* importing (for real this time)...') > #...import/reload by calling our func-y-module > module() > print('>>> {0}: {1}\n{2}\n'.format( > __name__, > modules_next.__code__, > modules_next.__dict__.keys(), > )) > > > print('--* throwing...') > modules_next.throw() > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/0550a789/attachment.html> From lemiant at hotmail.com Mon Mar 17 17:07:05 2014 From: lemiant at hotmail.com (Alex Rodrigues) Date: Mon, 17 Mar 2014 10:07:05 -0600 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight Message-ID: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> I'm +1 for this idea. It's not about whether the style is good (which its not), but its fundamentally about the fact that it will fix bugs. From ncoghlan at gmail.com Mon Mar 17 17:36:31 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2014 02:36:31 +1000 Subject: [Python-ideas] Function-like Modules (better control/introspection) In-Reply-To: <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> Message-ID: <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote: > > ok, let me try this again, sans distractions: > > could we consider adding __code__ and __call__ to module objects? __code__ would contain the code executed in the module __dict__, and __call__ would reload the module. > > thoughts? The question to ask is: what would such a change make possible that importlib doesn't already handle? This specific proposal wouldn't work right for extension modules, so people will still need to go through the importlib APIs anyway. If you haven't seen it yet, you may also want to give PEP 451 a read - we already overhauled several aspects of the import system for 3.4. Cheers, Nick. > > -- > > C Anthony > > > On Fri, Mar 14, 2014 at 4:03 PM, C Anthony Risinger <anthony at xtfx.me> wrote: >> >> hello, >> >> could we make modules work more like this... >> >> https://gist.github.com/xtfxme/9556806 >> >> tl;dr: __code__ object available (potentially a __self__ object), reload by calling, makes working with modules more natural/safe! >> >> ENCLOSED: modules-next.py >> >> -- >> >> C Anthony >> >> #----------------------------------------------------------------------------------------( modules-next.py ) >> # encoding: utf-8 >> # >> # modules-next >> # >> # C Anthony Risinger >> >> >> import sys >> >> >> m_name = 'modules_next' >> m_source = r"""\ >> if __name__ == '{0}': >> from . import __call__ as reexec >> from . import __code__ as code >> from . import __dict__ as ns >> print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) >> __name__ = 'modules_next_reloaded' >> reexec() >> print('>>> {{0}}: {{1}}\n{{2}}\n'.format(__name__, code, ns.keys())) >> else: >> print('!!! __name__ == {{0}}\n'.format(__name__)) >> >> def throw(): >> raise Exception(__name__) >> """.format(m_name) >> m_code = compile(m_source, m_name, 'exec') >> m_dict = { >> '__file__': __file__ + '.next', >> '__name__': m_name, >> '__path__': list(), >> '__doc__': None, >> } >> #...bind namespace/functions >> module = eval('lambda: None', m_dict) >> #...automatically expose as attributes >> module.__dict__ = module.__globals__ >> #...must be set twice, __code__ sees __dict__, but we see descriptor >> module.__name__ = m_name >> #...redefine the function body! >> module.__code__ = m_code >> #...yay! we haz legitimate module! >> sys.modules[m_name] = module >> >> >> print('--* importing...') >> import modules_next >> print('>>> {0}: {1}\n{2}\n'.format( >> __name__, >> modules_next.__code__, >> modules_next.__dict__.keys(), >> )) >> >> print('--* importing (for real this time)...') >> #...import/reload by calling our func-y-module >> module() >> print('>>> {0}: {1}\n{2}\n'.format( >> __name__, >> modules_next.__code__, >> modules_next.__dict__.keys(), >> )) >> >> >> print('--* throwing...') >> modules_next.throw() > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/258b5334/attachment.html> From solipsis at pitrou.net Mon Mar 17 18:11:55 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 17 Mar 2014 18:11:55 +0100 Subject: [Python-ideas] Function-like Modules (better control/introspection) References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> Message-ID: <20140317181155.5ea09201@fsol> On Tue, 18 Mar 2014 02:36:31 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote: > On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote: > > > > ok, let me try this again, sans distractions: > > > > could we consider adding __code__ and __call__ to module objects? > __code__ would contain the code executed in the module __dict__, and > __call__ would reload the module. > > > > thoughts? > > The question to ask is: what would such a change make possible that > importlib doesn't already handle? I suppose module.__code__ would be a lot less wasteful than module.__loader__.get_code(module.__name__) (which seems to reload the bytecode from disk every time). (while looking at this, I was surprised that dis.dis(some_module) 1) doesn't show a disassembly for the module toplevel 2) shows disassembly for objects which some_module imports from other modules) Regards Antoine. From guido at python.org Mon Mar 17 18:24:08 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2014 10:24:08 -0700 Subject: [Python-ideas] Function-like Modules (better control/introspection) In-Reply-To: <20140317181155.5ea09201@fsol> References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> <20140317181155.5ea09201@fsol> Message-ID: <CAP7+vJK4hBwDdoPgr2Q333EKrHGO=Vs7RgR7ZHgVA3s7RAr=SA@mail.gmail.com> I don't know whether it's worth the savings, but the __code__ for a module's toplevel is thrown away because it is not needed after it has run. Reading it back from disk is wasteful if you do it every time for every module, but how often is it used in a situation where speed matters? Saving __code__ might be useful so you can tell with certainty which code was run -- the file may have been modified since it was imported. On Mon, Mar 17, 2014 at 10:11 AM, Antoine Pitrou <solipsis at pitrou.net>wrote: > On Tue, 18 Mar 2014 02:36:31 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: > > On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote: > > > > > > ok, let me try this again, sans distractions: > > > > > > could we consider adding __code__ and __call__ to module objects? > > __code__ would contain the code executed in the module __dict__, and > > __call__ would reload the module. > > > > > > thoughts? > > > > The question to ask is: what would such a change make possible that > > importlib doesn't already handle? > > I suppose module.__code__ would be a lot less wasteful than > module.__loader__.get_code(module.__name__) (which seems to reload the > bytecode from disk every time). > > (while looking at this, I was surprised that dis.dis(some_module) > 1) doesn't show a disassembly for the module toplevel 2) shows > disassembly for objects which some_module imports from other modules) > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/e25b80ff/attachment.html> From ncoghlan at gmail.com Mon Mar 17 18:38:47 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2014 03:38:47 +1000 Subject: [Python-ideas] Function-like Modules (better control/introspection) In-Reply-To: <20140317181155.5ea09201@fsol> References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> <20140317181155.5ea09201@fsol> Message-ID: <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com> On 18 Mar 2014 03:16, "Antoine Pitrou" <solipsis at pitrou.net> wrote: > > On Tue, 18 Mar 2014 02:36:31 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: > > On 18 Mar 2014 01:59, "C Anthony Risinger" <anthony at xtfx.me> wrote: > > > > > > ok, let me try this again, sans distractions: > > > > > > could we consider adding __code__ and __call__ to module objects? > > __code__ would contain the code executed in the module __dict__, and > > __call__ would reload the module. > > > > > > thoughts? > > > > The question to ask is: what would such a change make possible that > > importlib doesn't already handle? > > I suppose module.__code__ would be a lot less wasteful than > module.__loader__.get_code(module.__name__) (which seems to reload the > bytecode from disk every time). Yep, it's a space/speed trade-off - keeping this info around could waste a lot of memory, and most code will never need it. It's a similar reason to why we throw source code away after compiling it, but then also route inspect.getsource() through the linecache module. Cheers, Nick. > > (while looking at this, I was surprised that dis.dis(some_module) > 1) doesn't show a disassembly for the module toplevel 2) shows > disassembly for objects which some_module imports from other modules) I don't think we have any tests to see how dis.dis behaves with module objects - it's likely just hitting the path that is designed to disassemble all methods on a class. (We throw away the bytecode after execution for classes as well) Cheers, Nick. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/18cfdc44/attachment.html> From solipsis at pitrou.net Mon Mar 17 18:41:20 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 17 Mar 2014 18:41:20 +0100 Subject: [Python-ideas] Function-like Modules (better control/introspection) References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> <20140317181155.5ea09201@fsol> <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com> Message-ID: <20140317184120.5d333d0d@fsol> On Tue, 18 Mar 2014 03:38:47 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote: > > > > (while looking at this, I was surprised that dis.dis(some_module) > > 1) doesn't show a disassembly for the module toplevel 2) shows > > disassembly for objects which some_module imports from other modules) > > I don't think we have any tests to see how dis.dis behaves with module > objects - it's likely just hitting the path that is designed to disassemble > all methods on a class. (We throw away the bytecode after execution for > classes as well) A comment in dis.dis's source code does talk about modules, so apparently someone thought about them, but not very hard :-) (edit: that someone is you - b2a9a59c2d57 ;-P) Regards Antoine. From njs at pobox.com Tue Mar 18 00:08:51 2014 From: njs at pobox.com (Nathaniel Smith) Date: Mon, 17 Mar 2014 23:08:51 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> Message-ID: <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > A few other miscellaneous comments: > > - nice work on the PEP Nathaniel! Thanks! > - as with others, "@" as the operator doesn't thrill me, but I also think it > crosses the threshold of "good enough given the constraints" > - the PEP should probably recommend adding an "operator.matmul" function, a > "PyObject_MatrixMultiply" C API and consider whether or not the new special > method should be given a C level type slot. operator.matmul and PyObject_MatrixMultiply are obvious enough, but I'm afraid I'm not too clear on the tradeoffs about adding a C level type slot, or even entirely sure what the alternative is. (I guess I just assumed that all special methods used C level type slots and there was nothing to think about.) Do you (or anyone) have any thoughts? -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From ncoghlan at gmail.com Tue Mar 18 00:32:55 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2014 09:32:55 +1000 Subject: [Python-ideas] Function-like Modules (better control/introspection) In-Reply-To: <20140317184120.5d333d0d@fsol> References: <CAGAVQTEpT2HFDconso6tDkwqu4n9dReOpOvyYfgK4miUBfOLrw@mail.gmail.com> <CAGAVQTHserJ6OQxRK81C-GJHxZ2e9XyiRScqjxt2A6TE9+vXyw@mail.gmail.com> <CADiSq7eqVXywxgTy=Nbc7pAjapgihdd--huiX3_Urt60XGJyDg@mail.gmail.com> <20140317181155.5ea09201@fsol> <CADiSq7dxiNskRAMDj=HupLghuFc5kF68+hLCwamJ8LYk4J7DOg@mail.gmail.com> <20140317184120.5d333d0d@fsol> Message-ID: <CADiSq7exvqYYdDHkYiQP-Tkoz1ibkB7w8RMmnP7FrSqACcuTMQ@mail.gmail.com> On 18 Mar 2014 03:45, "Antoine Pitrou" <solipsis at pitrou.net> wrote: > > On Tue, 18 Mar 2014 03:38:47 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: > > > > > > (while looking at this, I was surprised that dis.dis(some_module) > > > 1) doesn't show a disassembly for the module toplevel 2) shows > > > disassembly for objects which some_module imports from other modules) > > > > I don't think we have any tests to see how dis.dis behaves with module > > objects - it's likely just hitting the path that is designed to disassemble > > all methods on a class. (We throw away the bytecode after execution for > > classes as well) > > A comment in dis.dis's source code does talk about modules, so > apparently someone thought about them, but not very hard :-) > > (edit: that someone is you - b2a9a59c2d57 ;-P) Huh, clearly I put a lot of thought into that one... One day I will learn to go check the source rather than speculating from memory, but apparently yesterday is not that day :) Cheers, Nick. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/5d098d50/attachment-0001.html> From bruce at leapyear.org Tue Mar 18 05:14:03 2014 From: bruce at leapyear.org (Bruce Leban) Date: Mon, 17 Mar 2014 21:14:03 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> References: <5322D702.6040907@egenix.com> <lfuouo$v25$1@ger.gmane.org> <5322EC7D.1020908@egenix.com> <53230E87.7070503@stoneleaf.us> <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> Message-ID: <CAGu0AnuCUVV8FOcAOvS1kaYA1eU2erTZK1grQ02wDChp082VOA@mail.gmail.com> On Sun, Mar 16, 2014 at 10:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > There are plenty of existing left-associative operators that could be > used for that if Mike Bayer wanted (or could be convinced) to do so, > so I don't see how that observation is relevant to the question of > whether or not this particular proposal should be for a > right-associative operator. > I knew I might get in trouble by giving such a specific example. My view is that a new @ operator is "baggage free" in terms of implied semantics -- since it has defined semantics only for matrices. To the extent that it has implied semantics of being multiplication-like, I think it would be surprising that one multiplication operator is left-associative and one is right-associative. Does @ have higher or lower precedence than *? The draft PEP says @ is the same as * but it can't be if it has opposite associativity. The interpretations of these two expressions depends on the precedence: A @ B * C A * B @ C Someone mentioned that APL is right-associative and has no operator-precedence. Those two things are related. There's no operator precedence because there are too many operators for everyone to remember the exact order of precedence. Given that, all operators need to be left or right associative. I don't know the reason for sure but I have heard that one reason is that and given that monadic (unary) operators are right associative, making all operators right associative means that you can simply interpret an APL statement reading from right to left. I don't think APL is the right model to look at. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140317/45429967/attachment.html> From steve at pearwood.info Tue Mar 18 05:58:04 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 18 Mar 2014 15:58:04 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lg6j55$7g1$1@ger.gmane.org> References: <CACac1F_T-nR+SwiPGSh0FRgESwuAmH-dpXwJZuzAjrwVjL162Q@mail.gmail.com> <CAKJDb-OCATCzHBAOYbrBmLHUf-trwnzhMqX02WLjbwU26JuS1Q@mail.gmail.com> <CACac1F8w0KofW98BJb95+sGtS9c6e544Rtqw2UrLo6AnGuYBAw@mail.gmail.com> <CADQLQrVxacppw_yyNdVh0jum0RuYw94=BWD_NYqgVzYXhUrgBw@mail.gmail.com> <53238DB1.7060608@canterbury.ac.nz> <CAGu0AnsjEU_G0VLDN5CiNxN2whRvpu1gVowmBXqSouA6eyS8_w@mail.gmail.com> <20140315073508.GB16526@ando> <CAGu0Anvf9L2U=CKXSuJWrgsLiwpNAsGdH4zLRoc6eFpsQFO_ww@mail.gmail.com> <CADiSq7duJirwK9aCGD+8YpXUPdf=ypkOS2RcbPhYUdUpKDnf4A@mail.gmail.com> <lg6j55$7g1$1@ger.gmane.org> Message-ID: <20140318045803.GA29663@ando> On Mon, Mar 17, 2014 at 10:35:37AM +0000, Robert Kern wrote: > On 2014-03-17 05:03, Nick Coghlan wrote: > >There are plenty of existing left-associative operators that could be > >used for that if Mike Bayer wanted (or could be convinced) to do so, > >so I don't see how that observation is relevant to the question of > >whether or not this particular proposal should be for a > >right-associative operator. > > Although this does suggest that having a right-associative operator might > provide some useful variety in the operator-overloading-as-DSL ecosystem. That's a good point. That suggests that adding a second right-associative operator may be a good idea even if it's not necessary for numpy. -- Steven From stephen at xemacs.org Tue Mar 18 07:29:45 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 18 Mar 2014 15:29:45 +0900 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> Message-ID: <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp> Alex Rodrigues writes: > I'm +1 for this idea. It's not about whether the style is good > (which its not), but its fundamentally about the fact that it will > fix bugs. It's not about fixing bugs. There are lots of bugs which could be fixed by changing Python behavior. It's about the fact that it almost surely doesn't penalize any correct programs that depend on the existing behavior. From ncoghlan at gmail.com Tue Mar 18 09:02:10 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2014 18:02:10 +1000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> Message-ID: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> On 18 Mar 2014 09:08, "Nathaniel Smith" <njs at pobox.com> wrote: > > On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > > A few other miscellaneous comments: > > > > - nice work on the PEP Nathaniel! > > Thanks! > > > - as with others, "@" as the operator doesn't thrill me, but I also think it > > crosses the threshold of "good enough given the constraints" > > - the PEP should probably recommend adding an "operator.matmul" function, a > > "PyObject_MatrixMultiply" C API and consider whether or not the new special > > method should be given a C level type slot. > > operator.matmul and PyObject_MatrixMultiply are obvious enough, but > I'm afraid I'm not too clear on the tradeoffs about adding a C level > type slot, or even entirely sure what the alternative is. (I guess I > just assumed that all special methods used C level type slots and > there was nothing to think about.) Do you (or anyone) have any > thoughts? I suspect you're going to want one, as without it, the implementation method ends up in the class dict instead (the context management protocol works that way). I suspect the design we will want is a new struct for Py_Matrix slots (akin to those for numbers, etc). The alternative would be to just add more "Number" slots, but that isn't really accurate. I'll come up with a more specific proposal after refreshing my memory of the exact details of the current layout. Cheers, Nick. > > -- > Nathaniel J. Smith > Postdoctoral researcher - Informatics - University of Edinburgh > http://vorpus.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/63640e51/attachment.html> From abarnert at yahoo.com Tue Mar 18 11:39:26 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Mar 2014 03:39:26 -0700 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> Message-ID: <68ED2909-56B0-4BF3-A6E9-0FB10435B6F7@yahoo.com> On Mar 18, 2014, at 1:02, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 18 Mar 2014 09:08, "Nathaniel Smith" <njs at pobox.com> wrote: > > > > On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > > > A few other miscellaneous comments: > > > > > > - nice work on the PEP Nathaniel! > > > > Thanks! > > > > > - as with others, "@" as the operator doesn't thrill me, but I also think it > > > crosses the threshold of "good enough given the constraints" > > > - the PEP should probably recommend adding an "operator.matmul" function, a > > > "PyObject_MatrixMultiply" C API and consider whether or not the new special > > > method should be given a C level type slot. > > > > operator.matmul and PyObject_MatrixMultiply are obvious enough, but > > I'm afraid I'm not too clear on the tradeoffs about adding a C level > > type slot, or even entirely sure what the alternative is. (I guess I > > just assumed that all special methods used C level type slots and > > there was nothing to think about.) Do you (or anyone) have any > > thoughts? > > I suspect you're going to want one, as without it, the implementation method ends up in the class dict instead (the context management protocol works that way). > > I suspect the design we will want is a new struct for Py_Matrix slots (akin to those for numbers, etc). The alternative would be to just add more "Number" slots, but that isn't really accurate. > Someone needs to work out how these changes will affect the numpy C API (the Py_Array dotfunc slot, the PyArray interface, etc.). Whoever is doing that--and the numpy C API users who are interested in it--will probably care more about how matmul fits into the python C API than anyone else does. As for whether this is another numeric operation or a new kind of operation, I think that depends on how you intuitively interpret matrix operations, or rather which intuition numpy and similar libraries should be encouraging. The fact that the proposal suggests that >2d arrays should handle matmul by broadcasting it over the final 2d subarrays seems like it's treating it fundamentally as a numeric operation on (vectors and) matrices. On the other hand, I don't think anyone implementing a split-complex or quaternion library on top of numpy would want to use @ for multiplying their numbers, even if they used @ on the matrices as the implementation. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/402ec756/attachment-0001.html> From robert.kern at gmail.com Tue Mar 18 11:47:02 2014 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 18 Mar 2014 10:47:02 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> Message-ID: <lg986n$n63$1@ger.gmane.org> On 2014-03-18 08:02, Nick Coghlan wrote: > > On 18 Mar 2014 09:08, "Nathaniel Smith" > <njs at pobox.com > <mailto:njs at pobox.com>> wrote: > > > > On Fri, Mar 14, 2014 at 8:15 PM, Nick Coghlan > <ncoghlan at gmail.com > <mailto:ncoghlan at gmail.com>> wrote: > > > A few other miscellaneous comments: > > > > > > - nice work on the PEP Nathaniel! > > > > Thanks! > > > > > - as with others, "@" as the operator doesn't thrill me, but I also think it > > > crosses the threshold of "good enough given the constraints" > > > - the PEP should probably recommend adding an "operator.matmul" function, a > > > "PyObject_MatrixMultiply" C API and consider whether or not the new special > > > method should be given a C level type slot. > > > > operator.matmul and PyObject_MatrixMultiply are obvious enough, but > > I'm afraid I'm not too clear on the tradeoffs about adding a C level > > type slot, or even entirely sure what the alternative is. (I guess I > > just assumed that all special methods used C level type slots and > > there was nothing to think about.) Do you (or anyone) have any > > thoughts? > > I suspect you're going to want one, as without it, the implementation method > ends up in the class dict instead (the context management protocol works that way). > > I suspect the design we will want is a new struct for Py_Matrix slots (akin to > those for numbers, etc). The alternative would be to just add more "Number" > slots, but that isn't really accurate. Would it be more palatable if the name were something like __altmul__ or __auxmul__ rather than __matmul__? Really, it's just a second multiplication-like operator. The leading use case for a second multiplication-like operator happens to be matrix multiplication, but I strongly suspect it will get used for other mathematical things like symbolic function composition or operator application (as in "linear operator", not +-*/) and maybe some secondary multiplication types in the weirder groups and fields (you can bet I will resurrect my Clifford algebra module to use this operator for one of the several types of multiplication they support). Granted, there is still some awkwardness in that *none* of the builtin number types will support it. And hey, look! It makes the @ux or @lt multiplication operator makes a sensible, language-independent mnemonic! (not that "auxiliary" or "alternate" work in all languages, but at least the 'a' is baked into the special method name) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ncoghlan at gmail.com Tue Mar 18 12:27:36 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2014 21:27:36 +1000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lg986n$n63$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> <lg986n$n63$1@ger.gmane.org> Message-ID: <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote: > On 2014-03-18 08:02, Nick Coghlan wrote: >> > operator.matmul and PyObject_MatrixMultiply are obvious enough, but >> > I'm afraid I'm not too clear on the tradeoffs about adding a C level >> > type slot, or even entirely sure what the alternative is. (I guess I >> > just assumed that all special methods used C level type slots and >> > there was nothing to think about.) Do you (or anyone) have any >> > thoughts? >> >> I suspect you're going to want one, as without it, the implementation >> method >> ends up in the class dict instead (the context management protocol works >> that way). >> >> I suspect the design we will want is a new struct for Py_Matrix slots >> (akin to >> those for numbers, etc). The alternative would be to just add more >> "Number" >> slots, but that isn't really accurate. So, here's the change to PyHeapType object that makes the most sense to me (assuming both "@" and "@@" are added - drop the new "power" methods if "@@" is dropped from the PEP): - add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject - define PyMatrixMethods as: typedef struct { binaryfunc mt_multiply; binaryfunc mt_power; binaryfunc mt_inplace_multiply; binaryfunc mt_inplace_power; } PyMatrixMethods; This approach increases the size of all type objects by one pointer. The other way to do it would be to just add four new slots to PyNumberMethods: binaryfunc nb_matrix_multiply; binaryfunc nb_matrix_power; binaryfunc nb_inplace_matrix_multiply; binaryfunc nb_inplace_matrix_power; This approach increases the size of all type objects that define one or more of the numeric functions by four pointers, and doesn't really make sense at a conceptual level. The latter is the main reason I prefer the separate PyMatrixMethods struct. Other "should probably be listed in the PEP for completeness" change is that this will need new opcodes and AST nodes. Reviewing the current opcode list and node names, I would suggest: BINARY_MATRIX_MULTIPLY BINARY_MATRIX_POWER INPLACE_MATRIX_MULTIPLY INPLACE_MATRIX_POWER MatMult | MatPow > Would it be more palatable if the name were something like __altmul__ or > __auxmul__ rather than __matmul__? Really, it's just a second > multiplication-like operator. The leading use case for a second > multiplication-like operator happens to be matrix multiplication, but I > strongly suspect it will get used for other mathematical things like > symbolic function composition or operator application (as in "linear > operator", not +-*/) and maybe some secondary multiplication types in the > weirder groups and fields (you can bet I will resurrect my Clifford algebra > module to use this operator for one of the several types of multiplication > they support). Granted, there is still some awkwardness in that *none* of > the builtin number types will support it. I think "matmul" is fine. That makes the primary intended use case clear, without preventing its use for other purposes (like vector dot products or more exotic things). The magic method names are "add", "mul", "div", "mod", etc, even though we occasionally use them for other purposes (e.g. concatenation, sequence repetition, path joining, interpolation). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mal at egenix.com Tue Mar 18 13:18:56 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 18 Mar 2014 13:18:56 +0100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> <lg986n$n63$1@ger.gmane.org> <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> Message-ID: <532839B0.7020108@egenix.com> On 18.03.2014 12:27, Nick Coghlan wrote: > On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote: >> On 2014-03-18 08:02, Nick Coghlan wrote: >>> > operator.matmul and PyObject_MatrixMultiply are obvious enough, but >>> > I'm afraid I'm not too clear on the tradeoffs about adding a C level >>> > type slot, or even entirely sure what the alternative is. (I guess I >>> > just assumed that all special methods used C level type slots and >>> > there was nothing to think about.) Do you (or anyone) have any >>> > thoughts? >>> >>> I suspect you're going to want one, as without it, the implementation >>> method >>> ends up in the class dict instead (the context management protocol works >>> that way). >>> >>> I suspect the design we will want is a new struct for Py_Matrix slots >>> (akin to >>> those for numbers, etc). The alternative would be to just add more >>> "Number" >>> slots, but that isn't really accurate. > > > So, here's the change to PyHeapType object that makes the most sense > to me (assuming both "@" and "@@" are added - drop the new "power" > methods if "@@" is dropped from the PEP): > > - add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject > - define PyMatrixMethods as: > > typedef struct { > binaryfunc mt_multiply; > binaryfunc mt_power; > binaryfunc mt_inplace_multiply; > binaryfunc mt_inplace_power; > } PyMatrixMethods; > > This approach increases the size of all type objects by one pointer. > > The other way to do it would be to just add four new slots to PyNumberMethods: > > binaryfunc nb_matrix_multiply; > binaryfunc nb_matrix_power; > binaryfunc nb_inplace_matrix_multiply; > binaryfunc nb_inplace_matrix_power; > > This approach increases the size of all type objects that define one > or more of the numeric functions by four pointers, and doesn't really > make sense at a conceptual level. The latter is the main reason I > prefer the separate PyMatrixMethods struct. I don't think that it's a good idea to make all type objects larger just to address matrix multiplications which none of the builtin types will actually use, so +1 on adding to the number methods, even if a matrix isn't a number (they still use numbers, so it's not that far off :-)), but -1 on adding another slot struct. Aside: Mechanisms such as namedtuple add lots of new type objects to the runtime environment, so it's no longer safe to assume that the size of type objects doesn't really matter much in real-life applications. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 18 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 11 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 22 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From guido at python.org Tue Mar 18 16:06:35 2014 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2014 08:06:35 -0700 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp> References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com> FWIW what *is* the status of this proposal? It feels reasonable to change this in 3.5. My friend at SourceGraph found no evidence of the behavior being relied on in real code (it did find the unit test :-). Although I'm sure he only searched a limited corpus. If someone wants to do a more thorough search before deciding I can point them to the software needed. On Mon, Mar 17, 2014 at 11:29 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote: > Alex Rodrigues writes: > > > I'm +1 for this idea. It's not about whether the style is good > > (which its not), but its fundamentally about the fact that it will > > fix bugs. > > It's not about fixing bugs. There are lots of bugs which could be > fixed by changing Python behavior. It's about the fact that it almost > surely doesn't penalize any correct programs that depend on the > existing behavior. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/bd5bd0fb/attachment.html> From donald at stufft.io Tue Mar 18 16:10:10 2014 From: donald at stufft.io (Donald Stufft) Date: Tue, 18 Mar 2014 11:10:10 -0400 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com> References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com> Message-ID: <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io> On Mar 18, 2014, at 11:06 AM, Guido van Rossum <guido at python.org> wrote: > FWIW what *is* the status of this proposal? It feels reasonable to change this in 3.5. My friend at SourceGraph found no evidence of the behavior being relied on in real code (it did find the unit test :-). Although I'm sure he only searched a limited corpus. If someone wants to do a more thorough search before deciding I can point them to the software needed. > > > On Mon, Mar 17, 2014 at 11:29 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > Alex Rodrigues writes: > > > I'm +1 for this idea. It's not about whether the style is good > > (which its not), but its fundamentally about the fact that it will > > fix bugs. > > It's not about fixing bugs. There are lots of bugs which could be > fixed by changing Python behavior. It's about the fact that it almost > surely doesn't penalize any correct programs that depend on the > existing behavior. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ Right now I think it?s waiting on someone to actually do it. Personally I was waiting on development on 3.5 to open up before I bothered to see if I could figure out how to do this or not :) If someone else gets to it before me all the better though! ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/0211e27f/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140318/0211e27f/attachment-0001.sig> From ethan at stoneleaf.us Tue Mar 18 16:18:53 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 18 Mar 2014 08:18:53 -0700 Subject: [Python-ideas] Please reconsider the Boolean evaluation of midnight In-Reply-To: <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io> References: <BLU171-W88536CD9D3BE6B8BBE79CDB97D0@phx.gbl> <87vbvcj9ye.fsf@uwakimon.sk.tsukuba.ac.jp> <CAP7+vJLo-CU_vNQeyR5duMA+xx5YQBk1n-sxaFDM+dUTFCLwOA@mail.gmail.com> <AC6EF771-5521-4477-97C0-A047AA900191@stufft.io> Message-ID: <532863DD.6030202@stoneleaf.us> On 03/18/2014 08:10 AM, Donald Stufft wrote: > On Mar 18, 2014, at 11:06 AM, Guido van Rossum wrote: >> >> FWIW what *is* the status of this proposal? > > Right now I think it?s waiting on someone to actually do it. Personally I was > waiting on development on 3.5 to open up before I bothered to see if I could > figure out how to do this or not :) > > If someone else gets to it before me all the better though! It's on my to-do list, but it'll be a couple months before I get to it. Race ya! :) -- ~Ethan~ From abarnert at yahoo.com Tue Mar 18 20:59:30 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Mar 2014 12:59:30 -0700 (PDT) Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <532839B0.7020108@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> <lg986n$n63$1@ger.gmane.org> <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> <532839B0.7020108@egenix.com> Message-ID: <1395172770.13326.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: M.-A. Lemburg <mal at egenix.com> Sent: Tuesday, March 18, 2014 5:18 AM > On 18.03.2014 12:27, Nick Coghlan wrote: >> The other way to do it would be to just add four new slots to >> PyNumberMethods: >> >> ? ? ? ? binaryfunc nb_matrix_multiply; >> ? ? ? ? binaryfunc nb_matrix_power; >> ? ? ? ? binaryfunc nb_inplace_matrix_multiply; >> ? ? ? ? binaryfunc nb_inplace_matrix_power; >> >> This approach increases the size of all type objects that define one >> or more of the numeric functions by four pointers, and doesn't really >> make sense at a conceptual level. The latter is the main reason I >> prefer the separate PyMatrixMethods struct. > > I don't think that it's a good idea to make all type objects > larger just to address matrix multiplications which none of the > builtin types will actually use, so +1 on adding to the number > methods, even if a matrix isn't a number (they still use numbers, > so it's not that far off :-)), but -1 on adding another slot > struct. I don't see how putting them in PyNumberMethods doesn't make sense at a conceptual level. (Matrix, +, @) is a ring, and they form an assocative algebra. Sure, matrices are collections of simpler numbers, and their operations are defined in terms of the operations of those simpler numbers, but the same thing is true for complex numbers, or even rationals. From a mathematical point of view, there's no principled definition of "number" (or, rather, if there is, it's only going to include the naturals or maybe the ordinals). If anything, the float type has a lot more problems as a "number" than a matrix type does. So if there is a good practical reason to call matrix multiplication a numeric operation, why not? From tarek at ziade.org Tue Mar 18 20:57:48 2014 From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 18 Mar 2014 20:57:48 +0100 Subject: [Python-ideas] A resources tracker ? Message-ID: <5328A53C.2080604@ziade.org> Hey ResourceWarning instances that are created in some classes' __del__ function, like FileIO are a great tool to track down a program bad behavior. In network programming, that's even more important to avoid crashes, or huge leaks. But some of them are very hard to fix because we don't get much context, we just get warnings at the end of the program execution, when gc.collect is called. sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb'> sys:1: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb'> Here I just know that somewhere, 2 file descriptors where not closed. What I'd like to be able to do is to track down the origin of those warnings. Since __del__ is called asynchronously, it's impossible to track it right now (or I don't know how) What we need is a way to keep track of any resource allocation *when it happens*. Here's an idea: let's add three private functions in Python's io: def __allocate_resource(fd) => records the file descriptor that was allocated, along with the current traceback. def __free_resource(fd) => removes the fd from the list. def __is_resource_allocated(fd) => tell if the resource is in the list. These three functions, plugged in somewhere in io's classes, could be used in conjunction with ResourceWarning: when __del__ is called, if the resource was not freed - we'd be able to know where it was created. Of course these functions are just a brain dump - I have no idea how io internals work. But unless I missed it, something like I've just described is missing in Python. Cheers Tarek From abarnert at yahoo.com Tue Mar 18 21:49:58 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Mar 2014 13:49:58 -0700 Subject: [Python-ideas] A resources tracker ? In-Reply-To: <5328A53C.2080604@ziade.org> References: <5328A53C.2080604@ziade.org> Message-ID: <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com> On Mar 18, 2014, at 12:57, Tarek Ziad? <tarek at ziade.org> wrote: > def __allocate_resource(fd) => records the file descriptor that was > allocated, along with the current traceback. > def __free_resource(fd) => removes the fd from the list. > def __is_resource_allocated(fd) => tell if the resource is in the list I like the general idea. I've actually written wrappers (in Python 2.x and other languages like C++, never Python 3, but similar idea...) to track these kinds of leaks myself. I don't think you need all of these methods. Just the first one will do it: if you get to __del__ without a close and emit a ResourceWarning, use the info stashed by the allocate function; otherwise, it never gets looked at. And I'm not sure it needs to be a method after all. What about cases where the io object doesn't actually allocate anything (because you got it from an fd or a socket object or similar?), but you still need to call close. Don't you want the traceback in those cases too? Also, in the most common cases (like open), you're actually creating a chain of two or three objects; do all of them need to store this info? I don't think you want this on all the time... But when exactly _do_ you want it on? Debug mode? If the warning is enabled at allocation time? A global flag on the io module? Also, storing an actual traceback is probably a bad idea, as it keeps all kinds of things alive for a very long time. Maybe storing a string representation, or enough info to generate such a representation in the ResourceWarning? Also, I suspect that knowing the arguments used to allocate the resource (like the filename passed to open or the object's constructor, in the most common case, but also things like the cloexec flag when debugging multi-process apps) might be at least as useful as the traceback, so you might as well add that too. Anyway, there's pure Python code in Lib/io.py that wraps up the C code. To experiment with this without getting into the C stuff, I think you could edit it to use your own classes that wrap _io.BufferedWriter, etc. instead of just exporting those directly. From ncoghlan at gmail.com Tue Mar 18 21:58:20 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Mar 2014 06:58:20 +1000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <532839B0.7020108@egenix.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> <lg986n$n63$1@ger.gmane.org> <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> <532839B0.7020108@egenix.com> Message-ID: <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com> On 18 Mar 2014 22:18, "M.-A. Lemburg" <mal at egenix.com> wrote: > > On 18.03.2014 12:27, Nick Coghlan wrote: > > On 18 March 2014 20:47, Robert Kern <robert.kern at gmail.com> wrote: > >> On 2014-03-18 08:02, Nick Coghlan wrote: > >>> > operator.matmul and PyObject_MatrixMultiply are obvious enough, but > >>> > I'm afraid I'm not too clear on the tradeoffs about adding a C level > >>> > type slot, or even entirely sure what the alternative is. (I guess I > >>> > just assumed that all special methods used C level type slots and > >>> > there was nothing to think about.) Do you (or anyone) have any > >>> > thoughts? > >>> > >>> I suspect you're going to want one, as without it, the implementation > >>> method > >>> ends up in the class dict instead (the context management protocol works > >>> that way). > >>> > >>> I suspect the design we will want is a new struct for Py_Matrix slots > >>> (akin to > >>> those for numbers, etc). The alternative would be to just add more > >>> "Number" > >>> slots, but that isn't really accurate. > > > > > > So, here's the change to PyHeapType object that makes the most sense > > to me (assuming both "@" and "@@" are added - drop the new "power" > > methods if "@@" is dropped from the PEP): > > > > - add "PyMatrixMethods as_matrix;" as a new field in PyHeapTypeObject > > - define PyMatrixMethods as: > > > > typedef struct { > > binaryfunc mt_multiply; > > binaryfunc mt_power; > > binaryfunc mt_inplace_multiply; > > binaryfunc mt_inplace_power; > > } PyMatrixMethods; > > > > This approach increases the size of all type objects by one pointer. > > > > The other way to do it would be to just add four new slots to PyNumberMethods: > > > > binaryfunc nb_matrix_multiply; > > binaryfunc nb_matrix_power; > > binaryfunc nb_inplace_matrix_multiply; > > binaryfunc nb_inplace_matrix_power; > > > > This approach increases the size of all type objects that define one > > or more of the numeric functions by four pointers, and doesn't really > > make sense at a conceptual level. The latter is the main reason I > > prefer the separate PyMatrixMethods struct. > > I don't think that it's a good idea to make all type objects > larger just to address matrix multiplications which none of the > builtin types will actually use, so +1 on adding to the number > methods, even if a matrix isn't a number (they still use numbers, > so it's not that far off :-)), but -1 on adding another slot > struct. > > Aside: Mechanisms such as namedtuple add lots of new type > objects to the runtime environment, so it's no longer safe to > assume that the size of type objects doesn't really matter much > in real-life applications. So, here's the problem: operand precedence for sequences implemented in C is currently broken in certain relatively obscure cases (and has been since forever - we only found out about due to an SQL Alchemy failure when porting to PyPy revealed PyPy was right and CPython was wrong). This is why returning NotImplemented from sq_concat and sq_repeat doesn't work right. The problem doesn't arise for sequences implemented in Python, because we always populate the nb_add and nb_multiply slots for those. I've tried fixing that directly, and it turned abstract.c into an unmaintainable mess, so I abandoned that approach. My current preferred solution is now similar to the one we use from Python: drop the direct calls to sq_concat and sq_repeat from abstract.c, and instead automatically add nb_add and nb_multiply implementations during type creation that delegate dynamically to the sequence slots. So a *lot* of types already have their PyNumberMethods slot allocated (including sequence types implemented in Python), and I'd like to expand that to include all sequence types, even those implemented in C. That makes the space trade-off here substantially less clear, particularly if matrix power ends up being added in the future (it has been dropped from the current PEP). I accept that adding the new slots to PyNumberMethods is more conceptually coherent than I thought, though. There's also the fact that we already mess about populating different slots for pragmatic reasons that have nothing to do with conceptual integrity :) Cheers, Nick. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 18 2014) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2014-03-29: PythonCamp 2014, Cologne, Germany ... 11 days to go > 2014-04-09: PyCon 2014, Montreal, Canada ... 22 days to go > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/85498646/attachment-0001.html> From ncoghlan at gmail.com Tue Mar 18 22:04:25 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Mar 2014 07:04:25 +1000 Subject: [Python-ideas] A resources tracker ? In-Reply-To: <5328A53C.2080604@ziade.org> References: <5328A53C.2080604@ziade.org> Message-ID: <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com> On 19 Mar 2014 06:09, "Tarek Ziad?" <tarek at ziade.org> wrote: > > Hey > > ResourceWarning instances that are created in some classes' __del__ > function, like FileIO > are a great tool to track down a program bad behavior. > > In network programming, that's even more important to avoid crashes, or > huge leaks. > > But some of them are very hard to fix because we don't get much context, > we just > get warnings at the end of the program execution, when gc.collect is called. > > sys:1: ResourceWarning: unclosed file <_io.FileIO name=18 mode='wb'> > sys:1: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb'> > > Here I just know that somewhere, 2 file descriptors where not closed. > > What I'd like to be able to do is to track down the origin of those > warnings. > > Since __del__ is called asynchronously, it's impossible to track it > right now (or I don't know how) > > What we need is a way to keep track of any resource allocation *when it > happens*. > > Here's an idea: let's add three private functions in Python's io: > > def __allocate_resource(fd) => records the file descriptor that was > allocated, along with the current traceback. > def __free_resource(fd) => removes the fd from the list. > def __is_resource_allocated(fd) => tell if the resource is in the list. > > These three functions, plugged in somewhere in io's classes, could be > used in conjunction with ResourceWarning: > when __del__ is called, if the resource was not freed - we'd be able to > know where it was created. > > Of course these functions are just a brain dump - I have no idea how io > internals work. But unless I missed it, > something like I've just described is missing in Python. You should be able to experiment with something based on tracemalloc (although it may require patching the io implementation or else installing a GC callback that looks for particular types). Cheers, Nick. > > Cheers > Tarek > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/e746ce73/attachment.html> From tarek at ziade.org Tue Mar 18 22:07:28 2014 From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=) Date: Tue, 18 Mar 2014 22:07:28 +0100 Subject: [Python-ideas] A resources tracker ? In-Reply-To: <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com> References: <5328A53C.2080604@ziade.org> <3F8B02EB-BBAA-4663-8396-E445BF8344E0@yahoo.com> Message-ID: <5328B590.80700@ziade.org> Le 18/03/14 21:49, Andrew Barnert a ?crit : > On Mar 18, 2014, at 12:57, Tarek Ziad? <tarek at ziade.org> wrote: > >> def __allocate_resource(fd) => records the file descriptor that was >> allocated, along with the current traceback. >> def __free_resource(fd) => removes the fd from the list. >> def __is_resource_allocated(fd) => tell if the resource is in the list > I like the general idea. I've actually written wrappers (in Python 2.x and other languages like C++, never Python 3, but similar idea...) to track these kinds of leaks myself. > > I don't think you need all of these methods. Just the first one will do it: if you get to __del__ without a close and emit a ResourceWarning, use the info stashed by the allocate function; otherwise, it never gets looked at. And I'm not sure it needs to be a method after all. True. > > What about cases where the io object doesn't actually allocate anything (because you got it from an fd or a socket object or similar?), but you still need to call close. Don't you want the traceback in those cases too? Also, in the most common cases (like open), you're actually creating a chain of two or three objects; do all of them need to store this info? I guess the allocate_resource function would be only called when the FD is created. And we'd want this initial traceback afaik > > I don't think you want this on all the time... But when exactly _do_ you want it on? Debug mode? If the warning is enabled at allocation time? A global flag on the io module? Yeah that would be costly, I was thinking about some kind of debug flag to activate it. I would not mind having to compile python --with-debug-leaks just to get that kind of tooling. > > Also, storing an actual traceback is probably a bad idea, as it keeps all kinds of things alive for a very long time. Maybe storing a string representation, or enough info to generate such a representation in the ResourceWarning? yeah good point. > > Also, I suspect that knowing the arguments used to allocate the resource (like the filename passed to open or the object's constructor, in the most common case, but also things like the cloexec flag when debugging multi-process apps) might be at least as useful as the traceback, so you might as well add that too. > > Anyway, there's pure Python code in Lib/io.py that wraps up the C code. To experiment with this without getting into the C stuff, I think you could edit it to use your own classes that wrap _io.BufferedWriter, etc. instead of just exporting those directly. According to my attempts, It's hard to make sure all the calls are really going through those classes. Another approach is to hack ResourceWarning itself to give the idea a try Cheers From tarek at ziade.org Tue Mar 18 22:19:06 2014 From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 18 Mar 2014 22:19:06 +0100 Subject: [Python-ideas] A resources tracker ? In-Reply-To: <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com> References: <5328A53C.2080604@ziade.org> <CADiSq7eU7P+Vi=zc-s16LD71FHu4WvxWA+4J0rhLJ_AqowF=Rw@mail.gmail.com> Message-ID: <5328B84A.5000602@ziade.org> Le 18/03/14 22:04, Nick Coghlan a ?crit : > > .. > > You should be able to experiment with something based on tracemalloc > (although it may require patching the io implementation or else > installing a GC callback that looks for particular types). > We've played a little bit with Victor on this today, using his script here: https://bitbucket.org/haypo/misc/src/tip/python/res_warn.py That did not fully work - I still need to investigate, but that's the general idea. I suspect modifying ResourceWarning itself to track down things would ensure we're not missing anything. Cheers Tarek From njs at pobox.com Tue Mar 18 22:48:27 2014 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 18 Mar 2014 21:48:27 +0000 Subject: [Python-ideas] overloading chained comparison Message-ID: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> Hi python-ideas, Since Nick is being a killjoy and shutting down bikeshedding in the @ thread [1] until the numpy folks finish bikeshedding over it themselves [2], I thought I'd throw out another idea for... brainstorming [3]. Guido has suggested that while PEP 335 (overloading 'and', 'or', 'not') is rejected, he might be amenable to making chained comparisons like a < b < c overloadable [4]. The idea is that right now, a < b < c always expands to (a < b) and (b < c) and the 'and' forces boolean coercion. When working with arrays, '<' is often used elementwise, so we want the 'and' to apply elementwise as well; in numpy, this is done using '&' (since we can't overload 'and'). So if a, b, c are numpy arrays, we'd like this to instead expand to (a < b) & (b < c) Similar considerations apply to other systems that overload the comparison operators, e.g. DSLs for generating SQL queries. This seems like a good and straightforward enough idea to me in principle, but it's not at all clear to me what the best way to accomplish it in practice is. I thought of three options, but none is obviously Right. To have a way to talk about our options precisely, let's pretend that there's something called operator.chain_comparison, and that the way it works is that a < b <= c produces a call like operator.chain_comparison([a, b, c], [operator.lt, operator.le]) Right now, the semantics are: # Current def chain_comparison(args, ops): for i, op in enumerate(ops): result = op(args[i], args[i + 1]): # short-circuit if not result: return result return result (Of course in reality in CPython the compiler unrolls the loop and inlines this directly into the bytecode, but whatever, that's an implementation detail.) IDEA 1: simple, neat and (sort of) wrong Let's define a new special method __chain_and__; whenever we do a chain comparison, we check for the presence of this method, and if found, we call it instead of using 'and'. The intuition is that if we have a < b < c then this expands to either (a < b) and (b < c) or (a < b).__chain_and__(b < c) depending on whether hasattr((a < b), "__chain_and__"). Notice that the first case is short-circuiting, and the second is not. Which seems totally fine (contra PEP 335), because short-circuiting by definition requires that you make a boolean decision (quit early/don't quit early), and the whole point of these overloads is to avoid boolean coercion. I think in general the semantics here look like: # __chain_and__ def chain_comparison(args, ops): so_far = True for i, op in enumerate(ops): result = op(args[i], args[i + 1]) if hasattr(so_far, "__chain_and__"): so_far = so_far.__chain_and__(result) else: so_far = so_far and result # short-circuit, but only if the next reduction would use 'and': if not hasattr(so_far, "__chain_and__") and not so_far: return so_far return so_far So if 'arr' is a numpy array, then code like 0 < arr < 1 will now work great, because (0 < arr) will return an array of booleans, and this array will have the __chain_and__ method, so we'll end up doing (0 < arr).__chain_and__(arr < 1) and successfully return a single array of booleans indicating which locations in 'arr' are between 0 and 1. But -- suppose that we have, say, a GUI displaying a barchart, and we have a horizontal "threshold" line across the barchart that the user control. The idea is that they can move it up and down, and all the bars that it overlaps will change color, so we can easily see which bars are above the threshold and which are below it. So given the current threshold 't' (an ordinary float or int), and a 1d array holding the various bar heights, we can request the set of overlapping bars as: 0 < x < arr Okay, first problem is that this fails because here we don't want to do (0 < x).__chain_and__(x < arr) because (0 < x) is just False. So okay, let's say we enhance the above definition to allow for __rchain_and__, and we get (x < arr).__rchain_and__(0 < x) Great! So we implement it and we test it in our program and it works -- under ordinary conditions, this spits out a nice 1d array of booleans, our code continues on its merry way, using the values in this array to decide how each bar in our plot should be colored. Until the user slides the threshold line down below zero, at which point (0 < x) starts returning False, and we short-circuit out before even evaluating (x < arr). And then our code blows up because instead of getting an array like we expected, we just get False instead. Oops. __rchain_and__ is useless. So I guess the solution is to write arr > x > 0 which will always work? I'm not sure how common this case is in practice, but I find it somewhat disturbing that with the __chain_and__ approach, a < b < c and c > b > a can return completely different values even for completely well-behaved objects, and there's nothing the person writing overload methods can do about it. Certainly the bug in our original code is not obvious to the casual reader. (And AFAICT the proposal in PEP 335 also has this problem -- in fact PEP 335 basically is the same as this proposal -- so no help there.) IDEA 2: the FULLY GENERAL solution (uh oh) So we started with a call like operator.chain_comparison([a, b, c], [operator.lt, operator.le]) Maybe what we should do is treat *this* as the basic operator, and try calling a special __chain_comparison__ method on a, b, and/or c. Of course this immediately runs into a problem, because all python's existing operators have only 1 or 2 arguments [5], not an indefinite and varying number of arguments. So we can't use the standard __X__/__rX__ dispatch strategy. We need something like multimethod dispatch. (Cue thunder, ominous music.) I know this has been discussed to death, and I haven't read the discussion, so I guess people can have fun educating me if they feel like it. But what I'd suggest in this case is to do what we did in numpy to solve a similar problem [6]. Instead of using a "real" multimethod system, just directly generalize the __X__/__rX__ trick: when looking for a candidate object to call __chain_comparison__ on, take the leftmost one that (a) hasn't been tried, and (b) doesn't have another object which is a proper subclass of its type that also hasn't been tried. If you get NotImplemented, keep trying until you run out of candidates; then fall back to the traditional 'and'-based semantics. This does solve our problematic case above: only 'arr' implements __chain_comparison__, so we have 0 < x < arr becoming arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt]) and then that can just call the underlying rich comparison operators in a non-short-circuiting loop, and combine the results using '&' instead of 'and'. But it does require a somewhat odd looking piece of machinery for the dispatch. OPTION 3: the FULLY SPECIFIC solution A nice thing about the __chain_comparison__ method is that it will actually be identical for all array-like objects, no matter which library they're defined in. So, if there are multiple array-like objects from different libraries in a single chain, it doesn't matter who gets picked to handle the overall evaluation -- any one of them is good enough, and then the actual interoperability problems are delegated to the rich comparison methods, which already have to have a strategy for dealing with them. In fact, this same __chain_comparison__ can also probably be used for just about anyone who wants to define it -- e.g., I think most DB query DSLs are going to overload & to mean "and", right? So, maybe all we need is a flag that says: if you see an object with this flag anywhere in a chain, then switch to these semantics: # &-flag def chain_comparison(args, ops): for i, op in enumerate(ops): result = op(args[i], args[i + 1]): if i == 0: combined = result else: combined &= result return combined ...and otherwise, use the standard semantics. So that's what I got. None of these approaches seems obviously Right, but they're all at least sort of viable. Anyone else got any ideas? -n [1] https://mail.python.org/pipermail/python-ideas/2014-March/027174.html [2] http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069444.html [3] "We're more of the love, bikeshedding, and rhetoric school. Well, we can do you bikeshedding and love without the rhetoric, and we can do you bikeshedding and rhetoric without the love, and we can do you all three concurrent or consecutive. But we can't give you love and rhetoric without the bikeshedding. Bikeshedding is compulsory." [4] https://mail.python.org/pipermail/python-dev/2012-March/117510.html [5] Except pow(), but that doesn't really count because it never dispatches on the third argument. [6] The next release of numpy allows non-numpy array-like classes to handle all numpy math functions in a generic way by dispatch to a special __numpy_ufunc__ method; I can provide more details on what's going on here if anyone's curious: http://docs.scipy.org/doc/numpy-dev/reference/arrays.classes.html#numpy.class.__numpy_ufunc__ -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From ncoghlan at gmail.com Tue Mar 18 23:35:54 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Mar 2014 08:35:54 +1000 Subject: [Python-ideas] overloading chained comparison In-Reply-To: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> Message-ID: <CADiSq7f4x1Jj9Z9TnE4yqHhRvxH3_=iAaOkA4nRNkKXts0Bndw@mail.gmail.com> On 19 Mar 2014 07:49, "Nathaniel Smith" <njs at pobox.com> wrote: > [3] "We're more of the love, bikeshedding, and rhetoric school. Well, > we can do you bikeshedding and love without the rhetoric, and we can > do you bikeshedding and rhetoric without the love, and we can do you > all three concurrent or consecutive. But we can't give you love and > rhetoric without the bikeshedding. Bikeshedding is compulsory." I haven't formulated an opinion on the actual question yet, but I just wanted to say this is one of my favourite descriptions of python-ideas/dev ever :) Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/e1e9e168/attachment.html> From mal at egenix.com Wed Mar 19 00:38:54 2014 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Mar 2014 00:38:54 +0100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <CAGRr6BFcfThXqzAqfYEzOS2Wg8g1VEv56CYW+LZjcGREsh8hDg@mail.gmail.com> <CADiSq7fEYeENppwruSHTEqfx9A0tv+wmMW5EuoNPrU9M-xqPwQ@mail.gmail.com> <CAPJVwBm2oqRnXaE+DAvVzDUQ=p-SnqA23CAfiJ4qEe_DetJgAg@mail.gmail.com> <CADiSq7c48OvkftXJJc7hjfkgS6qiGmAzS_aty6GNpADx3z6s3g@mail.gmail.com> <lg986n$n63$1@ger.gmane.org> <CADiSq7e_pk7TZqEtQK8f-ZBS+6=5tfwymRU69evf55K8mnzmTg@mail.gmail.com> <532839B0.7020108@egenix.com> <CADiSq7e26UDibOdwEJ=zsWSXobngy+sVOMX41_XnM9FWUYxvhA@mail.gmail.com> Message-ID: <5328D90E.4030603@egenix.com> On 18.03.2014 21:58, Nick Coghlan wrote: > On 18 Mar 2014 22:18, "M.-A. Lemburg" <mal at egenix.com> wrote: >> [...matrix method slots space tradeoff...] > [... need for implementing number methods even on non-number types ...] > So a *lot* of types already have their PyNumberMethods slot allocated > (including sequence types implemented in Python), and I'd like to expand > that to include all sequence types, even those implemented in C. > > That makes the space trade-off here substantially less clear, particularly > if matrix power ends up being added in the future (it has been dropped from > the current PEP). Ok, point taken :-) > I accept that adding the new slots to PyNumberMethods is more conceptually > coherent than I thought, though. There's also the fact that we already mess > about populating different slots for pragmatic reasons that have nothing to > do with conceptual integrity :) Perhaps something to revisit for Python 4 ?! The refactoring could lead to some space savings, untangle code and make the C API easier to understand. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 19 2014) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2014-03-29: PythonCamp 2014, Cologne, Germany ... 10 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 21 days to go ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From greg.ewing at canterbury.ac.nz Wed Mar 19 01:03:06 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Mar 2014 13:03:06 +1300 Subject: [Python-ideas] overloading chained comparison In-Reply-To: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> Message-ID: <5328DEBA.1040000@canterbury.ac.nz> Nathaniel Smith wrote: > This does solve our problematic case above: only 'arr' implements > __chain_comparison__, so we have > 0 < x < arr > becoming > arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt]) I don't see how that works, because you need to evaluate arr in order to tell whether it has a __chain_comparison__ method. So chained comparisons would always have to evaluate all operands and could never short-circuit. -- Greg From aj at erisian.com.au Wed Mar 19 01:54:54 2014 From: aj at erisian.com.au (Anthony Towns) Date: Wed, 19 Mar 2014 10:54:54 +1000 Subject: [Python-ideas] OrderedDict literals Message-ID: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> Hi, I was re-reading some old threads about ordereddict literals to see if any of them had gotten anywhere. Amongst them, I came across a post by Tim Delaney: https://mail.python.org/pipermail/python-ideas/2011-January/009049.html that mentioned an odict literal of ['key': 'value', 'key2': 'value2'] could be confused with slice notation. >From a syntax point-of-view, that doesn't seem to be true (as mentioned in some of the replies to that thread), but it seems like you can abuse the similarity to make it a little easier to declare ordereddicts: from collections import OrderedDict class ODSlicer(object): def __getitem__(self, key): if type(key) is slice: key = [key] od = OrderedDict() for k in key: if type(k) is slice: od[k.start] = k.stop else: od[k] = k return od od = ODSlicer() print(od[1:2]) print(od["a":"b", "c":5]) print(od['a':'b', 'c':'d', ..., 'a':10, 'e':'f']) You could then replace: mydict = { 'foo': 'bar', 'baz': 'quux', } with: mydict = od[ 'foo': 'bar', 'baz': 'quux', ] if you need to convert a hardcoded dict into a hardcoded ordereddict. Works fine in python2.7 and python3.4. At this point, I'd like to note in my defence that this isn't called the python-good-ideas list :) What's the actual objection to supporting ['foo': 'bar'] odict literals? I saw Guido gave a -100, way back in the day, but no actual explanation for why it was distasteful? https://mail.python.org/pipermail/python-ideas/2009-June/004924.html Cheers, aj -- Anthony Towns <aj at erisian.com.au> From njs at pobox.com Wed Mar 19 02:56:00 2014 From: njs at pobox.com (Nathaniel Smith) Date: Wed, 19 Mar 2014 01:56:00 +0000 Subject: [Python-ideas] overloading chained comparison In-Reply-To: <5328DEBA.1040000@canterbury.ac.nz> References: <CAPJVwBnxxx-svhwTzjj8jFnROrFkMAAWZj9MC_zXkGrhv=v5Cg@mail.gmail.com> <5328DEBA.1040000@canterbury.ac.nz> Message-ID: <CAPJVwB=Hb-i+2ixmpO7gP_E6ugfWpjbhkUSc+iOe4PC7=YhkQQ@mail.gmail.com> On Wed, Mar 19, 2014 at 12:03 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Nathaniel Smith wrote: > >> This does solve our problematic case above: only 'arr' implements >> __chain_comparison__, so we have >> 0 < x < arr >> becoming >> arr.__chain_comparison__([0, x, arr], [operator.lt, operator.lt]) > > > I don't see how that works, because you need to evaluate > arr in order to tell whether it has a __chain_comparison__ > method. So chained comparisons would always have to evaluate > all operands and could never short-circuit. Blah, you're absolutely right of course, total brainfart about what short-circuiting means, even though it was right there in the disassembly I was looking at. So yes, I guess the only meaningful part of my message is just the observation that short-circuiting makes any kind of chained comparison overload IMHO rather unsatisfying, because it forces you to make the ugly rule that the special overloading arguments must always occur in one of the first two positions. Which is probably true by accident 99% of the time, so as long as short-circuiting exists at all, supporting chained comparison overload will mean creating an obscure trap. On the one hand, I am kind of terrified of the idea of code that depends on short-circuiting here, like: 0 < x < side_effecting_function_only_called_for_negative_x() It would be interesting to know how many chained comparisons exist in the wild with non-trivial 3rd+ arguments. On the other hand, disabling short-circuiting in general for chained comparisons would still be a technical compatibility break. Somehow adding py3's first __future__ feature just for this seems like a lot to ask... -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From abarnert at yahoo.com Wed Mar 19 07:44:41 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 18 Mar 2014 23:44:41 -0700 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> Message-ID: <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com> On Mar 18, 2014, at 17:54, Anthony Towns <aj at erisian.com.au> wrote: > I was re-reading some old threads about ordereddict literals to see if > any of them had gotten anywhere. Skimming the linked thread, there were a bunch of messages with variations of "It might be easier to judge if you came up with a real use case", but nobody ever did. So, do you have a real use case here? It might look better with headers from some protocol that need to be in order (except I can't think of any such protocol) or whatever you're hoping for than with examples like 'a' or 'foo' or 1. From aj at erisian.com.au Wed Mar 19 08:39:11 2014 From: aj at erisian.com.au (Anthony Towns) Date: Wed, 19 Mar 2014 17:39:11 +1000 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com> Message-ID: <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com> On 19 March 2014 16:44, Andrew Barnert <abarnert at yahoo.com> wrote: > On Mar 18, 2014, at 17:54, Anthony Towns <aj at erisian.com.au> wrote: >> I was re-reading some old threads about ordereddict literals to see if >> any of them had gotten anywhere. > Skimming the linked thread, there were a bunch of messages with variations of "It might be easier to judge if you came up with a real use case", but nobody ever did. > So, do you have a real use case here? It might look better with headers from some protocol that need to be in order (except I can't think of any such protocol) or whatever you're hoping for than with examples like 'a' or 'foo' or 1. Oh, sure. The case that I've done most recently was when writing a tool to let me convert yaml files to and from jira issues. I want to specify how to convert each field between yaml and jira, and I also want to specify the order the fields end up in in the yaml file so that they're predictable. That's static information that I want to include in my source, and having it as a single ordereddict is the easiest way to write and store it. (The reason I want the ordering is both so that (a) if I dump an issue from jira to a file one day, and store it in git, then do it again the next day, the diff catches any intentional changes, not just random ordering changes, and (b) when I'm looking at the yaml file, the fields are in an order that makes sense to me so I know where to look for particular information). The code I ended up with was: class InterestingFields(OrderedDict): def __init__(self, extras): super(InterestingFields, self).__init__() D_key = D("key") D_name = D("name") s = S() t = T() f = self f["project"] = D_key f["issuetype"] = D_name f["summary"] = s # 18 more key/value pairs ... for k,v in extras: f[k] = v where D,S,T are classes that provide functions to convert from the Jira REST representation to the python version of the YAML representation I want and vice-versa. It took me a while to do the above in a way I didn't totally hate ("oh, I'll just do an OrderedDict( project=D_key, issuetype=D_name, ...)... huh, why didn't that work?"), and I can't say I *liked* what I ended up with. Better than specifying the fields twice (once for how to convert, once for the ordering), and better than all the heiroglyphics involved in writing it as a list of pairs though. The use case that was crossing my mind this morning was being able to write: return { "result": "ok", "details": "everything worked wonderfully" } from a function whose result gets turned into json and returned over a REST API, and have the order preserved so that when debugging with curl, I wouldn't see the "details" come first and confuse myself. Haven't decided how much I care about that yet. Cheers, aj -- Anthony Towns <aj at erisian.com.au> From mertz at gnosis.cx Wed Mar 19 09:00:51 2014 From: mertz at gnosis.cx (David Mertz) Date: Wed, 19 Mar 2014 01:00:51 -0700 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> Message-ID: <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com> It doesn't feel to me like the existing (non-literal) spelling is particularly bad: >>> o1 = OrderedDict(foo=1, bar=2, baz=3) >>> o2 = OrderedDict([('foo',1), ('bar',2), ('baz',3)]) >>> o1 OrderedDict([('bar', 2), ('foo', 1), ('baz', 3)]) >>> o2 OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)]) So sure, you can't use the kw=val style of initialization, but the "list of pairs" style works fine. It's not that much longer than a literal would be. On Tue, Mar 18, 2014 at 5:54 PM, Anthony Towns <aj at erisian.com.au> wrote: > Hi, > > I was re-reading some old threads about ordereddict literals to see if > any of them had gotten anywhere. Amongst them, I came across a post by > Tim Delaney: > > https://mail.python.org/pipermail/python-ideas/2011-January/009049.html > > that mentioned an odict literal of ['key': 'value', 'key2': 'value2'] > could be confused with slice notation. > > From a syntax point-of-view, that doesn't seem to be true (as > mentioned in some of the replies to that thread), but it seems like > you can abuse the similarity to make it a little easier to declare > ordereddicts: > > from collections import OrderedDict > class ODSlicer(object): > def __getitem__(self, key): > if type(key) is slice: > key = [key] > od = OrderedDict() > for k in key: > if type(k) is slice: > od[k.start] = k.stop > else: > od[k] = k > return od > od = ODSlicer() > > print(od[1:2]) > print(od["a":"b", "c":5]) > print(od['a':'b', 'c':'d', ..., 'a':10, 'e':'f']) > > > You could then replace: > > mydict = { > 'foo': 'bar', > 'baz': 'quux', > } > > with: > > mydict = od[ > 'foo': 'bar', > 'baz': 'quux', > ] > > if you need to convert a hardcoded dict into a hardcoded ordereddict. > Works fine in python2.7 and python3.4. > > > At this point, I'd like to note in my defence that this isn't called > the python-good-ideas list :) > > What's the actual objection to supporting ['foo': 'bar'] odict > literals? I saw Guido gave a -100, way back in the day, but no actual > explanation for why it was distasteful? > > https://mail.python.org/pipermail/python-ideas/2009-June/004924.html > > Cheers, > aj > > -- > Anthony Towns <aj at erisian.com.au> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/430b9bac/attachment.html> From abarnert at yahoo.com Wed Mar 19 19:08:54 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Mar 2014 11:08:54 -0700 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> <49CC8A0A-5735-4B75-BDD2-3ABE815DB58E@yahoo.com> <CAJS_LCU9H1VTDEYnq_zPxtzG=1cZHuyq_f86q2CVNmbSi013uw@mail.gmail.com> Message-ID: <6A8172AE-7BBF-4005-9D6F-74D5B9259A41@yahoo.com> On Mar 19, 2014, at 0:39, Anthony Towns <aj at erisian.com.au> wrote: > class InterestingFields(OrderedDict): > def __init__(self, extras): > super(InterestingFields, self).__init__() > > D_key = D("key") > D_name = D("name") > s = S() > t = T() > f = self > > f["project"] = D_key > f["issuetype"] = D_name > f["summary"] = s > # 18 more key/value pairs ... > > for k,v in extras: > f[k] = v > > where D,S,T are classes that provide functions to convert from the > Jira REST representation to the python version of the YAML > representation I want and vice-versa. My first question is: why are you retrieving all 21 fields into named variables, and then putting them all into the dict? Why not just: f["project"] = D("key") Also, why does this have to be an OrderedDict in the first place? This isn't a collection of arbitrary fields where you want to preserve the order you found them in, it's primarily a collection of a fixed set of fields, which may have some extra fields at the end. In other words, an object. If the only issue is that you want to dump all your fields to YAML, you just need a reduce-type method that dumps them in the right order. You could do that in a variety of different ways; using an OrderedDict for your instance __dict__ is the obvious one, but your could also use a namedtuple instead of a normal class, use __slots__ (and store the extras in a single OrderedDict attribute), etc. If the problem is how to take the extras in a way that's nicer than an iterable of pairs so you don't need all those parens and brackets when you construct it, you can remove either the brackets or the parens or both by just using *extras at the end and taking the pairs as args, or taking the keys and values as alternating args. I don't mean to nitpick here, but I suspect this is exactly the kind of thing people were expecting--use cases that do unpythonic things like blur the line between objects and dicts ala JavaScript. Of course when you're dealing in JSON, or to some extent YAML, it's hard to completely avoid that blurring, but blurring it even farther isn't the solution. From ericsnowcurrently at gmail.com Wed Mar 19 22:24:16 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 19 Mar 2014 15:24:16 -0600 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com> Message-ID: <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com> On Wed, Mar 19, 2014 at 2:00 AM, David Mertz <mertz at gnosis.cx> wrote: > It doesn't feel to me like the existing (non-literal) spelling is > particularly bad: > >>>> o1 = OrderedDict(foo=1, bar=2, baz=3) >>>> o2 = OrderedDict([('foo',1), ('bar',2), ('baz',3)]) >>>> o1 > OrderedDict([('bar', 2), ('foo', 1), ('baz', 3)]) >>>> o2 > OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)]) > > So sure, you can't use the kw=val style of initialization, FYI I'm working on a proposal to support preserving the call order of **kwargs. This would allow initializing an OrderedDict correctly using keyword args. I may have something to show for it by the end of the PyCon sprints. -eric From random832 at fastmail.us Wed Mar 19 23:01:20 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Wed, 19 Mar 2014 18:01:20 -0400 Subject: [Python-ideas] OrderedDict literals In-Reply-To: <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com> References: <CAJS_LCV6vWYXfT2sw1_oUu5owMhLFR1A17HND0v-zNnsYUJvCw@mail.gmail.com> <CAEbHw4Zgh+U8O-x3Wrt6k71-kxE=eh1+3Rcu8=ADhifYK6=Nuw@mail.gmail.com> <CALFfu7ChCNAbeXPJR4NHT5_NELi8rVagEEWXOe+kq_RXMSt54A@mail.gmail.com> Message-ID: <1395266480.19830.96554629.30A054FF@webmail.messagingengine.com> On Wed, Mar 19, 2014, at 17:24, Eric Snow wrote: > FYI I'm working on a proposal to support preserving the call order of > **kwargs. This would allow initializing an OrderedDict correctly > using keyword args. I may have something to show for it by the end of > the PyCon sprints. I know that it's possible to support this* in dict, but is kwargs required to be a dict? What route are you going: modify dict to allow this and require dict to support it; make a new class for kwargs and require this behavior of kwargs; modify dict on CPython to allow it but only require that kwargs support it "somehow" on non-CPython? *where "this" is e.g. "preservation of original insertion order for iterating when there have been no deletions" From ericsnowcurrently at gmail.com Thu Mar 20 00:32:27 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Wed, 19 Mar 2014 17:32:27 -0600 Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict literals) Message-ID: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> On Wed, Mar 19, 2014 at 4:01 PM, <random832 at fastmail.us> wrote: > On Wed, Mar 19, 2014, at 17:24, Eric Snow wrote: >> FYI I'm working on a proposal to support preserving the call order of >> **kwargs. This would allow initializing an OrderedDict correctly >> using keyword args. I may have something to show for it by the end of >> the PyCon sprints. > > I know that it's possible to support this* in dict, but is kwargs > required to be a dict? I'm not talking about the ** unpacking syntax, but about the function definition syntax for **kwargs: def spam(a, b, **kwargs): ... > What route are you going: modify dict to allow > this and require dict to support it; make a new class for kwargs and > require this behavior of kwargs; modify dict on CPython to allow it but > only require that kwargs support it "somehow" on non-CPython? Hopefully I'll have time to write a proto-PEP on this in the next couple weeks, but the gist is that I see 2 options: 1. Change kwargs to an OrderedDict for functions decorated a special decorator: @preserve_kwargs_order def spam(a, b, **kwargs): ... # kwargs will be an OrderedDict 2. Store the ordered keys in a list and bind that to a special local variable, but only for functions that have **kwargs: def spam(a, b, **kwargs): kwargs = OrderedDict((k, kwargs[k]) for k in __kwargs_order__) ... My gut feeling is that option 2 is the better choice. And before you ask, Guido has already vetoed making **kwargs always be an OrderedDict. There are a few other details to work out, but the above is the main thing. Also, an OrderedDict implemented in C will likely be a prerequisite, but luckily I've had a patch up since July(?) (#16991). It doesn't apply cleanly at present, but I expect that's only barely and I should be able to get a clean patch up when I have a chance. There wasn't a lot of interest in reviewing such a large patch so I'd back-burnered it until Python 3.4 got released (which just happened). Once I get a couple reviews I should be able to get that in (wink wink nudge nudge). -eric From rosuav at gmail.com Thu Mar 20 01:13:28 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Mar 2014 11:13:28 +1100 Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict literals) In-Reply-To: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> Message-ID: <CAPTjJmoKzMB+cef-1RD8FwBYVpAf6GQQBmBD-y=ayJocGLO7vw@mail.gmail.com> On Thu, Mar 20, 2014 at 10:32 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote: > 1. Change kwargs to an OrderedDict for functions decorated a special decorator: > > @preserve_kwargs_order > def spam(a, b, **kwargs): > ... # kwargs will be an OrderedDict > > 2. Store the ordered keys in a list and bind that to a special local > variable, but only for functions that have **kwargs: > > def spam(a, b, **kwargs): > kwargs = OrderedDict((k, kwargs[k]) for k in __kwargs_order__) > ... > > My gut feeling is that option 2 is the better choice. Strongly against any proposal that makes argument passing behave differently based on the target - it means the process of building up the arguments has to check some attribute on the function. Order of kwargs is a problem for any function that passes args through to another function. def wrap_spam(*args, **kwargs): try: return spam(*args, **kwargs) except EggsError: return False Does wrap_spam have to be aware that spam cares about keyword argument order? Should it preserve order "just in case"? Should the __kwargs_order__ list be automatically propagated? And what happens if the args aren't passed through perfectly, but some are added/removed? ChrisA From abarnert at yahoo.com Thu Mar 20 02:58:30 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Mar 2014 18:58:30 -0700 Subject: [Python-ideas] Preserving **kwargs order (was: Re: OrderedDict literals) In-Reply-To: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> Message-ID: <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> On Mar 19, 2014, at 16:32, Eric Snow <ericsnowcurrently at gmail.com> wrote: > Hopefully I'll have time to write a proto-PEP on this in the next > couple weeks, but the gist is that I see 2 options: Last time around, someone suggested that ***kwargs could get you the args as a list of pairs, or maybe an OrderedDict. While that would require a bit of complexity in the CALL_FUNCTION code, it seems like it would be simpler than your option 2. But it doesn't solve the main problem. Right now, you can forward any arguments perfectly by doing this: def wrapper(*args, **kwargs): return wrappee(*args, **kwargs) Your option 2 would require much more verbose code to forward perfectly. The triple-star idea makes it a lot nicer (assuming that ** passing respects the order, or a new *** is added there as well), but it would still break the thousands of wrapper functions out there written with **kwargs. From python at mrabarnett.plus.com Thu Mar 20 03:42:54 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 20 Mar 2014 02:42:54 +0000 Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> Message-ID: <532A55AE.5040509@mrabarnett.plus.com> On 2014-03-20 01:58, Andrew Barnert wrote:> On Mar 19, 2014, at 16:32, Eric Snow <ericsnowcurrently at gmail.com> wrote: > >> Hopefully I'll have time to write a proto-PEP on this in the next >> couple weeks, but the gist is that I see 2 options: > > Last time around, someone suggested that ***kwargs could get you the > args as a list of pairs, or maybe an OrderedDict. While that would > require a bit of complexity in the CALL_FUNCTION code, it seems like > it would be simpler than your option 2. > > But it doesn't solve the main problem. Right now, you can forward > any arguments perfectly by doing this: > > def wrapper(*args, **kwargs): > return wrappee(*args, **kwargs) > > Your option 2 would require much more verbose code to forward > perfectly. The triple-star idea makes it a lot nicer (assuming that > ** passing respects the order, or a new *** is added there as well), > but it would still break the thousands of wrapper functions out > there written with **kwargs. > Wouldn't it be a problem only if the dict were unpacked and then repacked? In the code above, it's merely passing the input kwargs on to 'wrappee'. From antony.lee at berkeley.edu Thu Mar 20 07:03:14 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Wed, 19 Mar 2014 23:03:14 -0700 Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <532A55AE.5040509@mrabarnett.plus.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> <532A55AE.5040509@mrabarnett.plus.com> Message-ID: <CAGRr6BFtZStpvqSZCsrwYo9hp82Si2wWfxsVSRWtwpmzKfHnMw@mail.gmail.com> Somehow it feels wrong to add a fairly complex new feature to the language (namely, ordered keyword arguments) to handle the very specific case of ordered dict constructors (or is there some other (clearly) different use intended?). If anything, I prefer the abuse of "slicing" syntax (so that one can write "od['a': 'b', 'c': 'd']" which is not much worse than "{'a': 'b', 'c': 'd'}"). Let's say we start with def wrapper(**kwargs): <some code that manipulates kwargs> return g(**kwargs) def g(**kwargs): pass wrapper(a=1, b=2) Nothing requires ordered keyword arguments, wrapper doesn't bother saving the argument order, everything is fine (but because there is some code that manipulates kwargs, an actual dict must be created, not just some sort of proxy)... now, just after wrapper is executed and before g gets called, globals()["g"] gets changed (by another thread...) to a function that does require ordered keyword arguments. What now? Antony 2014-03-19 19:42 GMT-07:00 MRAB <python at mrabarnett.plus.com>: > On 2014-03-20 01:58, Andrew Barnert wrote:> On Mar 19, 2014, at 16:32, > Eric Snow <ericsnowcurrently at gmail.com> wrote: > > > >> Hopefully I'll have time to write a proto-PEP on this in the next > >> couple weeks, but the gist is that I see 2 options: > > > > Last time around, someone suggested that ***kwargs could get you the > > args as a list of pairs, or maybe an OrderedDict. While that would > > require a bit of complexity in the CALL_FUNCTION code, it seems like > > it would be simpler than your option 2. > > > > But it doesn't solve the main problem. Right now, you can forward > > any arguments perfectly by doing this: > > > > def wrapper(*args, **kwargs): > > return wrappee(*args, **kwargs) > > > > Your option 2 would require much more verbose code to forward > > perfectly. The triple-star idea makes it a lot nicer (assuming that > > ** passing respects the order, or a new *** is added there as well), > > but it would still break the thousands of wrapper functions out > > there written with **kwargs. > > > > Wouldn't it be a problem only if the dict were unpacked and then > repacked? In the code above, it's merely passing the input kwargs on to > 'wrappee'. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140319/1b016632/attachment.html> From abarnert at yahoo.com Thu Mar 20 07:43:21 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 19 Mar 2014 23:43:21 -0700 (PDT) Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <532A55AE.5040509@mrabarnett.plus.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> <532A55AE.5040509@mrabarnett.plus.com> Message-ID: <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com> From: MRAB <python at mrabarnett.plus.com> Sent: Wednesday, March 19, 2014 7:42 PM > On 2014-03-20 01:58, Andrew Barnert wrote: >> On Mar 19, 2014, at 16:32, >> Eric Snow <ericsnowcurrently at gmail.com> wrote: >>> Hopefully I'll have time to write a proto-PEP on this in the next >>> couple weeks, but the gist is that I see 2 options: >> >> But it doesn't solve the main problem. Right now, you can forward >> any arguments perfectly by doing this: >> >> ? ? ? def wrapper(*args, **kwargs): >> ? ? ? ? ? return wrappee(*args, **kwargs) >> >> Your option 2 would require much more verbose code to forward >> perfectly. >> ? it would still break the thousands of wrapper functions out >> there written with **kwargs. > > Wouldn't it be a problem only if the dict were unpacked and then > repacked? In the code above, it's merely passing the input kwargs on to > 'wrappee'. No, a **kwargs parameter packs the keyword arguments into a dict, and a **kwargs argument unpacks a dict into the keyword arguments; that's exactly what they do. And dicts have arbitrary order. Besides, the whole point of option 2 is that kwargs?is still just a plain dict, and the order is passed via a magic hidden parameter __kwargs_order__. So, even if you _were_ just passing kwargs on without doing anything to it, that still wouldn't help.?As long as wrapper (and every other general-purpose wrapper every written) doesn't do anything with __kwargs_order__, the order is not going to get passed to wrapped. The obvious way to fix this is to make a **kwargs parameter pack the keyword arguments into an OrderedDict, but Guido has already rejected that, which is why Eric Snow had to come up with his two other options. But they don't solve the problem. From jared.grubb at gmail.com Thu Mar 20 08:41:40 2014 From: jared.grubb at gmail.com (Jared Grubb) Date: Thu, 20 Mar 2014 00:41:40 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? Message-ID: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions") Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it. I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special? I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :)) Jared QUICK SUMMARY OF ANDREW'S EMAIL: (Not a proposal for syntax or naming) m `cross` n m `dot` n a `Pair` b a `Tree` (b `Tree` c `Tree` d) `Tree` e And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too: m `mmul` n m `mdiv` n From ncoghlan at gmail.com Thu Mar 20 09:02:54 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 20 Mar 2014 18:02:54 +1000 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> Message-ID: <CADiSq7d8727d+rK7oVX0M-D2-BUd_+thAnBXUZr2G5F-VLw5hQ@mail.gmail.com> On 20 March 2014 17:41, Jared Grubb <jared.grubb at gmail.com> wrote: > A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions") > > Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it. > > I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special? Yes, basically - a large portion of the PEP is about pointing out that matrix multiplication really is that special. All past attempts at getting syntax for it *have* been generalised along that lines of this suggestion from Andrew, and have ended up failing on the grounds of "just use a method or function instead". Nathaniel's PEP is about looking at those past proposals and the experience of the numeric computing community and asking "Which operator have we *really* missed?". And they realised that it was only one: a way to spell matrix multiplication, since "*" has already been claimed for element-wise multiplication. The generalised proposals failed because they were still clumsy for the matrix multiplication use case, and there wasn't a compelling justification for the extra complexity that came with the generalisation. By contrast, Nathaniel's latest PEP looked at solving the *exact* problem the numeric community has: a clean and concise way to spell matrix multiplication. It hit the sweet spot of design proposals: a clean targeted solution that is as simple as it can possibly be, while still solving the problem that needs to be solved. It may have *looked* deceptively fast in terms of core devs saying "yes, that's a good idea", but the speed makes more sense if you start the timer from some of the earlier PEPs referenced from Nathaniel's one - the "matrix multiplication is clumsy compared to MATLAB" problem has been around for as long as people have been doing numeric computing in Python. Cheers, Nick. > > I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :)) > > Jared > > QUICK SUMMARY OF ANDREW'S EMAIL: > (Not a proposal for syntax or naming) > > m `cross` n > m `dot` n > > a `Pair` b > a `Tree` (b `Tree` c `Tree` d) `Tree` e > > And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too: > > m `mmul` n > m `mdiv` n > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From rosuav at gmail.com Thu Mar 20 09:58:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 20 Mar 2014 19:58:47 +1100 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> Message-ID: <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com> On Thu, Mar 20, 2014 at 6:41 PM, Jared Grubb <jared.grubb at gmail.com> wrote: > But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :)) > > Jared > > QUICK SUMMARY OF ANDREW'S EMAIL: > (Not a proposal for syntax or naming) > > m `cross` n > m `dot` n Sometimes a proposal fails because no good syntax can be found for it. The strongest objection to my PEP 463 on exception expressions is "I don't like the syntax", and to date nobody's come up with a syntax that everyone likes. (Four proposals in the PEP itself, all somewhat flawed.) In the above case, the proposal somewhat falls down for the same reason. Backticks have been, by BDFL fiat, banned from syntax; either that decision has to be reversed, or some other syntax has to be figured out. People will love or hate the proposal based on the tentative syntax, and the "grit on Tim's monitor" check, and such. ChrisA From steve at pearwood.info Thu Mar 20 11:24:44 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 20 Mar 2014 21:24:44 +1100 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> Message-ID: <20140320102443.GS16526@ando> On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote: > A couple months back, Andrew Barnet brought up the idea of adding a > way to apply functions in an infix style. It really didnt gain much > traction. (Subject line "Infix Functions") > > Nathaniel Smith has just proposed a PEP to add an infix matrix > multiply operator, and everyone seems to love it. > > I honestly am surprised at the difference in reaction. Why are we so > quick to add a single-purpose punctuation mark, but reject a named > infix operator that works for many general cases? Is the > matrix-multiply use case so special? Yes. Every day, there are probably thousands of numpy users (to say nothing of other matrix-multiplication library users) who do matrix multiplication. I can't think of the last time I've wanted to invent my own infix operator, and if I ever did, I'd probably be the only person using it. I don't think I'm too unusual. Custom infix operators for arbitrary functions are extremely niche, because apart from matrix multiplication, nearly all the common, useful ones are already available. We have + - * / & ^ | etc., that covers the common cases. The ones remaining are (apart from matrix multiplication) uncommon. The beauty of infix operators is that they give an extremely compact representation, which is valuable for mathematics but not so much for general purpose computing. Consequently, it would be useful to write something like: matrix @ vector versus matrix.mult(vector) because the symbol for the operator is so compact. Having to write it as this instead: matrix`mult`vector saves you one character over the method syntax, or even costs you one character if you write it like this: matrix `mult` vector which is not enough of a saving over the method syntax to make it worthwhile. Having operators like + and * is a good thing, but if we didn't have them, and had to write this instead: x`plus`y a`times`b there'd be no advantage to using pseudo-operators as shown. You might as well use regular method calls: x.plus(y) a.times(b) (which you'll note is already pseudo-infix). So, in my opinion, until such time as Python supports Unicode symbols (around Python 5000 perhaps?) as operators, there is no point in trying to fake them with spelled-out names. If you're going to use a spelled-out name, use a method, or function. -- Steven From robert.kern at gmail.com Thu Mar 20 11:32:33 2014 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 20 Mar 2014 10:32:33 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> Message-ID: <lgeg3i$2o9$1@ger.gmane.org> On 2014-03-14 17:53, Guido van Rossum wrote: > - Did you consider a duck-typing (is that the word?) attribute? > E.g. a*b is elementwise multiplication; a.M*b must be used for > matrix multiplication. (Your use of .T as "transpose" made me think > of this.) Of course the question is, can you get those packages > that currently use * for matrix multiply to comply? (I don't consider > this a serious counter-proposal. But you list a bunch of rejected > alternatives; this could be in that list. Apparently this *was* considered in PEP 225 under "Alternatives to adding new operators", numbers 3 and 4. http://legacy.python.org/dev/peps/pep-0225/ Of course, number 6 is "Introducing a single operator, such as @, for matrix multiplication." :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Thu Mar 20 13:58:27 2014 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 20 Mar 2014 12:58:27 +0000 Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> <532A55AE.5040509@mrabarnett.plus.com> <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com> Message-ID: <532AE5F3.9040508@mrabarnett.plus.com> On 2014-03-20 06:43, Andrew Barnert wrote: > From: MRAB <python at mrabarnett.plus.com> > > Sent: Wednesday, March 19, 2014 7:42 PM > > > > On 2014-03-20 01:58, Andrew Barnert wrote: > >> On Mar 19, 2014, at 16:32, > >> Eric Snow <ericsnowcurrently at gmail.com> wrote: > >>> Hopefully I'll have time to write a proto-PEP on this in the > >>> next couple weeks, but the gist is that I see 2 options: > >> > >> But it doesn't solve the main problem. Right now, you can forward > >> any arguments perfectly by doing this: > >> > >> def wrapper(*args, **kwargs): > >> return wrappee(*args, **kwargs) > >> > >> Your option 2 would require much more verbose code to forward > >> perfectly. > >> ? it would still break the thousands of wrapper functions out > > >> there written with **kwargs. > > > > Wouldn't it be a problem only if the dict were unpacked and then > > repacked? In the code above, it's merely passing the input kwargs > > on to 'wrappee'. > > No, a **kwargs parameter packs the keyword arguments into a dict, and > a **kwargs argument unpacks a dict into the keyword arguments; > that's exactly what they do. And dicts have arbitrary order. > When I decompile, what I get is: >>> dis(compile('''wrappee(**kwargs)''', '<string>', 'exec')) 1 0 LOAD_NAME 0 (wrappee) 3 LOAD_NAME 1 (kwargs) 6 CALL_FUNCTION_KW 0 (0 positional, 0 keyword pair) 9 POP_TOP 10 LOAD_CONST 0 (None) 13 RETURN_VALUE It looks like the dict 'kwargs' is being passed straight to 'wrappee'. > Besides, the whole point of option 2 is that kwargs is still just a > plain dict, and the order is passed via a magic hidden parameter > __kwargs_order__. So, even if you _were_ just passing kwargs on > without doing anything to it, that still wouldn't help. As long as > wrapper (and every other general-purpose wrapper every written) > doesn't do anything with __kwargs_order__, the order is not going to > get passed to wrapped. > > The obvious way to fix this is to make a **kwargs parameter pack the > keyword arguments into an OrderedDict, but Guido has already rejected > that, which is why Eric Snow had to come up with his two other > options. But they don't solve the problem. > From abarnert at yahoo.com Thu Mar 20 17:06:09 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Mar 2014 09:06:09 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <20140320102443.GS16526@ando> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> Message-ID: <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> On Mar 20, 2014, at 3:24, Steven D'Aprano <steve at pearwood.info> wrote: > On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote: > > The beauty of infix operators is that they give an extremely compact > representation, which is valuable for mathematics but not so much for > general purpose computing. Consequently, it would be useful to write > something like: > > matrix @ vector > > versus > > matrix.mult(vector) > > because the symbol for the operator is so compact. Having to write it > as this instead: > > matrix`mult`vector > > saves you one character over the method syntax, It's not about saving one character, it's about saving the (possibly overly-nested) parentheses. The PEP explains that, as does the earlier thread, and all of the previous threads and PEPs on related ideas, so I won't go into it again. However, the other answers are right; matrix multiplication _is_ special. As an alternative to the 2000 PEP that proposed a suite of new operators by allowing ~ as a prefix to 11 existing operators )or to the other proposal to allow any string of operator characters to be defined as an operator), I think general infix wins. However, the NumPy community is no longer suggesting either of those; after 14 years of living with the limitations of the current system, they've decided that only one operator is really a problem. And against that proposal, general infix loses. One is a good number of special cases for the same reason 11 is a bad number of special cases. Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway. From abarnert at yahoo.com Thu Mar 20 17:28:34 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Mar 2014 09:28:34 -0700 Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <532AE5F3.9040508@mrabarnett.plus.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> <532A55AE.5040509@mrabarnett.plus.com> <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com> <532AE5F3.9040508@mrabarnett.plus.com> Message-ID: <87C2AE45-4FD2-418D-9288-B5BD51D06F27@yahoo.com> On Mar 20, 2014, at 5:58, MRAB <python at mrabarnett.plus.com> wrote: > > No, a **kwargs parameter packs the keyword arguments into a dict, and > > a **kwargs argument unpacks a dict into the keyword arguments; > > that's exactly what they do. And dicts have arbitrary order. > > > When I decompile, what I get is: > > >>> dis(compile('''wrappee(**kwargs)''', '<string>', 'exec')) > 1 0 LOAD_NAME 0 (wrappee) > 3 LOAD_NAME 1 (kwargs) > 6 CALL_FUNCTION_KW 0 (0 positional, 0 keyword pair) > 9 POP_TOP > 10 LOAD_CONST 0 (None) > 13 RETURN_VALUE This is basically an optimization, where the unpacking happens inside CALL_FUNCTION_KW instead of in a bunch of separate bytecodes that pushed them onto the stack to be popped off by CALL_FUNCTION. The end result is identical. Inside CALL_FUNCTION_KW, instead of starting with an empty dict and then popping keywords into it, it starts with a copy of kwargs and then pops keywords into it. If that weren't true, you wouldn't be able to do this: def foo(a, **kwargs): pass foo(**{'a': 1, 'b': 2}) Or: def foo(**kwargs): pass foo(b=2, **{'a': 1}) (If you replace a or b with self, this should look more familiar.) And, even if you don't have any missing or extra arguments, you are still going to get a new dict inside the callee. Otherwise this would do the wrong thing: def foo(**kwargs): kwargs['a'] = 3 d = {'a': 1, 'b': 2} foo(**d) And, even if you changed Python so that, in the case where there are no extra or missing keywords and the dict isn't mutated inside foo, you just passed it as-is and skipped the copy, that _still_ wouldn't solve the forwarding problem, because it's still just a dict in arbitrary order. And adding a separate hidden parameter to pass the order along still wouldn't help unless you also forwarded _that_ parameter somehow. And as far as I can see, there is no way to solve this problem. The only way to add keyword order information without breaking all existing forwarding functions is to make the **kwargs parameter always preserve order (and make the **kwargs calling syntax preserve order if present)--that is, to make it always an OrderedDict, which Guido has already rejected. From ron3200 at gmail.com Thu Mar 20 21:14:35 2014 From: ron3200 at gmail.com (Ron Adam) Date: Thu, 20 Mar 2014 16:14:35 -0400 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> Message-ID: <lgfi6t$45l$1@ger.gmane.org> On 03/20/2014 12:06 PM, Andrew Barnert wrote: > Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway. I like that idea also, but ... s = 'a' 'b' 'c' Is 'b' a string op or a string? To make it work, it needs an introducer. Which kind of defeats some of the advantage. Or it needs to be defined with a keyword. This actually makes sense as it's changes the effect of a name. (like local, non-local, and global.) def foo(...): symbol b s = 'a' b 'c' One of the objects would need a method to get the symbol, and the other argument. Cheers, Ron From bwmaister at gmail.com Thu Mar 20 22:04:25 2014 From: bwmaister at gmail.com (Brandon W Maister) Date: Thu, 20 Mar 2014 17:04:25 -0400 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <lgfi6t$45l$1@ger.gmane.org> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> Message-ID: <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> On Thu, Mar 20, 2014 at 4:14 PM, Ron Adam <ron3200 at gmail.com> wrote: > def foo(...): > symbol b > > s = 'a' b 'c' > Actually makes me think of decorators, which means that you wouldn't even need new tokens or keywords: from functools import make_operator bop = make_operator(b) s = 'a' bop 'b' class Ops: @make_operator def cat(left, right): return left + right t = 'a' Ops.cat 'b' It reduces the grit, but also reduces certain kinds of obviousness. bwm -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140320/800cf2e9/attachment-0001.html> From greg.ewing at canterbury.ac.nz Thu Mar 20 22:34:59 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Mar 2014 10:34:59 +1300 Subject: [Python-ideas] Preserving **kwargs order In-Reply-To: <532AE5F3.9040508@mrabarnett.plus.com> References: <CALFfu7A=gEZ0GupZcJDM+biVTr1fkiAWexb-Aq1wgROqiKN3Xg@mail.gmail.com> <E804B13F-9D15-46DE-9B17-5C43081FDF29@yahoo.com> <532A55AE.5040509@mrabarnett.plus.com> <1395297801.17430.YahooMailNeo@web181006.mail.ne1.yahoo.com> <532AE5F3.9040508@mrabarnett.plus.com> Message-ID: <532B5F03.90409@canterbury.ac.nz> MRAB wrote: > It looks like the dict 'kwargs' is being passed straight to 'wrappee'. But the interpreter doesn't just pass the object given as a ** argument straight to the function's ** parameter; it gets repacked into a new dict: >>> def f(**kw): ... print id(kw) ... >>> d = {'a':1, 'b':2} >>> print id(d) 2816160 >>> f(**d) 2817744 So even if you pass an OrderedDict or some other custom object as a ** argument, it's a plain dict by the time it comes out the other end. -- Greg From steve at pearwood.info Fri Mar 21 01:15:39 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Mar 2014 11:15:39 +1100 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> Message-ID: <20140321001539.GU16526@ando> On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote: > Actually makes me think of decorators, which means that you wouldn't even > need new tokens or keywords: But you would require new syntax. Your suggested syntax s = 'a' bop 'b' is currently a syntax error. Keep in mind that any changes to syntax would allow bop to be any arbitrary object. Python cannot tell at compile-time whether bop is an "operator" or a list or a string or a float, only at run-time. > from functools import make_operator > bop = make_operator(b) Do you have any suggestion as to how make_operator would work? What should it do? What sort of object is bop? -- Steven From bwmaister at gmail.com Fri Mar 21 02:28:03 2014 From: bwmaister at gmail.com (Brandon W Maister) Date: Thu, 20 Mar 2014 21:28:03 -0400 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <20140321001539.GU16526@ando> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> Message-ID: <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> On Mar 20, 2014 8:16 PM, "Steven D'Aprano" <steve at pearwood.info> wrote: > > On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote: > > > Actually makes me think of decorators, which means that you wouldn't even > > need new tokens or keywords: > > But you would require new syntax. Your suggested syntax > > s = 'a' bop 'b' > > is currently a syntax error. 100% agree, I was careful with my language, if not my thinking :-) > Keep in mind that any changes to syntax would allow bop to be any > arbitrary object. Python cannot tell at compile-time whether bop is an > "operator" or a list or a string or a float, only at run-time. I admit that when writing I didn't think about the fact that I was suggesting adding dynamic names to what is currently a static table. > > from functools import make_operator > > bop = make_operator(b) > > Do you have any suggestion as to how make_operator would work? What > should it do? Originally I was thinking something along the lines of "just add it to the 'operators' dict" but obviously that won't work since there's no such thing. Before I defend the idea any further I need to point out that I don't think it's a *good* idea. I kind of like the idea of arbitrary infix functions, so I threw out an idea that I haven't seen before. We could, hypothetically, create an "operators" dict (or dicts) alongside locals and globals. Then, if an unexpected token shows up in operator position (which is probably well-defined?) before a SyntaxError is raised the token is looked up in the operators dict. I think that the biggest problem I have with that solution is that philosophically it seems like it would take Python a bit far into the lisp-2 camp, which I would not like. > What sort of object is bop? A regular function that takes one or two args. Honestly the nice thing about the "look it up somewhere if it causes the right kind of SyntaxError" solution is that it would probably also allow arbitrary unary operators. Although why anyone would want *that* is beyond me, so maybe it's not nice :-) bwm -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140320/48bd111a/attachment.html> From stephen at xemacs.org Fri Mar 21 02:39:00 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 21 Mar 2014 10:39:00 +0900 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <CAPTjJmqRykhh_G=UBF90Fu6z-cCQgTOLv3io6kkd3E5njw7zig@mail.gmail.com> Message-ID: <87mwgkib4b.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Angelico writes: > figured out. People will love or hate the proposal based on the > tentative syntax, and the "grit on Tim's monitor" check, and such. Why the lets-use-some-punctuation-for-syntax crowd doesn't just buy Tim a lifetime subscription to monitor-protecting clearsheets so he can remove the grit every day, I just don't know! Win-win AFAICS.... From rosuav at gmail.com Fri Mar 21 03:49:12 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Mar 2014 13:49:12 +1100 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> Message-ID: <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com> wrote: > We could, hypothetically, create an "operators" dict (or dicts) alongside > locals and globals. Then, if an unexpected token shows up in operator > position (which is probably well-defined?) before a SyntaxError is raised > the token is looked up in the operators dict. The SyntaxError is raised at compilation time; locals and globals are at run time. The only way that would work is if you do something like this: operators["b"] = lambda left, right: whatever import module_using_b ChrisA From abarnert at yahoo.com Fri Mar 21 04:40:46 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Thu, 20 Mar 2014 20:40:46 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> Message-ID: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> On Mar 20, 2014, at 19:49, Chris Angelico <rosuav at gmail.com> wrote: > On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com> wrote: >> We could, hypothetically, create an "operators" dict (or dicts) alongside >> locals and globals. Then, if an unexpected token shows up in operator >> position (which is probably well-defined?) before a SyntaxError is raised >> the token is looked up in the operators dict. > > The SyntaxError is raised at compilation time; locals and globals are > at run time. The only way that would work is if you do something like > this: > > operators["b"] = lambda left, right: whatever > import module_using_b Or, alternatively, instead of a SyntaxError this would compile into a call to b, which would then give a NameError at runtime. And I suppose you could argue that's just taking dynamic typing one step further. But I seriously doubt anyone actually wants typos like "esle" to compile successfully... Also, imagine what it would look like to chain these: spam spam eggs baked beans spam spam Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have? Obviously that example is over the top, but try it with a tree constructor or the other examples from this thread and it's still painful. The Haskell-style backticks solve all of these problems: the coder can't accidentally write an infix operator when he meant something else, the parser can distinguish infix operators from typos at compile time, and the reader can tell (unless he's stolen Tim's monitor grit) which things are operators and which are operands. (Of course you could deliberately confuse people with weird spacing, but Python doesn't have to make it impossible to write intentionally obfuscated code, it just has to avoid making it easy to write accidentally obfuscated code.) From ethan at stoneleaf.us Fri Mar 21 06:01:11 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 20 Mar 2014 22:01:11 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> Message-ID: <532BC797.1060106@stoneleaf.us> On 03/20/2014 08:40 PM, Andrew Barnert wrote: > > Also, imagine what it would look like to chain these: > > spam spam eggs baked beans spam spam > > Maybe the parser can figure out that the spam, baked, and spam are > operators and the spam, eggs, beans, and spam are operands, but what > hope does any human have? Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around. -- ~Ethan~ From rosuav at gmail.com Fri Mar 21 06:43:48 2014 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 21 Mar 2014 16:43:48 +1100 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <532BC797.1060106@stoneleaf.us> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <532BC797.1060106@stoneleaf.us> Message-ID: <CAPTjJmpGZqZkmdsNX-AE7+O96rZ1wp6hCHbD=gisKJNFivWZFg@mail.gmail.com> On Fri, Mar 21, 2014 at 4:01 PM, Ethan Furman <ethan at stoneleaf.us> wrote: > On 03/20/2014 08:40 PM, Andrew Barnert wrote: >> >> >> Also, imagine what it would look like to chain these: >> >> spam spam eggs baked beans spam spam >> >> Maybe the parser can figure out that the spam, baked, and spam are >> operators and the spam, eggs, beans, and spam are operands, but what >> hope does any human have? > > > Why would spam be able to be both a variable and an operator? Hmm, perhaps > because of the first-class citizen thing -- so the leading grit says "make > it work" and without it is just passing it around. If the creation of an operator is done by putting something into the current namespace, then it has to be available as some sort of object. On the other hand, if it's something done as a declaration, or a pre-import keyword mapping, then it should be a keyword (like other word operators, "if" and "else" and such), and unavailable for assignment. In theory, this could be done as: from __aether__ import spam as operator where "as operator", instead of specifying the name to bind to, specifies that it be something other than a name. (Preferably, it should use a keyword, rather than "operator", which is a valid name.) This would go immediately under future statements, and would affect the compilation of that module: any instance of the word "spam" is now a keyword operator, so the above would be invalid. I'm still not convinced it would be a good thing, but this at least gives a per-module way to create operators. ChrisA From Andy.Henshaw at gtri.gatech.edu Fri Mar 21 15:19:42 2014 From: Andy.Henshaw at gtri.gatech.edu (Henshaw, Andy) Date: Fri, 21 Mar 2014 14:19:42 +0000 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <532BC797.1060106@stoneleaf.us> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <532BC797.1060106@stoneleaf.us> Message-ID: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org> From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman >On 03/20/2014 08:40 PM, Andrew Barnert wrote: >> >> Also, imagine what it would look like to chain these: >> >> spam spam eggs baked beans spam spam >> >> Maybe the parser can figure out that the spam, baked, and spam are >> operators and the spam, eggs, beans, and spam are operands, but what >> hope does any human have? > > Why would spam be able to be both a variable and an operator? > Hmm, perhaps because of the first-class citizen thing -- so the > leading grit says "make it work" and without it is just passing it around. Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo From cfkaran2 at gmail.com Fri Mar 21 17:24:28 2014 From: cfkaran2 at gmail.com (Cem Karan) Date: Fri, 21 Mar 2014 12:24:28 -0400 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <532BC797.1060106@stoneleaf.us> <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org> Message-ID: <33AF08DB-28BF-4290-89D4-5FFA485AF8C2@gmail.com> On Mar 21, 2014, at 10:19 AM, "Henshaw, Andy" <Andy.Henshaw at gtri.gatech.edu> wrote: > From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman > >> On 03/20/2014 08:40 PM, Andrew Barnert wrote: >>> >>> Also, imagine what it would look like to chain these: >>> >>> spam spam eggs baked beans spam spam >>> >>> Maybe the parser can figure out that the spam, baked, and spam are >>> operators and the spam, eggs, beans, and spam are operands, but what >>> hope does any human have? >> >> Why would spam be able to be both a variable and an operator? >> Hmm, perhaps because of the first-class citizen thing -- so the >> leading grit says "make it work" and without it is just passing it around. > > Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo For those whose first language is not USA English: http://en.wikipedia.org/wiki/Buffalo_Buffalo_Buffalo_Buffalo_Buffalo_Buffalo Thanks, Cem Karan From ethan at stoneleaf.us Fri Mar 21 17:47:21 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 21 Mar 2014 09:47:21 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <532BC797.1060106@stoneleaf.us> <6499c5746acc45cfbb9cfcefa11de5e0@apatlisdmail01.core.gtri.org> Message-ID: <532C6D19.5050803@stoneleaf.us> On 03/21/2014 07:19 AM, Henshaw, Andy wrote: > From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu at python.org] On Behalf Of Ethan Furman > >> On 03/20/2014 08:40 PM, Andrew Barnert wrote: >>> >>> Also, imagine what it would look like to chain these: >>> >>> spam spam eggs baked beans spam spam >>> >>> Maybe the parser can figure out that the spam, baked, and spam are >>> operators and the spam, eggs, beans, and spam are operands, but what >>> hope does any human have? >> >> Why would spam be able to be both a variable and an operator? >> Hmm, perhaps because of the first-class citizen thing -- so the >> leading grit says "make it work" and without it is just passing it around. > > Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo Python, however, is not English, and would only recognize two different objects from the above, so that would be a SyntaxError. -- ~Ethan~ From daoust.mj at gmail.com Fri Mar 21 21:06:54 2014 From: daoust.mj at gmail.com (Mark Daoust) Date: Fri, 21 Mar 2014 16:06:54 -0400 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> Message-ID: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com> >>> spam `spam` eggs `baked` beans `spam` spam I'm not sure if the backticks help. When you read that, I think you're reading the spacing not the ticks. You can already do something very similar by using built-in operators to "quote" the custom "operators", if you set the appropriate operator overloads. >>> spam *spam* eggs <baked> beans /spam/ spam "<spam>" is potentially the most readable, for it's parenthesis like shape , but you can't overload the "and" it expands into. Mark Daoust On Thu, Mar 20, 2014 at 11:40 PM, Andrew Barnert <abarnert at yahoo.com> wrote: > On Mar 20, 2014, at 19:49, Chris Angelico <rosuav at gmail.com> wrote: > > > On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister at gmail.com> > wrote: > >> We could, hypothetically, create an "operators" dict (or dicts) > alongside > >> locals and globals. Then, if an unexpected token shows up in operator > >> position (which is probably well-defined?) before a SyntaxError is > raised > >> the token is looked up in the operators dict. > > > > The SyntaxError is raised at compilation time; locals and globals are > > at run time. The only way that would work is if you do something like > > this: > > > > operators["b"] = lambda left, right: whatever > > import module_using_b > > Or, alternatively, instead of a SyntaxError this would compile into a call > to b, which would then give a NameError at runtime. And I suppose you could > argue that's just taking dynamic typing one step further. But I seriously > doubt anyone actually wants typos like "esle" to compile successfully... > > Also, imagine what it would look like to chain these: > > spam spam eggs baked beans spam spam > > Maybe the parser can figure out that the spam, baked, and spam are > operators and the spam, eggs, beans, and spam are operands, but what hope > does any human have? Obviously that example is over the top, but try it > with a tree constructor or the other examples from this thread and it's > still painful. > > The Haskell-style backticks solve all of these problems: the coder can't > accidentally write an infix operator when he meant something else, the > parser can distinguish infix operators from typos at compile time, and the > reader can tell (unless he's stolen Tim's monitor grit) which things are > operators and which are operands. (Of course you could deliberately confuse > people with weird spacing, but Python doesn't have to make it impossible to > write intentionally obfuscated code, it just has to avoid making it easy to > write accidentally obfuscated code.) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140321/c7353cbc/attachment.html> From skip at pobox.com Fri Mar 21 21:14:21 2014 From: skip at pobox.com (Skip Montanaro) Date: Fri, 21 Mar 2014 15:14:21 -0500 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com> Message-ID: <CANc-5UzOXGG5SZGb=hxYXHADbmREvm0kZMe0We=zuocvLB2rHA@mail.gmail.com> On Fri, Mar 21, 2014 at 3:06 PM, Mark Daoust <daoust.mj at gmail.com> wrote: >>>> spam `spam` eggs `baked` beans `spam` spam > > I'm not sure if the backticks help. Backticks never help. They hide amongst the beefier characters. That's one reason why $(...) is preferred by most Bash users and repr() by most Python users. Let them die the quiet death they deserve. Skip From abarnert at yahoo.com Fri Mar 21 23:09:06 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Fri, 21 Mar 2014 15:09:06 -0700 Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? In-Reply-To: <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com> References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <CAFg8gYz_=OwYgW1JAtvVcLLpMT34dPxFJ4RrSmskMw0PgukbpQ@mail.gmail.com> Message-ID: <6AC16A81-BCF8-4690-954C-4A14DA704993@yahoo.com> On Mar 21, 2014, at 13:06, Mark Daoust <daoust.mj at gmail.com> wrote: > >>> spam `spam` eggs `baked` beans `spam` spam > > I'm not sure if the backticks help. > > When you read that, I think you're reading the spacing not the ticks. No, you're reading a combination of the two. Compare it to the same without backticks: >>> spam `spam` eggs `baked` brand `spam` spam >>> spam spam eggs baked beans spam spam >>> spam spam eggs baked beans spam spam There's no way to differentiate the operators from the operands with _just_ spacing, but with spacing plus anything else, there is. And lest you think this is just an artifact of passing spam to spam, here it is with more realistic names; >>> mata mmul matb mmul matb mmul vec >>> mata `mmul` matb `mmul` matb `mmul` vec No matter how much you like or dislike the latter, the former is clearly a lot worse. > You can already do something very similar by using built-in operators to "quote" the custom "operators", if you set the appropriate operator overloads. Yes, I mentioned the |op| hack the last time I brought this up, and two other people suggested it after that. But if you want to go over it again: no symbol has a precedence right below the bitwise ops, comparison operators don't work because of chaining, there are types that will handle __op__ so your fake operator doesn't get a chance to do its __rop__, there's no way to prevent "2 |op" from being a valid but confusing expression (and there's a temptation to use it as a half-assed form of operator sectioning), it's hard for syntax highlighters and linters and the like to understand |op| is an operator, there's no way to restrict arbitrary expressions as operators, you have to declare all infixable operators with a decorator (as opposed to having the language choose whether all functions are infixable, or we use a decorator, or something else), any symbol you use already has another meaning that will confuse people, it's a bit less efficient, and, finally, it's not a consistent language feature (having |op| be an operator in one module, *op* in another, etc. makes the language harder to learn). From vernondcole at gmail.com Sun Mar 23 10:31:03 2014 From: vernondcole at gmail.com (Vernon D. Cole) Date: Sun, 23 Mar 2014 03:31:03 -0600 Subject: [Python-ideas] alternative-implementation friendly changes to Python Launcher for Windows Message-ID: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com> A new issue was recently raised on the IronPython group addressing the idea of including a copy of the *Python Launcher for Windows* in the IronPython binary distribution. http://www.python.org/dev/peps/pep-0397/ I would like to get feedback from this group, also, before opening a work item for the suggestion. Please add any comments or suggestions you may have. Here is a partial clip from our discussion... vernondcole <http://www.codeplex.com/site/users/view/vernondcole> wrote Fri > at 2:32 AM > I made a clone of the git version of PLW to look this idea over. It is a > smallish C++ program and should not be too hard to patch to make > alternate-implementation friendly. The problem today is that it is CPython > specific and searches the Windows registry to determine which versions of > Python are present. Alternate implementations are supported only using > entries in the py.ini file and setting the #! line in your source code > approprietly. (Which works great. I even have mine launching perl scripts, > just to proove that it can.) > > I propose adding two features to PLW. > > 1) extend the first command-line switch to also pick items from the py.ini > file. I would like to type: > py -ipy myprogram.py > and have the command associated with "ipy" in my py.ini file executed. > > 2) PLW should have (in py.ini) a [default] section identifying which > command in the [commands] section should be run in the absence of a > command-line switch. > > Alternate implementations should then install (or modify) py.ini to > support their interpreter. > > The extended PLW version ought to be backported to the CPython > distribution, so that any implementation of Python will work. I don't think > that the CPython group would object. > > Note 1: I have not compared to make sure the stand-alone PLW is identical > with the Python 3.3 version. > Note 2: We need to make sure that PLW will work correctly even if no > CPython versions are present. > Note 3: I fear that I have just volunteered to write the patch. Is someone > else out there willing? It's been a very long time since I wrote commercial > quality C code. > > jdhardy <http://www.codeplex.com/site/users/view/jdhardy> wrote Fri at > 5:33 AM > Sounds good. I remember from the original PLW discussion they want it to > work for other implementations but weren't sure what would be needed. It > sounds like you're making that list. :) > > It's probably worth opening bugs on the Python tracker describing what > needs to change, since it's not IronPython specific. If you don't feel > comfortable writing the code perhaps someone over there can, or at least > review the code for you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/d8411396/attachment.html> From sturla.molden at gmail.com Sun Mar 23 11:40:10 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Sun, 23 Mar 2014 11:40:10 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140315010944.6b8244e1@fsol> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> Message-ID: <lgmdm0$2ng$1@ger.gmane.org> On 15/03/14 01:09, Antoine Pitrou wrote: > Really? That should be up to the third-party library implementing the @ > operator for its types, not to the language itself: Python _suggests_ > an use case for @, it doesn't mandate it (especially as there's no > appropriate data type in the stdlib). array.array is an appropriate type for supporting @ for matrix multiplication. Sturla From jeanpierreda at gmail.com Sun Mar 23 11:51:24 2014 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 23 Mar 2014 03:51:24 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lgmdm0$2ng$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> Message-ID: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> On Sun, Mar 23, 2014 at 3:40 AM, Sturla Molden <sturla.molden at gmail.com> wrote: > On 15/03/14 01:09, Antoine Pitrou wrote: > >> Really? That should be up to the third-party library implementing the @ >> operator for its types, not to the language itself: Python _suggests_ >> an use case for @, it doesn't mandate it (especially as there's no >> appropriate data type in the stdlib). > > > array.array is an appropriate type for supporting @ for matrix > multiplication. I was thinking the function type, as a composition operator. Matrix multiplication corresponds exactly with composition of linear functions, so it makes some sort of sense. And it has some kind of neat similarity to decorators, which generalize function composition. If you rename "matmul" to "compose" and "matpow" to something like "iterate" (except less confusing), then you've generalized the operators a little and given them more potential use cases. -- Devin From p.f.moore at gmail.com Sun Mar 23 15:36:30 2014 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 23 Mar 2014 14:36:30 +0000 Subject: [Python-ideas] alternative-implementation friendly changes to Python Launcher for Windows In-Reply-To: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com> References: <CAH-ZgAfKiX-id1CUAt_BzxrJXnCYcZDmbki2EBnqAvAN3ht5bQ@mail.gmail.com> Message-ID: <CACac1F-WRYu+xcOLt5oVGXCS+G5uiQVKP0+PFHFz_9b4vuCZxQ@mail.gmail.com> On 23 March 2014 09:31, Vernon D. Cole <vernondcole at gmail.com> wrote: > A new issue was recently raised on the IronPython group addressing the idea > of including a copy of the Python Launcher for Windows in the IronPython > binary distribution. http://www.python.org/dev/peps/pep-0397/ [...] >> I propose adding two features to PLW. >> >> 1) extend the first command-line switch to also pick items from the py.ini >> file. I would like to type: >> py -ipy myprogram.py >> and have the command associated with "ipy" in my py.ini file executed. >> >> 2) PLW should have (in py.ini) a [default] section identifying which >> command in the [commands] section should be run in the absence of a >> command-line switch. >> >> Alternate implementations should then install (or modify) py.ini to >> support their interpreter. I like the idea of extending the version switch. I don't have a problem with a default command (although I probably wouldn't use it). I don't particularly like the idea of implementations modifying py.ini - that's a user configuration file and should not be modified by the system. Just document what needs to be added by the user for a particular implementation. The standalone version of the launcher is maintained by Vinay Sajip at https://bitbucket.org/pypa/pylauncher. I'd suggest adding issues to the tracker there. I don't know how (or if) that version gets folded back into the one distributed with Python, that's something that maybe Vinay can comment on. Paul From mistersheik at gmail.com Sun Mar 23 23:20:41 2014 From: mistersheik at gmail.com (Neil Girdhar) Date: Sun, 23 Mar 2014 15:20:41 -0700 (PDT) Subject: [Python-ideas] Make __reduce__ to correspond to __getnewargs_ex__ Message-ID: <fe676ada-d7bd-451c-b518-71d4cfb6411f@googlegroups.com> Currently __reduce__<http://docs.python.org/3.4/library/pickle.html#object.__reduce__>returns up to five things: (1) self.__new__ (or a substitute) (2) the result of __getnewargs__<http://docs.python.org/3.4/library/pickle.html#object.__getnewargs__>, which returns a tuple of positional arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__> , (3) the result of __getstate__<http://docs.python.org/3.4/library/pickle.html#object.__getstate__>, which returns an object to be passed to __setstate__<http://docs.python.org/3.4/library/pickle.html#object.__setstate__> (4) an iterator of values for appending to a sequence (5) an iterator of key-value pairs for setting on a string. Python 3.4 added the very useful (for me) __getnewargs_ex__<http://docs.python.org/3.4/library/pickle.html#object.__getnewargs_ex__>, which returns a pair: (1) a tuple of positional arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__> (2) a dict of keyword arguments for __new__<http://docs.python.org/3.4/reference/datamodel.html#object.__new__> Therefore, I am proposing that __reduce__ return somehow these keyword arguments for __new__. Best, Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/5fac6a92/attachment.html> From abarnert at yahoo.com Sun Mar 23 23:48:04 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 23 Mar 2014 15:48:04 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> Message-ID: <6E79C662-4C99-40BD-90AC-AFF39EFC8F04@yahoo.com> On Mar 23, 2014, at 3:51, Devin Jeanpierre <jeanpierreda at gmail.com> wrote: > On Sun, Mar 23, 2014 at 3:40 AM, Sturla Molden <sturla.molden at gmail.com> wrote: >> On 15/03/14 01:09, Antoine Pitrou wrote: >> >>> Really? That should be up to the third-party library implementing the @ >>> operator for its types, not to the language itself: Python _suggests_ >>> an use case for @, it doesn't mandate it (especially as there's no >>> appropriate data type in the stdlib). >> >> >> array.array is an appropriate type for supporting @ for matrix >> multiplication. > > I was thinking the function type, as a composition operator. Matrix > multiplication corresponds exactly with composition of linear > functions, so it makes some sort of sense. And it has some kind of > neat similarity to decorators, which generalize function composition. > > If you rename "matmul" to "compose" and "matpow" to something like > "iterate" (except less confusing), then you've generalized the > operators a little and given them more potential use cases. This sounds nice from a purity point of view, but I'm not sure it's a good idea. First, matrix multiplication only corresponds exactly with function composition if matrix*vector multiplication corresponds exactly with function calling, which is generally not true in programming. You can't assign a matrix to f, then write f(v) to apply the matrix as an operation; you write f @ v. Meanwhile, Python doesn't even have compose in functools, much less in builtins, and I don't think anyone had seriously proposed it in decades. If we don't even need it in the stdlib, do we really need is as an operator? Also, is it right-compose or left-compose? (That is, does (f at g)(2) call f then g, or g then f?) One answer is obvious to people who think mathematically, the other to novices. And even if you argue that the right answer to that is to teach the novices the right way to think about it clearly, reverse composition is still often useful; that's why many languages that have a compose function or operator also have rcompose and/or make it trivial to write as flip(compose). Also, how exactly do you define compose? This is easy in a language like Haskell, where every function takes one argument and returns one value (which, because of currying, may actually be a function that takes the next argument...), but in Python, where functions can take any number of arguments, with default values and *args and keyword-only arguments and so on, it's a lot less clear. Does the second function need to be a single-argument function, or do tuple returns match to *args, or something different? Different third-party function libraries use different answers to that, sometimes offering more than one under different names, but obviously the operator has to be either compose1 or composestar, and then the other one won't even be in the stdlib? Meanwhile, looking at languages that care about mathematical purity and about functional composition: you don't write . for matrix multiplication in Haskell, you write `multStd` or, more likely, define *, **, `mult`, or something else more readable. From steve at pearwood.info Mon Mar 24 01:49:43 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Mar 2014 11:49:43 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lgmdm0$2ng$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> Message-ID: <20140324004943.GX16526@ando> On Sun, Mar 23, 2014 at 11:40:10AM +0100, Sturla Molden wrote: > On 15/03/14 01:09, Antoine Pitrou wrote: > > >Really? That should be up to the third-party library implementing the @ > >operator for its types, not to the language itself: Python _suggests_ > >an use case for @, it doesn't mandate it (especially as there's no > >appropriate data type in the stdlib). > > array.array is an appropriate type for supporting @ for matrix > multiplication. No it isn't. The PEP even discusses it: [quote] array objects cannot represent multidimensional data at all, which makes ``__matmul__`` much less useful. Second, providing a quality implementation of matrix multiplication is highly non-trivial. Naive nested loop implementations are very slow and providing one in CPython would just create a trap for users. But the alternative -- providing a modern, competitive matrix multiply -- would require that CPython link to a BLAS library, which brings a set of new complications. In particular, several popular BLAS libraries (including the one that ships by default on OS X) currently break the use of ``multiprocessing`` [#blas-fork]_. And finally, we'd have to add quite a bit beyond ``__matmul__`` before ``memoryview`` or ``array.array`` would be useful for numeric work -- like elementwise versions of the other arithmetic operators, just to start. [end quote] I don't think we should be looking for additional use-cases for @. Either the PEP stands on its own, and @ is approved for matrix multiplication (and possibly @@ for exponentiation), or it isn't. If the PEP is accepted -- and I think it should be -- then people will invent new uses for @. Or they won't. Either way, it doesn't matter. -- Steven From steve at pearwood.info Mon Mar 24 02:08:24 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Mar 2014 12:08:24 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> Message-ID: <20140324010824.GY16526@ando> On Sun, Mar 23, 2014 at 03:51:24AM -0700, Devin Jeanpierre wrote: > If you rename "matmul" to "compose" and "matpow" to something like > "iterate" (except less confusing), then you've generalized the > operators a little and given them more potential use cases. -1 The dunder methods are typically named for the most common or typical use-case, not generalised or marginal ones e.g.: + __add__ not __concat__ * __mul__ not __repeat__ or __repetition__ < __lt__ not __subset__ & __and__ not __ampersand__ or __conjunction__ -- Steven From jeanpierreda at gmail.com Mon Mar 24 02:47:40 2014 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 23 Mar 2014 18:47:40 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140324010824.GY16526@ando> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> <20140324010824.GY16526@ando> Message-ID: <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> On Sun, Mar 23, 2014 at 6:08 PM, Steven D'Aprano <steve at pearwood.info> wrote: > -1 > > The dunder methods are typically named for the most common or > typical use-case, not generalised or marginal ones e.g.: > > + __add__ not __concat__ > * __mul__ not __repeat__ or __repetition__ > < __lt__ not __subset__ > & __and__ not __ampersand__ or __conjunction__ Thanks for taking issue with the not-silly point. Ironically enough, all of those but "and" are actually general names for operators, but I agree that probably wasn't intentional. Anyway, in practice people seem to get what it means for something to be "and"ed with another thing. There's an intuition that many things using the operator treat it in a roughly analogous way, and the same is true for many other overloadable operators in Python. These conventions help make code more understandable and are great. I'm not sure such a thing would happen with an operator for which the stdlib and documentation and everything insists is only for matrix multiplication. It'd be like if we had "__twos_complement_bitwise_and__". It's one thing to generalize an operator name, and another to not make it very specific. -- Devin From cjwilliams43 at gmail.com Mon Mar 24 03:55:16 2014 From: cjwilliams43 at gmail.com (Colin J. Williams) Date: Sun, 23 Mar 2014 22:55:16 -0400 Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199 In-Reply-To: <mailman.43259.1395623316.18129.python-ideas@python.org> References: <mailman.43259.1395623316.18129.python-ideas@python.org> Message-ID: <532F9E94.4090509@gmail.com> #4 seems to refer to the Python basic system. However numpy.array does deal with multi-dimension arrays. On 23-Mar-2014 9:08 PM, python-ideas-request at python.org wrote: > ------------------------------ Message: 4 Date: Mon, 24 Mar 2014 > 11:49:43 +1100 From: Steven D'Aprano <steve at pearwood.info> To: > python-ideas at python.org Subject: Re: [Python-ideas] [RFC] draft PEP: > Dedicated infix operators for matrix multiplication and matrix power > Message-ID: <20140324004943.GX16526 at ando> Content-Type: text/plain; > charset=us-ascii On Sun, Mar 23, 2014 at 11:40:10AM +0100, Sturla > Molden wrote: >> On 15/03/14 01:09, Antoine Pitrou wrote: >> >>> Really? That should be up to the third-party library implementing the @ >>> operator for its types, not to the language itself: Python _suggests_ >>> an use case for @, it doesn't mandate it (especially as there's no >>> appropriate data type in the stdlib). >> array.array is an appropriate type for supporting @ for matrix >> multiplication. > No it isn't. The PEP even discusses it: > > > [quote] > array objects cannot represent multidimensional data at all, > which makes ``__matmul__`` much less useful. Second, > providing a quality implementation of matrix multiplication > is highly non-trivial. Naive nested loop implementations > are very slow and providing one in CPython would just create > a trap for users. But the alternative -- providing a modern, > competitive matrix multiply -- would require that CPython link > to a BLAS library, which brings a set of new complications. > In particular, several popular BLAS libraries (including the > one that ships by default on OS X) currently break the use of > ``multiprocessing`` [#blas-fork]_. And finally, we'd have to > add quite a bit beyond ``__matmul__`` before ``memoryview`` > or ``array.array`` would be useful for numeric work -- like > elementwise versions of the other arithmetic operators, just > to start. > [end quote] > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140323/61eade49/attachment.html> From ben+python at benfinney.id.au Mon Mar 24 04:00:41 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 24 Mar 2014 14:00:41 +1100 Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199 References: <mailman.43259.1395623316.18129.python-ideas@python.org> <532F9E94.4090509@gmail.com> Message-ID: <85vbv4qp0m.fsf@benfinney.id.au> "Colin J. Williams" <cjwilliams43 at gmail.com> writes: > #4 seems to refer to the Python basic system. Howdy Colin. To participate in a discussion, please respond to individual messages so the threads of discussion make sense. To do that, first change your subscription to disable ?digest? mode <URL:http://www.list.org/mailman-member/node27.html> so that you receive the individual messages in the discussion. -- \ ?If I had known what it would be like to have it all... I might | `\ have been willing to settle for less.? ?Jane Wagner, via Lily | _o__) Tomlin | Ben Finney From steve at pearwood.info Mon Mar 24 04:12:42 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Mar 2014 14:12:42 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> <20140324010824.GY16526@ando> <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> Message-ID: <20140324031242.GZ16526@ando> On Sun, Mar 23, 2014 at 06:47:40PM -0700, Devin Jeanpierre wrote: > I'm not sure such a thing would happen with an operator for which the > stdlib and documentation and everything insists is only for matrix > multiplication. They would *insist* would they? The documentation would come with a great warning "Thou Shalt Not Use __matmul__ For Anything Except Matrix Multiplication"? I suppose that will be like the similar (imaginary) warning that __pow__ is only permitted to be used for numerical exponentiation and that any other use is forbidden on pain of being slapped with a fish. I think it is highly unlikely that people will be frightened off from overloading @ by the name. If people happily use __lt__ for subset checking, which is *nothing* like less-than, I'm sure they'll use @ as well. People do far weirder things -- there's even a recipe out there on the Internet for implementing arbitrary named pseudo-operators using __or__ : vector |dot| vector But suppose you're right, and people don't use it for anything but matrix multiplication. So what? Who cares? It's not like anyone is made worse off by the change. None of my code currently uses the @ operator. If numpy starts to use it, and I don't, I'll be no worse off than I am now. > It'd be like if we had "__twos_complement_bitwise_and__". Having __matmul__ would be like if we had __xor__ __pow__ __sub__ __div__ __mod__ __pos__ or __neg__ -- Steven From jeanpierreda at gmail.com Mon Mar 24 04:48:49 2014 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 23 Mar 2014 20:48:49 -0700 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140324031242.GZ16526@ando> References: <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> <20140324010824.GY16526@ando> <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> <20140324031242.GZ16526@ando> Message-ID: <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com> On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote: > I think it is highly unlikely that people will be frightened off from > overloading @ by the name. If people happily use __lt__ for subset > checking, which is *nothing* like less-than, Actually, no. <, or "less than", is the exact way it's spelled for any partial order, and subset relations are probably the most famous non-numeric example of a partial order. In fact, if you look at the most common construction of the natural numbers (0 = frozenset(), succ(x) = frozenset(x) | x), it is more complicated than it needs to be only so that the < operator means the same thing for natural numbers when treated as numbers or as sets. >> It'd be like if we had "__twos_complement_bitwise_and__". > > Having __matmul__ would be like if we had __xor__ __pow__ > __sub__ __div__ __mod__ __pos__ or __neg__ Then we disagree. -- Devin From tjreedy at udel.edu Mon Mar 24 05:58:03 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 24 Mar 2014 00:58:03 -0400 Subject: [Python-ideas] Python-ideas Digest, Vol 88, Issue 199 In-Reply-To: <85vbv4qp0m.fsf@benfinney.id.au> References: <mailman.43259.1395623316.18129.python-ideas@python.org> <532F9E94.4090509@gmail.com> <85vbv4qp0m.fsf@benfinney.id.au> Message-ID: <lgoe0l$fbf$2@ger.gmane.org> On 3/23/2014 11:00 PM, Ben Finney wrote: > "Colin J. Williams" > <cjwilliams43 at gmail.com> writes: > >> #4 seems to refer to the Python basic system. > > Howdy Colin. To participate in a discussion, please respond to > individual messages so the threads of discussion make sense. > > To do that, first change your subscription to disable ?digest? mode > <URL:http://www.list.org/mailman-member/node27.html> so that you receive > the individual messages in the discussion. Or disable mail messages and access the list as newsgroup gmane.comp.python.ideas as news.gmane.org (as I am because I also do not want a flood of messages to delete). -- Terry Jan Reedy From steve at pearwood.info Mon Mar 24 13:06:31 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 24 Mar 2014 23:06:31 +1100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com> References: <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> <20140324010824.GY16526@ando> <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> <20140324031242.GZ16526@ando> <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com> Message-ID: <20140324120631.GA16526@ando> On Sun, Mar 23, 2014 at 08:48:49PM -0700, Devin Jeanpierre wrote: > On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote: > > I think it is highly unlikely that people will be frightened off from > > overloading @ by the name. If people happily use __lt__ for subset > > checking, which is *nothing* like less-than, > > Actually, no. <, or "less than", is the exact way it's spelled for any > partial order, and subset relations are probably the most famous > non-numeric example of a partial order. I will accept that subsets are an example of partial order and so I was wrong to say that that it has nothing to do with __lt__ and __gt__. But I disagree that < is the usual symbol for subset. Neither Wikipedia nor Wolfram Mathworld mention it: https://en.wikipedia.org/wiki/Subset http://mathworld.wolfram.com/Subset.html I was taught to use symbols ? and ? for proper subset and subset-or- equal. I've also seen ? and ? used instead. But prior to Python, I've never seen < used. If it's used in mathematics, it's a niche use. -- Steven From ncoghlan at gmail.com Mon Mar 24 14:28:06 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 24 Mar 2014 23:28:06 +1000 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140324120631.GA16526@ando> References: <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <CABicbJK0ezYPqp9dQ9p+UKQ_rGvbrYWN3UOqmUoqdm6ZZiHi2w@mail.gmail.com> <20140324010824.GY16526@ando> <CABicbJL+RUc8oLACERsLO10=dGvigmzAzQ4BhXm=NxmgmEOJHg@mail.gmail.com> <20140324031242.GZ16526@ando> <CABicbJJY57Ec04jGB_u3WCfM_Vf5rZBCLm7N21hUvwx=zeDSsg@mail.gmail.com> <20140324120631.GA16526@ando> Message-ID: <CADiSq7eKPM75=efthpSYn_i9+mB6TWV0tAVix69z-5X0p5V1sQ@mail.gmail.com> On 24 March 2014 22:06, Steven D'Aprano <steve at pearwood.info> wrote: > On Sun, Mar 23, 2014 at 08:48:49PM -0700, Devin Jeanpierre wrote: >> On Sun, Mar 23, 2014 at 8:12 PM, Steven D'Aprano <steve at pearwood.info> wrote: >> > I think it is highly unlikely that people will be frightened off from >> > overloading @ by the name. If people happily use __lt__ for subset >> > checking, which is *nothing* like less-than, >> >> Actually, no. <, or "less than", is the exact way it's spelled for any >> partial order, and subset relations are probably the most famous >> non-numeric example of a partial order. > > I will accept that subsets are an example of partial order and so I was > wrong to say that that it has nothing to do with __lt__ and __gt__. I normally let python-ideas subthreads run indefinitely without commenting, but in this case... *really* not a productive tangent :) Ellipsis, extended slicing and memory views were added to the core language and C API definitions for the numeric computing folks without a compelling stdlib use case (at least at the time). Adding __matmul__ for their benefit really isn't much of a stretch, and it makes the distinction *they* care about clear (__mul__ = element-wise multiplication, __matmul__ = matrix multiplication). Previous proposals along these lines failed because they overgeneralised without compelling benefits from the extra complexity, while this one hits the sweet spot of solving the *actual* problem to be solved with just some minor technical details to work out. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From sturla.molden at gmail.com Tue Mar 25 07:05:04 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Tue, 25 Mar 2014 06:05:04 +0000 (UTC) Subject: [Python-ideas] Infix matrix-multiply, but not general infix operators? References: <F5ADF678-96C8-403C-AD65-7C8759376A58@gmail.com> <20140320102443.GS16526@ando> <3188722C-47C5-4A6C-8D4E-A633FB3F2448@yahoo.com> <lgfi6t$45l$1@ger.gmane.org> <CANNZFEmwNJxFzVZBxkUv0k1CUbV5vc-U6PcmEeBbXFphqSr4dg@mail.gmail.com> <20140321001539.GU16526@ando> <CANNZFE=TX=XyQMae=W83a_9JXHAZQ_DBoK=WfEUxfCrSqpF-Wg@mail.gmail.com> <CAPTjJmqWgWkaB_F33VSrnK42PRFo=8QKkdZkoYR5aLQHToOjig@mail.gmail.com> <39BC89A9-9077-46B1-97E7-812EF9018424@yahoo.com> <532BC797.1060106@stoneleaf.us> <CAPTjJmpGZqZkmdsNX-AE7+O96rZ1wp6hCHbD=gisKJNFivWZFg@mail.gmail.com> Message-ID: <564858166417420257.372728sturla.molden-gmail.com@news.gmane.org> Chris Angelico <rosuav at gmail.com> wrote: > where "as operator", instead of specifying the name to bind to, > specifies that it be something other than a name. (Preferably, it > should use a keyword, rather than "operator", which is a valid name.) > This would go immediately under future statements, and would affect > the compilation of that module: any instance of the word "spam" is now > a keyword operator, so the above would be invalid. I'm still not > convinced it would be a good thing, but this at least gives a > per-module way to create operators. Cluttering the syntax with arbitrary operators is a very bad idea, IMHO. Sturla From techtonik at gmail.com Tue Mar 25 07:29:44 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 25 Mar 2014 09:29:44 +0300 Subject: [Python-ideas] PEP feedback loop (with tracker account) Message-ID: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> It is not that hard to implement an approval function for the PEP site (to improve the visibility and the process). For example, on the page http://legacy.python.org/dev/peps/pep-0231/ the field "Status: Rejected" can be linkified to bring up a popup window with short info about quantity of people who approved the PEP (content details can be extended later). It is not that hard to add a disqus-style JS form to set status of your position towards PEP using tracker account. Will that be useful for Python community? -- anatoly t. From rosuav at gmail.com Tue Mar 25 10:07:07 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Mar 2014 20:07:07 +1100 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> Message-ID: <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com> On Tue, Mar 25, 2014 at 5:29 PM, anatoly techtonik <techtonik at gmail.com> wrote: > For example, on the page > http://legacy.python.org/dev/peps/pep-0231/ > the field "Status: Rejected" can be linkified to bring up a > popup window with short info about quantity of people who > approved the PEP (content details can be extended later). By "approved" do you mean those who wish it hadn't been rejected? I don't think that's terribly useful, and it implies a democratic approach to Python's development, which doesn't exist. More useful might be for the "Status: Rejected" to link to a python-dev post formally rejecting it, but that's about all. Can you elaborate on both what this would show, and how it would be useful? ChrisA From techtonik at gmail.com Tue Mar 25 17:13:32 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 25 Mar 2014 19:13:32 +0300 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <CAPTjJmqLQVguBtObq0EyatwHo6Gyd=6X_dktscprn52ydG3jNA@mail.gmail.com> Message-ID: <CAPkN8xJ6bZgoP9V+nN0-Db1aFRBo9RhU+XS9JstPtF0jt3u7XQ@mail.gmail.com> On Tue, Mar 25, 2014 at 12:07 PM, Chris Angelico <rosuav at gmail.com> wrote: > On Tue, Mar 25, 2014 at 5:29 PM, anatoly techtonik <techtonik at gmail.com> wrote: >> For example, on the page >> http://legacy.python.org/dev/peps/pep-0231/ >> the field "Status: Rejected" can be linkified to bring up a >> popup window with short info about quantity of people who >> approved the PEP (content details can be extended later). > > By "approved" do you mean those who wish it hadn't been rejected? I > don't think that's terribly useful, and it implies a democratic > approach to Python's development, which doesn't exist. More useful > might be for the "Status: Rejected" to link to a python-dev post > formally rejecting it, but that's about all. Can you elaborate on both > what this would show, and how it would be useful? By "approved" I mean that people sign the specific PEP version as "fully read and agree with all the clauses". For example, I don't have time to read PEPs thoroughly and I wonder how many people have. There might be a problem that PEPs don't receive enough exposure and timely feedback. I don't know - it is uncertain. Also having some kind of recorded "achievement" will help to get some motivation to do thorough reviews. Also many people don't imagine how many heads are working on a single PEP and think that there are a lot of people already. It may be that knowing the stats they will be more interested to contribute. The fourth reason is that this basic feedback system can be extended to include specific facts to be addressed, but that out of scope for now. -- anatoly t. From stephen at xemacs.org Tue Mar 25 20:16:46 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 26 Mar 2014 04:16:46 +0900 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> Message-ID: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > It is not that hard to implement an approval function for > the PEP site (to improve the visibility and the process). I gather you mean "thumbs up/down" buttons. > Will that be useful for Python community? No. As Chris points out, the PEP process is not a voting process. (I think that's what he meant. I would say that it is "democratic" in the sense that as much as possible we try to come to consensus on a "best" proposal.) And people do switch as PEPs evolve, but I doubt they'd be likely to change their "vote" on the PEP site. Links to the discussion (or perhaps even blog posts of folks who strongly dissent) are far more relevant. From rosuav at gmail.com Tue Mar 25 20:58:59 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Mar 2014 06:58:59 +1100 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <CAPTjJmryKhpBj9yeh9z1Dc87H6ShttZYteVibg86j7nF1RnSCw@mail.gmail.com> On Wed, Mar 26, 2014 at 6:16 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > No. As Chris points out, the PEP process is not a voting process. (I > think that's what he meant. I would say that it is "democratic" in > the sense that as much as possible we try to come to consensus on a > "best" proposal.) And people do switch as PEPs evolve, but I doubt > they'd be likely to change their "vote" on the PEP site. Yes, that's what I meant. Python is built, as most open source projects are, on a model of "a king with his ministers and citizens". The king, of course, is normally Guido [1]; core devs are Guido's ministers and close advisers; everyone else who uses Python is a citizen. (Drawing the line between ministers and citizens is a tad harder than in a parliamentary structure, so I'm going to horrendously cheat and say that ministers can be appointed and/or can step down very easily and quickly.) A PEP is basically a petition, raised and discussed. It has a champion (at least by the time it becomes a PEP), and the Champion will seek to demonstrate how the proposal will be of great benefit. This doesn't strictly equate to "number of signatures", which is how these sorts of petitions are often treated in modern politics; it's more about strength of argument. So not only is this system not democratic in the obvious sense of "majority rules" (if you get more + votes than -, the PEP is approved), it's also almost never even about gathering numerical support (which is what this *would* be able to show). There have been a few times when popular support is canvassed (PEP 308, for instance), but even then, it's certainly not "most popular option wins" (it wasn't then, although the chosen syntax was clearly in the popular group rather than the less-popular group). Recording who has fully read the PEP and agrees with every little part of it would be nearly impossible as the PEP evolves, so by the latest definition, it's impossible to calculate, as well as being almost-never-useful. Searching the python-ideas archive for the PEP number and reading through the discussion is probably the only way to find out after the event, so I sympathize with the desire to simply eyeball a figure and see; but I can't imagine it being truly useful in all that many cases. ChrisA [1] Some subprojects (like IDLE?) function identically with a different king. From barry at python.org Tue Mar 25 21:27:12 2014 From: barry at python.org (Barry Warsaw) Date: Tue, 25 Mar 2014 16:27:12 -0400 Subject: [Python-ideas] PEP feedback loop (with tracker account) References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> <CAPTjJmryKhpBj9yeh9z1Dc87H6ShttZYteVibg86j7nF1RnSCw@mail.gmail.com> Message-ID: <20140325162712.3f6ba000@anarchist.wooz.org> From PEP 1: What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing major new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as text files in a versioned repository, their revision history is the historical record of the feature proposal [1]. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140325/cd332abe/attachment.sig> From g.brandl at gmx.net Tue Mar 25 23:46:52 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 25 Mar 2014 23:46:52 +0100 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> Message-ID: <lgt10d$h6l$1@ger.gmane.org> Am 25.03.2014 07:29, schrieb anatoly techtonik: > It is not that hard to implement an approval function for > the PEP site (to improve the visibility and the process). > > For example, on the page > http://legacy.python.org/dev/peps/pep-0231/ > the field "Status: Rejected" can be linkified to bring up a > popup window with short info about quantity of people who > approved the PEP (content details can be extended later). > > It is not that hard to add a disqus-style JS form to set > status of your position towards PEP using tracker > account. > > Will that be useful for Python community? No. What is useful is to keep the "Post-History" header up to date, which refers people to the python-dev threads where all the discussion and "voting" takes place. I could also imagine adding direct links to the archive; however many PEPs already add references in the text where appropriate. Georg From ethan at stoneleaf.us Wed Mar 26 03:21:07 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 25 Mar 2014 19:21:07 -0700 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <lgt10d$h6l$1@ger.gmane.org> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <lgt10d$h6l$1@ger.gmane.org> Message-ID: <53323993.7030505@stoneleaf.us> On 03/25/2014 03:46 PM, Georg Brandl wrote: > > No. What is useful is to keep the "Post-History" header up to > date, which refers people to the python-dev threads where all the > discussion and "voting" takes place. So should that header just include the dates when new threads were spawned, or if significant time had passed since the last posting in the "current" thread? -- ~Ethan~ From abarnert at yahoo.com Wed Mar 26 06:22:23 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 25 Mar 2014 22:22:23 -0700 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <lgt10d$h6l$1@ger.gmane.org> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <lgt10d$h6l$1@ger.gmane.org> Message-ID: <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com> On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote: > Am 25.03.2014 07:29, schrieb anatoly techtonik: >> It is not that hard to implement an approval function for >> the PEP site (to improve the visibility and the process). >> >> For example, on the page >> http://legacy.python.org/dev/peps/pep-0231/ >> the field "Status: Rejected" can be linkified to bring up a >> popup window with short info about quantity of people who >> approved the PEP (content details can be extended later). >> >> It is not that hard to add a disqus-style JS form to set >> status of your position towards PEP using tracker >> account. >> >> Will that be useful for Python community? > > No. What is useful is to keep the "Post-History" header up to > date, which refers people to the python-dev threads where all the > discussion and "voting" takes place. > > I could also imagine adding direct links to the archive; however > many PEPs already add references in the text where appropriate. Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful. I can't think of anything else that would really be interesting at all. A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs. From g.brandl at gmx.net Wed Mar 26 07:17:17 2014 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 26 Mar 2014 07:17:17 +0100 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <53323993.7030505@stoneleaf.us> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <lgt10d$h6l$1@ger.gmane.org> <53323993.7030505@stoneleaf.us> Message-ID: <lgtrcu$t6i$1@ger.gmane.org> Am 26.03.2014 03:21, schrieb Ethan Furman: > On 03/25/2014 03:46 PM, Georg Brandl wrote: >> >> No. What is useful is to keep the "Post-History" header up to >> date, which refers people to the python-dev threads where all the >> discussion and "voting" takes place. > > So should that header just include the dates when new threads were spawned, or if significant time had passed since the > last posting in the "current" thread? PEP 1 says: """ The Created header records the date that the PEP was assigned a number, while Post-History is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. """ And new threads are usually accompanied by a posting of a revision of the PEP. Georg From abarnert at yahoo.com Wed Mar 26 09:34:01 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Wed, 26 Mar 2014 01:34:01 -0700 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <lgt10d$h6l$1@ger.gmane.org> <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com> <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com> Message-ID: <33AA2C5B-A9AF-4B05-A967-0FD41CD6C6B5@yahoo.com> On Mar 26, 2014, at 1:07, anatoly techtonik <techtonik at gmail.com> wrote: > On Wed, Mar 26, 2014 at 8:22 AM, Andrew Barnert <abarnert at yahoo.com> wrote: >> On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote: >>> >>> No. What is useful is to keep the "Post-History" header up to >>> date, which refers people to the python-dev threads where all the >>> discussion and "voting" takes place. >>> >>> I could also imagine adding direct links to the archive; however >>> many PEPs already add references in the text where appropriate. >> >> Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful. > > The goal to link threads and PEP versions is reachable at the next > iteration once status per person is implemented. The benefit from > the system is that you don't need to dedicate a person to do this. > >> A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs. > > It is not a post. It is status per revision. The difference between > post and status is that status gives immediate overview (diffstat) > while post needs English skills + time to bring an up-to-date > image in you head. So, status is declarative and can be processed > by machines. Post is procedural and requires manual human labor. The other difference is that nobody cares about a count of +/- status, so it doesn't really matter how easy it is to collect. If someone is trying to decide whether or not the PEP discussion is missing his point of view, a vote isn't going to tell him that. If Guido is trying to decide whether to approve or reject the PEP, he's not going to go by the vote. So what's the point of it? From techtonik at gmail.com Wed Mar 26 08:35:30 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 26 Mar 2014 10:35:30 +0300 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <87bnwuqeap.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <CAPkN8xJiND9FFQiqfd4gV4u2BH=6n7FWV7AgNeJPYW_YTfmxdQ@mail.gmail.com> On Tue, Mar 25, 2014 at 10:16 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote: > anatoly techtonik writes: > > > It is not that hard to implement an approval function for > > the PEP site (to improve the visibility and the process). > > I gather you mean "thumbs up/down" buttons. It is not primitive "thumbs up/down" button and not voting. The feedback loop goal to get information about community awareness - not for making decision to accept or reject a proposal. Therefore it is not democracy. The approval is not a simple boolean - it is the status object, which contains fields to make feedback loop actually useful, and may provide information, such as: 1. LGTM 2. Have questions 3. Have comments 4. Will watch later 5. tl;dr 6. meh 7. .... <link> to discussion 8. ... The status sums up opinion for a given person at a given moment. Benefits: - see who is interested - see who was able to review - see who was able to read it - see which version has questions I don't want to get into much featurecreeping, which is incrementally possible. Will that idea be useful for Python community? > Links to the discussion (or perhaps even blog posts of folks who > strongly dissent) are far more relevant. This can be added, so that folks can tie their posts to PEP feedback loop. 9. Dissent <link> (new word for me, sounds strange) -- anatoly t. From techtonik at gmail.com Wed Mar 26 09:07:32 2014 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 26 Mar 2014 11:07:32 +0300 Subject: [Python-ideas] PEP feedback loop (with tracker account) In-Reply-To: <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com> References: <CAPkN8x+jG67PpseMQNr0wc5spPZbNRMsrgw0tNFmySENAjbo-Q@mail.gmail.com> <lgt10d$h6l$1@ger.gmane.org> <810F9AEC-B071-434B-8B08-EC62988D4026@yahoo.com> Message-ID: <CAPkN8x+C66AUfDngA5_HYJZFaHp1VqhuJEoZk6h74=s4QsWJUQ@mail.gmail.com> On Wed, Mar 26, 2014 at 8:22 AM, Andrew Barnert <abarnert at yahoo.com> wrote: > On Mar 25, 2014, at 15:46, Georg Brandl <g.brandl at gmx.net> wrote: >> >> No. What is useful is to keep the "Post-History" header up to >> date, which refers people to the python-dev threads where all the >> discussion and "voting" takes place. >> >> I could also imagine adding direct links to the archive; however >> many PEPs already add references in the text where appropriate. > > Having someone dedicate themselves to linking in all significant -ideas and -dev discussions (and relevant blog posts, etc.) would be incredibly useful. The goal to link threads and PEP versions is reachable at the next iteration once status per person is implemented. The benefit from the system is that you don't need to dedicate a person to do this. > A +/- vote is meaningless, and expecting people to summarize their mailing-list discussions in a single post on the PEP site doesn't seem likely to capture sufficient interest on any but the most contentious PEPs. It is not a post. It is status per revision. The difference between post and status is that status gives immediate overview (diffstat) while post needs English skills + time to bring an up-to-date image in you head. So, status is declarative and can be processed by machines. Post is procedural and requires manual human labor. Status can serve as intermediate entrypoint to join current problem field (as opposed to rereading posts from the start). -- anatoly t. From g.rodola at gmail.com Wed Mar 26 15:37:40 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Wed, 26 Mar 2014 15:37:40 +0100 Subject: [Python-ideas] Turn signal.SIG* constants into enums Message-ID: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com> In Python 3.4 some module constants were turned into enums. The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*). For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_* constants: http://bugs.python.org/issue21068 To me signal module looks like another good candidate which could benefit from this. Here's a real world example: http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm Thoughts? -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140326/4e681b06/attachment.html> From guido at python.org Wed Mar 26 16:37:53 2014 From: guido at python.org (Guido van Rossum) Date: Wed, 26 Mar 2014 08:37:53 -0700 Subject: [Python-ideas] Turn signal.SIG* constants into enums In-Reply-To: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com> References: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com> Message-ID: <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com> SGTM. On Mar 26, 2014 7:38 AM, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote: > In Python 3.4 some module constants were turned into enums. > The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*). > For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_* constants: > http://bugs.python.org/issue21068 > To me signal module looks like another good candidate which could benefit > from this. > Here's a real world example: > > http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm > > Thoughts? > > -- > Giampaolo - http://grodola.blogspot.com > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140326/85e7fa14/attachment.html> From sturla.molden at gmail.com Wed Mar 26 19:19:15 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 26 Mar 2014 19:19:15 +0100 Subject: [Python-ideas] [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <20140324004943.GX16526@ando> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <20140314224812.GR12595@ando> <532393A3.9010005@canterbury.ac.nz> <20140315005032.773acc95@fsol> <532396FA.7010601@canterbury.ac.nz> <20140315010944.6b8244e1@fsol> <lgmdm0$2ng$1@ger.gmane.org> <20140324004943.GX16526@ando> Message-ID: <lgv5ml$ifu$1@ger.gmane.org> On 24/03/14 01:49, Steven D'Aprano wrote: > No it isn't. The PEP even discusses it: (...) Well, even with a single dimension, matrix multiplication has a useful interpretation (inner product). At least the intention in numpy is to interpret "vector @ vector" as the inner product. Thus it is useful both for list and array in the standard library. A possible usecase for it in the standard library would be to simplify the code in the statistics module. But the main consideration is to make @ benign to the rest of the Python community. > I don't think we should be looking for additional use-cases for @. > Either the PEP stands on its own, and @ is approved for matrix > multiplication (and possibly @@ for exponentiation), The consensus on the NumPy list seems to be that @@ can wait. It is also ambigous. For example matrix @@ 0.5 could be interpreted as the Cholesky factorization or the matrix square root, depending on context. Also an expression like Z = (X @@ -1) @ Y should for numerical stability (and speed) be computed as "solve X @ Z = Y" e.g. with LU, SVD or QR factorization. This would require calling a function like solve(X,Y). Matlab actually has a special back-divide operator for this reason: X \ Y. This means that @@ is ambiguous except for positive integer powers or -1. But the common case of matrix @@ -1 is not as useful as we might think. Then we are left with exponentiation to positive integer powers, which is not nearly common enough to justify a special operator. Sturla From sturla.molden at gmail.com Wed Mar 26 19:59:52 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 26 Mar 2014 19:59:52 +0100 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> Message-ID: <lgv82q$i25$1@ger.gmane.org> On 14/03/14 18:53, Guido van Rossum wrote: > - Did you consider a duck-typing (is that the word?) attribute? > E.g. a*b is elementwise multiplication; a.M*b must be used for > matrix multiplication. This is exactly what has been tried with the types numpy.matrix and numpy.array. For numpy.matrix the * operator means matrix multiplication and for numpy.array it means Hadamard product. In practice, it does not work as we want. That is the raison d'etre for this PEP. First, we often needs both operators in the same expression. This makes it very hard to read, in fact a function call is always more readable. Second, NumPy often has to return temporary arrays from binary operators, and the duck-typing of these must be controlled too. If we are unlucky a temporary array will have the wrong type, and we end up with errors that are very hard to track down. So for any serious code, matrix multiplication is written with a function call: numpy.dot. numpy.matrix is for this reason mostly useful for teaching students, not for serious numerical hacking with Python. It is due for retirement from NumPy. A special operator will work, however, because languages like Matlab and R have proven that it does. Many in the NumPy community have extensive experience with Matlab and R, and we know it works with two multiplication operators. In contrast, we have proven that duck-typing is not useful in this context. Though this PEP is written by Nathaniel, the whole NumPy dev team is behind it. It has been planned and discussed for a long time, both on the NumPy mailing list and on Github. But Nathaniel took the job to write it down. So other options has been considered very carefully. > - Is @@ really necessary? It seems the consensus on the NumPy list is that asking for @@ can wait. - An expression like matrix @@ 0.5 is ambiguous because it could mean the matrix square root or the Cholesky factorization. - An expression like Z = (X @@ -1) @ Y should for numerical stability ans speed be written as solving a set of equations: X @ Z = Y. Symbolically it is the same, but numerically it is not. So for most practical purposes, matrix exponentiation will require a function call anyway. The unambiguous case of positive integer powers might not be common enough. Sturla From robert.kern at gmail.com Wed Mar 26 20:14:09 2014 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 26 Mar 2014 19:14:09 +0000 Subject: [Python-ideas] Fwd: [RFC] draft PEP: Dedicated infix operators for matrix multiplication and matrix power In-Reply-To: <lgv82q$i25$1@ger.gmane.org> References: <CAPJVwBk-Jp+b=jzREN1uN7cB+e3Kgy__3_3XBr_jycahTXgMAQ@mail.gmail.com> <CAPJVwB=Lxrdoy-jEzbDXOi5w289ypd0oAtvKNx2MnkbHpi4xAw@mail.gmail.com> <CAP7+vJL90exeCWDD7002oWUudg+67X=r0mjhan0WP29U7CKJbA@mail.gmail.com> <CAP7+vJ+ioX-VHOQmqSMi2cAq1dQaSG-msMsUBVRHxmiCDz7J=A@mail.gmail.com> <lgv82q$i25$1@ger.gmane.org> Message-ID: <lgv8th$t0h$1@ger.gmane.org> On 2014-03-26 18:59, Sturla Molden wrote: > On 14/03/14 18:53, Guido van Rossum wrote: > >> - Did you consider a duck-typing (is that the word?) attribute? >> E.g. a*b is elementwise multiplication; a.M*b must be used for >> matrix multiplication. > > This is exactly what has been tried with the types numpy.matrix and numpy.array. For the record, no it hasn't. The proposal here is different than the numpy.matrix status quo. The proposed result of `a.M*b` would be an ndarray, not whatever the .M type is. The .M type would not be intended to be used outside of a multiplication expression. It is unlikely to propagate through ones code the way that numpy.matrix does (at least, it would be considered a bug to let a free .M type loose). One could safely write their functions expecting only numpy.ndarrays. While this proposal has been considered and rejected, it is not the same as numpy.matrix and has been rejected for different reasons. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From g.rodola at gmail.com Thu Mar 27 17:00:54 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Thu, 27 Mar 2014 17:00:54 +0100 Subject: [Python-ideas] Turn signal.SIG* constants into enums In-Reply-To: <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com> References: <CAFYqXL85wYL+WtTDginnZFQhoRGu+_8OsXo53ZdJpsv32+mU5Q@mail.gmail.com> <CAP7+vJL1=vrGY7Ybjzuv1Sy52ir2hfv3siQ_ZqwGrxUGu8VvoQ@mail.gmail.com> Message-ID: <CAFYqXL9d67v7zDQS8tsETEatRp4cuOixNn9kSpbB6Remk6NFOw@mail.gmail.com> Here's a patch: http://bugs.python.org/issue21076 On Wed, Mar 26, 2014 at 4:37 PM, Guido van Rossum <guido at python.org> wrote: > SGTM. > On Mar 26, 2014 7:38 AM, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote: > >> In Python 3.4 some module constants were turned into enums. >> The affected modules were socket (AF_INET*, SOCK_*) and pslistlib (FMT_*). >> For 3.5 there's a plan to do the same thing for ssl's PROTOCOL_* >> constants: >> http://bugs.python.org/issue21068 >> To me signal module looks like another good candidate which could benefit >> from this. >> Here's a real world example: >> >> http://docs.python.org/dev/library/asyncio-eventloop.html#example-set-signal-handlers-for-sigint-and-sigterm >> >> Thoughts? >> >> -- >> Giampaolo - http://grodola.blogspot.com >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140327/a38510d9/attachment.html> From clay.gerrard at gmail.com Thu Mar 27 21:16:50 2014 From: clay.gerrard at gmail.com (Clay Gerrard) Date: Thu, 27 Mar 2014 13:16:50 -0700 Subject: [Python-ideas] iterating over a generator while sending values Message-ID: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> Hi, I often use generators when I want to bake some smarts into my producers in a typical producer and consumer pattern. Twice in the past year or so on different projects after a few iterations on my producer/consumer code ("iterations", ha!) I would eventually find I wanted my consumer to send some simple feedback to the producer - when using generators send works great for this! I normally start here: q = producer() def consume(q): for item in q: success = handle(item) But then ended up here: def consume(q): success = None while True: try: item = q.send(success) except StopIteration: break success = handle(item) But this always feels clunky; almost "out of order" - I've normally got some logging and other bits in there. I'm just trying to iterate over the queue, the StopIteration just feels like noise. It seems I always started out with a simple iteration and then end up refactoring in the send function once I realize I need to add something extra into my producer: I want to just say: def consume(q): for item in q: success = handle(item) q.send(success) # oh how I wish I could just add this here! But I can just "tack on" the send because it consumes too, so I have to rephrase the "simple" iteration into a "while True try except StopIteration break" - yuk. I normally just end up adding a little helper function, which I've named "sending" and use it like this: def consume(q): for item in sending(q, lambda: success): success = handle(item) ^ much closer to the original iterative code! Here's the implementation if you didn't already guess: # this function is missing from stdlib for some reason... def sending(g, sender): """ Iterate over g with send :params g: the iterable :params sender: the callable returning the value to send """ yield g.next() while True: yield g.send(sender()) This will raise StopIteration as soon as g is exhausted, but since the caller is already consuming "sending" with a for loop it all works ok in the end. Does anyone else have a different way of iterating over a generator while sending values? Does everyone just already have a similar function stuck in their bag? Does this happen to other folks frequently enough that a generalizable solution could be added to stdlib? Maybe it's too simple to be worth anyone's time or i'm missing something that already works better... Thanks! -clayg -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140327/6748d86d/attachment.html> From tjreedy at udel.edu Thu Mar 27 23:24:54 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 27 Mar 2014 18:24:54 -0400 Subject: [Python-ideas] iterating over a generator while sending values In-Reply-To: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> Message-ID: <lh28fj$iif$1@ger.gmane.org> On 3/27/2014 4:16 PM, Clay Gerrard wrote: > Hi, > > I often use generators when I want to bake some smarts into my producers > in a typical producer and consumer pattern. Twice in the past year or > so on different projects after a few iterations on my producer/consumer > code ("iterations", ha!) I would eventually find I wanted my consumer to > send some simple feedback to the producer - when using generators send > works great for this! > > I normally start here: > > q = producer() > > def consume(q): > for item in q: > success = handle(item) > > But then ended up here: > > def consume(q): > success = None > while True: > try: > item = q.send(success) > except StopIteration: > break > success = handle(item) > > But this always feels clunky; almost "out of order" - I've normally got > some logging and other bits in there. I'm just trying to iterate over > the queue, the StopIteration just feels like noise. For Python, the normal mode of iteration is 'pull' mode -- the consumer pulls from the producer. I believe the send method was mostly designed for inversion of control with 'push' mode -- the producer pushes to the consumer with .send. The consumer might yield feedback to the producer. Sending feedback in pull mode is a secondary use. The iterator protocol was designed for for statements: the initial call to iter(), multiple calls to next() and catching of StopIteration are all handled internally. Any time you want to do anything non-standard, other that stop before the iterator is exhausted, you have to deal with iter, next, and StopIteration yourself. StopIteration is an essential part of the protocol, not noise. An example: def min_f(iterable): it = iter(iterable) try: val = next(it) except StopIteration: raise ValueError("an empty iterable has no minimum") for ob in it: if ob < val: val = ob return val > It seems I always started out with a simple iteration and then end up > refactoring in the send function once I realize I need to add something > extra into my producer: > > I want to just say: > > def consume(q): > for item in q: > success = handle(item) > q.send(success) # oh how I wish I could just add this here! > > But I can just "tack on" the send because it consumes too, I think you meant "can't", but the problem with .send is that is returns the next items prematurely in addition to receiving something. That is for the design reason given above. You could just 'tack on' if you wrote an iterator class with .__iter__, .__next__, and .send methods, where .send only received (and returned an ignorable None). > so I have to rephrase the "simple" iteration into a "while True try except > StopIteration break" - yuk. Catching StopIteration is part of explicit use of the iterator protocol, as in min_f above. In this case, one could make everything implicit again by using functools.reduce, with in its two parameter form, abstracts away the code pattern used in min_f. from functools import reduce def min_r(iterable): return reduce(lambda v, o: v if v <= o else o, iterable) s = [5,6,4,6,3] print(min_f(s), min_r(s)) >>> 3, 3 > I normally just end up adding a little helper function, which I've named > "sending" and use it like this: > > def consume(q): success = None > for item in sending(q, lambda: success): > success = handle(item) > # this function is missing from stdlib for some reason... Because it is short, special purpose, and pretty easy to write. > def sending(g, sender): > """ > Iterate over g with send > > :params g: the iterable > :params sender: the callable returning the value to send > """ > yield g.next() > while True: > yield g.send(sender()) Writing an iterator or generator that wrap an iterator or, as in this case, specifically a generator, is common. Some common patterns that work for iterators as input are included in itertools. More are given in the recipes section of the doc. The criterion for being in the module is that the abstraction either be common or useful in composition with others in the module. A specialized generator-input-only iterator would not fit. It is possible that a 'gentools' module would be useful, but it should start on PyPI. > This will raise StopIteration as soon as g is exhausted, as per the protocol > but since the caller is already consuming "sending" with a for loop > it all works ok in the end. It does not matter whether the caller uses your generator in a for loop or explicit iter-while-next-except-StopIteration construct. -- Terry Jan Reedy From greg.ewing at canterbury.ac.nz Fri Mar 28 00:17:43 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Mar 2014 12:17:43 +1300 Subject: [Python-ideas] iterating over a generator while sending values In-Reply-To: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> Message-ID: <5334B197.2050300@canterbury.ac.nz> Clay Gerrard wrote: > def consume(q): > success = None > while True: > try: > item = q.send(success) > except StopIteration: > break > success = handle(item) My first thought would be to ask whether you really need to do things this way round. Instead of the consumer pulling things from the producer, can you have the producer push them to the consumer? Then the response is just a function return value. Assuming you do need to pull, my solution would be something like this (not tested): class Feedbackerator: def __init__(self, base): self.base = base self.response = None def __next__(self): return self.base.send(self.response) def consume(q): f = Feedbackerator(q) for item in f: f.response = handle(item) -- Greg From aj at erisian.com.au Fri Mar 28 05:22:01 2014 From: aj at erisian.com.au (Anthony Towns) Date: Fri, 28 Mar 2014 14:22:01 +1000 Subject: [Python-ideas] iterating over a generator while sending values In-Reply-To: <5334B197.2050300@canterbury.ac.nz> References: <CA+_JKzp+5WGHN4fqsH+D=Y6CBXcYb97CcKgrA4MUjY2PrO4_9w@mail.gmail.com> <5334B197.2050300@canterbury.ac.nz> Message-ID: <CAJS_LCVWKbFppGcwtwrFMyp1rL0Vup+HM8xPJswezayfrwvumA@mail.gmail.com> On 28 March 2014 09:17, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Clay Gerrard wrote: >> def consume(q): >> success = None >> while True: >> try: >> item = q.send(success) >> except StopIteration: >> break >> success = handle(item) > > Assuming you do need to pull, my solution would be > something like this (not tested): > class Feedbackerator: > def __init__(self, base): > self.base = base > self.response = None > def __next__(self): > return self.base.send(self.response) > def consume(q): > f = Feedbackerator(q) > for item in f: > f.response = handle(item) Ooo, nice. Minor tweaks to make it work in py2, and not require you to set the response every iteration: class Feedbackerator(object): def __init__(self, iterator): self.iter = iter(iterator) self.resp = None def next(self): r, self.resp = self.resp, None return self.iter.send(r) def __iter__(self): return self def consume(q): qi = Feedbackerator(q) for item in qi: res = handle(item) if needs_feedback(res): qi.resp = feedback_for(res) Cheers, aj -- Anthony Towns <aj at erisian.com.au> From ncoghlan at gmail.com Fri Mar 28 11:27:33 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Mar 2014 20:27:33 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor Message-ID: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> One of the current annoyances with the bytes type in Python 3 is the way the constructor handles integers: >>> bytes(3) b'\x00\x00\x00' It would be far more consistent with the behaviour of other bytes interfaces if the result of that call was instead b'\x03'. Instead, to get that behaviour, you currently have to wrap it in a list or other iterable: >>> bytes([3]) b'\x03' The other consequence of this is that's currently no neat way to convert the integers produced by various bytes APIs back to a length one bytes object - we have no binary equivalent of "chr" to convert an integer in the range 0-255 inclusive to its bytes counterpart. The acceptance of PEP 361 means we'll get another option (b"%c".__mod__) but that's hardly what anyone would call obvious. However, during a conversation today, a possible solution occurred to me: a "bytes.chr" class method, that served as an alternate constructor. That idea results in the following 3 part proposal: 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP 361 defined "b'%c' % x" 2. Add "bytearray.allnull" and "bytes.allnull" such that "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int handling 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as not only ambiguous, but actually a genuine bug magnet (it's way too easy to accidentally pass a large integer and try to allocate a ridiculously large bytes object) For point 2, I also considered the following alternative names before settling on "allnull": - bytes.null sounds too much like an alias for b"\x00" - bytes.nulls just sounded too awkward to say (too many sibilants) - bytes.zeros I can never remember how to spell (bytes.zeroes?) - bytearray.cleared sort of worked, but bytes.cleared? - ditto for bytearray.prealloc and bytes.prealloc (latter makes no sense) That last is also a very C-ish name (although it is a rather C-ish operation). Anyway, what do people think? Does anyone actually *like* the way the bytes constructor in Python 3 currently handles integers and want to keep it forever? Does the above proposal sound like a reasonable suggestion for improvement in 3.5? Does this hit PEP territory, since it's changing the signature and API of a builtin? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Fri Mar 28 11:50:28 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 28 Mar 2014 21:50:28 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <20140328105027.GH16526@ando> On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote: > One of the current annoyances with the bytes type in Python 3 is the > way the constructor handles integers: [...] > 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP > 361 defined "b'%c' % x" +1 on the concept, but please not "chr". How about bytes.byte(x)? That emphasises that you are dealing with a single byte, and avoids conflating chars (strings of length 1) with bytes. > 2. Add "bytearray.allnull" and "bytes.allnull" such that > "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int > handling +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" instead. I'm indifferent about bytes.allnull. I'm not sure why I would use it in preference to b'\0'*x, or for that matter why I would use it at all. I suppose if there's a bytearray.allnul, for symmetry there should be a bytes.allnul as well. > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > not only ambiguous, but actually a genuine bug magnet (it's way too > easy to accidentally pass a large integer and try to allocate a > ridiculously large bytes object) +1 [...] > Anyway, what do people think? Does anyone actually *like* the way the > bytes constructor in Python 3 currently handles integers and want to > keep it forever? Not me! > Does the above proposal sound like a reasonable > suggestion for improvement in 3.5? Does this hit PEP territory, since > it's changing the signature and API of a builtin? On the vanishingly small chance that there is universal agreement on this, and a minimum of bike-shedding, I think it would still be useful to write up a brief half page PEP linking to the discussion here. -- Steven From ncoghlan at gmail.com Fri Mar 28 12:00:44 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Mar 2014 21:00:44 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328105027.GH16526@ando> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> Message-ID: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> On 28 March 2014 20:50, Steven D'Aprano <steve at pearwood.info> wrote: >> 2. Add "bytearray.allnull" and "bytes.allnull" such that >> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int >> handling > > +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" > instead. If I could consistently remember whether to include the "e" or not, "zeroes" would be my preference as well, but almost every time I go to type it, I have to pause... > I'm indifferent about bytes.allnull. I'm not sure why I would use it > in preference to b'\0'*x, or for that matter why I would use it at all. > I suppose if there's a bytearray.allnul, for symmetry there should be a > bytes.allnul as well. It's actually in the proposal solely because "bytes(x)" already works that way, and "this is deprecated, use bytes.allnull instead" is a much easier deprecation to sell. >> Does the above proposal sound like a reasonable >> suggestion for improvement in 3.5? Does this hit PEP territory, since >> it's changing the signature and API of a builtin? > > On the vanishingly small chance that there is universal agreement on > this, and a minimum of bike-shedding, I think it would still be useful > to write up a brief half page PEP linking to the discussion here. Yeah, that's make sense, so regardless of what happens in this thread, I'll still be putting a PEP together and asking Guido for his verdict. Heaps of time before 3.5 though... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From stephen at xemacs.org Fri Mar 28 12:23:06 2014 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 28 Mar 2014 20:23:06 +0900 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <874n2ipnxh.fsf@uwakimon.sk.tsukuba.ac.jp> Nick Coghlan writes: > One of the current annoyances with the bytes type in Python 3 is the > way the constructor handles integers: > > >>> bytes(3) > b'\x00\x00\x00' > > It would be far more consistent with the behaviour of other bytes > interfaces if the result of that call was instead b'\x03'. +1 ~ 13:49$ python3.3 Python 3.3.5 (default, Mar 9 2014, 08:10:50) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> str(3) '3' >>> list(3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not iterable >>> tuple(3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not iterable The behavior of str(3) is not exactly an argument *for* your claim, but it's not an argument against since there's no reason to expect a byte to have an ASCII interpretation in general. > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > not only ambiguous, but actually a genuine bug magnet (it's way too > easy to accidentally pass a large integer and try to allocate a > ridiculously large bytes object) bytes.chr(accidental_large_integer) presumably raises. I don't really see a bigger problem with the current behavior than that (in the accidentally large case). Both are likely to distress users. > Anyway, what do people think? Does anyone actually *like* the way the > bytes constructor in Python 3 currently handles integers and want to > keep it forever? No. bytearray I'd have to think about. > Does the above proposal sound like a reasonable suggestion for > improvement in 3.5? Yes. > Does this hit PEP territory, since it's changing the signature and > API of a builtin? No opinion. Steve From solipsis at pitrou.net Fri Mar 28 12:22:51 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 28 Mar 2014 12:22:51 +0100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <20140328122251.1933e4e4@fsol> On Fri, 28 Mar 2014 20:27:33 +1000 Nick Coghlan <ncoghlan at gmail.com> wrote: > One of the current annoyances with the bytes type in Python 3 is the > way the constructor handles integers: > > >>> bytes(3) > b'\x00\x00\x00' > > It would be far more consistent with the behaviour of other bytes > interfaces if the result of that call was instead b'\x03'. Which other bytes interfaces are you talking about? > However, during a conversation today, a possible solution occurred to > me: a "bytes.chr" class method, that served as an alternate > constructor. That idea results in the following 3 part proposal: > > 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP > 361 defined "b'%c' % x" You mean bytes.chr(x) is equivalent to bytes([x]). The intent is slightly more obvious indeed, so I'd inclined to be +0, but Python isn't really a worse language if it doesn't have that alternative spelling. > Anyway, what do people think? Does anyone actually *like* the way the > bytes constructor in Python 3 currently handles integers and want to > keep it forever? I don't like it, but I also don't think it's enough of a nuisance to be deprecated. (the indexing behaviour of bytes objects is far more annoying) Regards Antoine. From cory at lukasa.co.uk Fri Mar 28 12:28:57 2014 From: cory at lukasa.co.uk (Cory Benfield) Date: Fri, 28 Mar 2014 11:28:57 +0000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <CAH_hAJEu5m74QWYavwxQcmiMc3M1ZMLjstDQYQMqZUXZzid8uA@mail.gmail.com> On 28 March 2014 10:27, Nick Coghlan <ncoghlan at gmail.com> wrote: > Anyway, what do people think? Does anyone actually *like* the way the > bytes constructor in Python 3 currently handles integers and want to > keep it forever? I hate it, I'd love to see a change. It bit me when I wrote my first Python-3-only project, and while it wasn't hard to debug it was utterly baffling for quite a while. Writing bytes([integer]) is pretty ugly, and it makes me sad each time I see it. I've even commented occurrences of it to remind myself that this is really the way I meant to do it. > Does the above proposal sound like a reasonable > suggestion for improvement in 3.5? Does this hit PEP territory, since > it's changing the signature and API of a builtin? I'd like to see it improved, even if I can't take advantage of that improvement for quite some time. Sounds like a PEP to me. From ncoghlan at gmail.com Fri Mar 28 12:59:33 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Mar 2014 21:59:33 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328122251.1933e4e4@fsol> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> Message-ID: <CADiSq7ej4ENX52HK-n716cvK8Sg7Be51KMAnq-dZqp7NciTvXw@mail.gmail.com> On 28 March 2014 21:22, Antoine Pitrou <solipsis at pitrou.net> wrote: > On Fri, 28 Mar 2014 20:27:33 +1000 > Nick Coghlan <ncoghlan at gmail.com> wrote: >> One of the current annoyances with the bytes type in Python 3 is the >> way the constructor handles integers: >> >> >>> bytes(3) >> b'\x00\x00\x00' >> >> It would be far more consistent with the behaviour of other bytes >> interfaces if the result of that call was instead b'\x03'. > > Which other bytes interfaces are you talking about? The ones where a length 1 bytes object and the corresponding integer are interchangeable. For example, containment testing: >>> b"a" in b"abc" True >>> 97 in b"abc" True Compare: >>> len(bytes(b"a")) 1 >>> len(bytes(97)) 97 That's the inconsistency that elevates the current constructor behaviour from weird to actively wrong for me - it doesn't match the way other bytes interfaces have evolved over the course of the Python 3 series. >> However, during a conversation today, a possible solution occurred to >> me: a "bytes.chr" class method, that served as an alternate >> constructor. That idea results in the following 3 part proposal: >> >> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP >> 361 defined "b'%c' % x" > > You mean bytes.chr(x) is equivalent to bytes([x]). The intent is > slightly more obvious indeed, so I'd inclined to be +0, but Python > isn't really a worse language if it doesn't have that alternative > spelling. > >> Anyway, what do people think? Does anyone actually *like* the way the >> bytes constructor in Python 3 currently handles integers and want to >> keep it forever? > > I don't like it, but I also don't think it's enough of a nuisance to be > deprecated. > (the indexing behaviour of bytes objects is far more annoying) Oops, I forgot to explain the context where this idea came up: I was trying to figure out how to iterate over a bytes object or wrap an indexing operation to get a length 1 byte sequence rather than an integer. Currently: probably muck about with lambda or a comprehension With this change (using Steven D'Aprano's suggested name): for x in map(bytes.byte, data): # x is a length 1 bytes object, not an int x = bytes.byte(data[0]) # ditto bytes.byte could actually apply the same policy as some other APIs and also accept ASCII text code points in addition to length 1 bytes objects and integers below 256. Since changing the iteration and indexing behaviour of bytes and bytearray within the Python 3 series isn't feasible, this idea is about making the current behaviour easier to deal with. And yes, this is definitely going to need a PEP :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From python at 2sn.net Fri Mar 28 13:06:28 2014 From: python at 2sn.net (Alexander Heger) Date: Fri, 28 Mar 2014 23:06:28 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> Message-ID: <CAN3CYHz=e1DXR+izFn33BxcRNGJzoHLR6HY8Ukg4Uix7OgmNEw@mail.gmail.com> >> +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" >> instead. > > If I could consistently remember whether to include the "e" or not, > "zeroes" would be my preference as well, but almost every time I go to > type it, I have to pause... in Numpy a similar routine is In [1]: np.zeros_like Out[1]: <function numpy.core.numeric.zeros_like> so keeping the spelling consistent with other key libraries, bytes.zeros() and bytearray.zeros() would be better. -Alexander From greg.ewing at canterbury.ac.nz Fri Mar 28 13:16:16 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 29 Mar 2014 01:16:16 +1300 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> Message-ID: <53356810.5060602@canterbury.ac.nz> Nick Coghlan wrote: > If I could consistently remember whether to include the "e" or not, > "zeroes" would be my preference as well, but almost every time I go to > type it, I have to pause... Whichever you choose, please make sure it's the same as whatever numpy uses! -- Greg From ron3200 at gmail.com Fri Mar 28 14:41:02 2014 From: ron3200 at gmail.com (Ron Adam) Date: Fri, 28 Mar 2014 09:41:02 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <lh3u4v$44v$1@ger.gmane.org> On 03/28/2014 06:27 AM, Nick Coghlan wrote: > However, during a conversation today, a possible solution occurred to > me: a "bytes.chr" class method, that served as an alternate > constructor. That idea results in the following 3 part proposal: > > 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP > 361 defined "b'%c' % x" I'd expect to use it this way... bytes.chr('3') Which wouldn't be correct. We have this... >>> (3).to_bytes(1,"little") b'\x03' OR... >>> def hexclean(h): ... if h.startswith('0x'): ... h = h[2:] ... if not len(h)//2: ... h = '0' + h ... return h ... >>> bytes.fromhex(hexclean(hex(3))) b'\x03' Both of those seem like way too much work to me. Bytes is a container object, so it makes sense to do... bytes([3]) The other interface you're suggesting is more along the lines of converting an object to bytes. That would be a nice feature if it can be made to work with more objects, and then covert back again. (even if only a few types was supported) b_three = bytes.from_obj(3) three = b_three.to_obj(int) That wouldn't be limited to values between 0 and 255 like the hex version above. (it doesn't need a length, or byteorder args.) > 2. Add "bytearray.allnull" and "bytes.allnull" such that > "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int > handling Numpy uses zeros and shape... >>> y = np.zeros((2, 3, 4)) >>> y.shape (2, 3, 4) (Without the e.) I'm not sure if it's a good idea to use zeros with a different signature. There has been mentioned, that adding multidimensional slicing to python may be done. So it may be a good idea to follow numpy's usage. > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > not only ambiguous, but actually a genuine bug magnet (it's way too > easy to accidentally pass a large integer and try to allocate a > ridiculously large bytes object) Agree Cheers, Ron From barry at python.org Fri Mar 28 15:28:17 2014 From: barry at python.org (Barry Warsaw) Date: Fri, 28 Mar 2014 10:28:17 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <20140328102817.76d2b7f9@anarchist.wooz.org> Great idea Nick. If I may dip my brush in some paint buckets. On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote: >However, during a conversation today, a possible solution occurred to >me: a "bytes.chr" class method, that served as an alternate >constructor. That idea results in the following 3 part proposal: > >1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP >361 defined "b'%c' % x" I agree with Steven that bytes.byte() is a better spelling. >2. Add "bytearray.allnull" and "bytes.allnull" such that >"bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int >handling I like bytearray.fill() for this. The first argument would be the fill count, but it could take an optional second argument for the byte value to fill it with, which would of course default to zero. E.g. >>> bytearray.fill(5) bytearray(b'\x00\x00\x00\x00\x00') >>> bytearray.fill(5, 97) bytearray(b'aaaaa') >3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as >not only ambiguous, but actually a genuine bug magnet (it's way too >easy to accidentally pass a large integer and try to allocate a >ridiculously large bytes object) +1 >Does the above proposal sound like a reasonable suggestion for improvement in >3.5? Very much so. >Does this hit PEP territory, since it's changing the signature and API >of a builtin? I don't much care either way. A PEP is not *wrong* (especially if we all start painting), but I think a tracker issue would be fine too. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/03763789/attachment.sig> From breamoreboy at yahoo.co.uk Fri Mar 28 15:48:01 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 28 Mar 2014 14:48:01 +0000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <lh422u$ngs$1@ger.gmane.org> On 28/03/2014 14:28, Barry Warsaw wrote: > Great idea Nick. If I may dip my brush in some paint buckets. > > On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote: > >> However, during a conversation today, a possible solution occurred to >> me: a "bytes.chr" class method, that served as an alternate >> constructor. That idea results in the following 3 part proposal: >> >> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP >> 361 defined "b'%c' % x" > > I agree with Steven that bytes.byte() is a better spelling. > >> 2. Add "bytearray.allnull" and "bytes.allnull" such that >> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int >> handling > > I like bytearray.fill() for this. The first argument would be the fill count, > but it could take an optional second argument for the byte value to fill it > with, which would of course default to zero. E.g. > >>>> bytearray.fill(5) > bytearray(b'\x00\x00\x00\x00\x00') >>>> bytearray.fill(5, 97) > bytearray(b'aaaaa') > >> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as >> not only ambiguous, but actually a genuine bug magnet (it's way too >> easy to accidentally pass a large integer and try to allocate a >> ridiculously large bytes object) > > +1 > >> Does the above proposal sound like a reasonable suggestion for improvement in >> 3.5? > > Very much so. > >> Does this hit PEP territory, since it's changing the signature and API >> of a builtin? > > I don't much care either way. A PEP is not *wrong* (especially if we all > start painting), but I think a tracker issue would be fine too. > > Cheers, > -Barry > I was under the impression that Ethan Furman had raised an issue, or at least commented on one, but I couldn't find such a thing, am I simply mistaken? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From ethan at stoneleaf.us Fri Mar 28 15:31:01 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 28 Mar 2014 07:31:01 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <533587A5.9010701@stoneleaf.us> On 03/28/2014 07:28 AM, Barry Warsaw wrote: > Great idea Nick. If I may dip my brush in some paint buckets. > > On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote: > >> However, during a conversation today, a possible solution occurred to >> me: a "bytes.chr" class method, that served as an alternate >> constructor. That idea results in the following 3 part proposal: >> >> 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP >> 361 defined "b'%c' % x" > > I agree with Steven that bytes.byte() is a better spelling. +1 >> 2. Add "bytearray.allnull" and "bytes.allnull" such that >> "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int >> handling > > I like bytearray.fill() for this. The first argument would be the fill count, > but it could take an optional second argument for the byte value to fill it > with, which would of course default to zero. E.g. +1 >> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as >> not only ambiguous, but actually a genuine bug magnet (it's way too >> easy to accidentally pass a large integer and try to allocate a >> ridiculously large bytes object) > > +1 +1 >> Does the above proposal sound like a reasonable suggestion for improvement in >> 3.5? +1 >> Does this hit PEP territory, since it's changing the signature and API >> of a builtin? > > I don't much care either way. A PEP is not *wrong* (especially if we all > start painting), but I think a tracker issue would be fine too. You mean like http://bugs.python.org/issue20895 ? :) -- ~Ethan~ From barry at python.org Fri Mar 28 16:13:47 2014 From: barry at python.org (Barry Warsaw) Date: Fri, 28 Mar 2014 11:13:47 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <533587A5.9010701@stoneleaf.us> Message-ID: <20140328111347.20495c04@anarchist.wooz.org> On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote: >You mean like http://bugs.python.org/issue20895 ? :) Step *away* from the time machine. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/53c52266/attachment-0001.sig> From nas-python at arctrix.com Fri Mar 28 16:21:15 2014 From: nas-python at arctrix.com (Neil Schemenauer) Date: Fri, 28 Mar 2014 09:21:15 -0600 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328105027.GH16526@ando> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> Message-ID: <20140328152115.GA8168@python.ca> On 2014-03-28, Steven D'Aprano wrote: > On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote: > +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" > instead. I think numpy uses 'zeros' so we should use that. > > Anyway, what do people think? Does anyone actually *like* the way the > > bytes constructor in Python 3 currently handles integers and want to > > keep it forever? > > Not me! It's odd, IMHO. When looking to implement %-interpolation for bytes I did some searching to see how widely it is used. There are a few uses in the standard library but it really should have been a special constructor. Accidental uses probably outnumber legitimate ones, quite a bad API design. Neil From alexander.belopolsky at gmail.com Fri Mar 28 16:28:39 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 28 Mar 2014 11:28:39 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <lh3u4v$44v$1@ger.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <lh3u4v$44v$1@ger.gmane.org> Message-ID: <CAP7h-xY8fit8s-b2QcTYq=nktO4dTDTRqrCxwUgo3ZcXZ-hEjA@mail.gmail.com> On Fri, Mar 28, 2014 at 9:41 AM, Ron Adam <ron3200 at gmail.com> wrote: > >>> y = np.zeros((2, 3, 4)) > >>> y.shape > (2, 3, 4) > > (Without the e.) I'm not sure if it's a good idea to use zeros with a > different signature. > np.zeros() can be called with an integer (length) argument as well: >>> np.zeros(3) array([ 0., 0., 0.]) To be completely analogous to the proposed bytes constructor, you would have to specify the data type, though: >>> np.zeros(3, 'B') array([0, 0, 0], dtype=uint8) +1 for bytes.zeros(n) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/7d23c9e3/attachment.html> From alexander.belopolsky at gmail.com Fri Mar 28 16:36:40 2014 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 28 Mar 2014 11:36:40 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <CAP7h-xZOhPxWV2rB7MVQ_ny1kr=eRifz05=tdMM7tRokYxjGmQ@mail.gmail.com> On Fri, Mar 28, 2014 at 10:28 AM, Barry Warsaw <barry at python.org> wrote: > I like bytearray.fill() for this. The first argument would be the fill > count, > but it could take an optional second argument for the byte value to fill it > with, which would of course default to zero. E.g. > > >>> bytearray.fill(5) > bytearray(b'\x00\x00\x00\x00\x00') > >>> bytearray.fill(5, 97) > bytearray(b'aaaaa') > In numpy this constructor is spelled "full": >>> np.full(3, 42) array([ 42., 42., 42.]) (Which makes more sense in the full context: np.zeros, np.ones, np.empty, np.full.) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/e197e8ed/attachment.html> From solipsis at pitrou.net Fri Mar 28 17:29:27 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 28 Mar 2014 17:29:27 +0100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <20140328172927.42983ab6@fsol> On Fri, 28 Mar 2014 10:28:17 -0400 Barry Warsaw <barry at python.org> wrote: > > I like bytearray.fill() for this. The first argument would be the fill count, > but it could take an optional second argument for the byte value to fill it > with, which would of course default to zero. E.g. > > >>> bytearray.fill(5) > bytearray(b'\x00\x00\x00\x00\x00') > >>> bytearray.fill(5, 97) > bytearray(b'aaaaa') Do we need another spelling for a very uncommon use case? >>> bytearray([0]) * 5 bytearray(b'\x00\x00\x00\x00\x00') >>> bytearray([97]) * 5 bytearray(b'aaaaa') Let's keep it simple. Regards Antoine. From ericsnowcurrently at gmail.com Fri Mar 28 17:34:20 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 28 Mar 2014 10:34:20 -0600 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> Message-ID: <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com> On Mar 28, 2014 5:01 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote: > If I could consistently remember whether to include the "e" or not, > "zeroes" would be my preference as well, but almost every time I go to > type it, I have to pause... However, it reads better, which trumps the possible minor spelling inconvenience. (all else equal, readability trumps write-ability, etc.) > It's actually in the proposal solely because "bytes(x)" already works > that way, and "this is deprecated, use bytes.allnull instead" is a > much easier deprecation to sell. Fair point, though I also agree with Steven here. Could the replacement (b'\0'*x) be a doc note instead of a new method? -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/3ca290ec/attachment.html> From ericsnowcurrently at gmail.com Fri Mar 28 17:37:03 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Fri, 28 Mar 2014 10:37:03 -0600 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328122251.1933e4e4@fsol> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> Message-ID: <CALFfu7DBiMXOqctxwqTb6HB05334CaYEaWZfcbtf6y=v435C4Q@mail.gmail.com> On Mar 28, 2014 5:24 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote: > (the indexing behaviour of bytes objects is far more annoying) +1 -eric -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/e56e4291/attachment.html> From jeanpierreda at gmail.com Fri Mar 28 17:40:00 2014 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Fri, 28 Mar 2014 09:40:00 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> <CADiSq7d2+y+jkA-sgkeQ11OdR81eyi7j5=y+F4Qye-WV5ZRZXg@mail.gmail.com> <CALFfu7D9h6_a=2i53OiEVpSYuSyHK73g=VPkbUUp18LLt40tBg@mail.gmail.com> Message-ID: <CABicbJL2LigSbOAZpyodWbh77FK-wfMdfS_n77qroF5gzey6sg@mail.gmail.com> On Fri, Mar 28, 2014 at 9:34 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote: > > On Mar 28, 2014 5:01 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote: >> If I could consistently remember whether to include the "e" or not, >> "zeroes" would be my preference as well, but almost every time I go to >> type it, I have to pause... > > However, it reads better, which trumps the possible minor spelling > inconvenience. (all else equal, readability trumps write-ability, etc.) If I have to pause while reading to figure out if the code has a typo, that isn't better. -- Devin From steve at pearwood.info Fri Mar 28 18:02:23 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 29 Mar 2014 04:02:23 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328152115.GA8168@python.ca> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328105027.GH16526@ando> <20140328152115.GA8168@python.ca> Message-ID: <20140328170222.GK16526@ando> On Fri, Mar 28, 2014 at 09:21:15AM -0600, Neil Schemenauer wrote: > On 2014-03-28, Steven D'Aprano wrote: > > On Fri, Mar 28, 2014 at 08:27:33PM +1000, Nick Coghlan wrote: > > +1 on bytearray.allnull, with a mild preference for spelling it "zeroes" > > instead. > > I think numpy uses 'zeros' so we should use that. We would we want to duplicate their spelling error? *wink* I think I'd rather Nick's original suggestion allnull than "zeros". "Zeros" sounds like it ought to be the name of a Greek singer :-) I'm aware that zeros/zeroes are both considered acceptable variant spellings. Not acceptable to me *wink* -- Steven From benjamin at python.org Fri Mar 28 18:06:24 2014 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 28 Mar 2014 17:06:24 +0000 (UTC) Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> Message-ID: <loom.20140328T180549-288@post.gmane.org> Antoine Pitrou <solipsis at ...> writes: > I don't like it, but I also don't think it's enough of a nuisance to be > deprecated. > (the indexing behaviour of bytes objects is far more annoying) I agree. We would add a method that iterates over single bytes as bytes objects instead of integers. From guido at python.org Fri Mar 28 18:17:34 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 28 Mar 2014 10:17:34 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <loom.20140328T180549-288@post.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> Message-ID: <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson <benjamin at python.org>wrote: > Antoine Pitrou <solipsis at ...> writes: > > I don't like it, but I also don't think it's enough of a nuisance to be > > deprecated. > > (the indexing behaviour of bytes objects is far more annoying) > > I agree. We would add a method that iterates over single bytes as bytes > objects instead of integers. > Now that's making sense. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/89438e19/attachment.html> From benjamin at python.org Fri Mar 28 18:23:09 2014 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 28 Mar 2014 17:23:09 +0000 (UTC) Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> Message-ID: <loom.20140328T182140-300@post.gmane.org> Guido van Rossum <guido at ...> writes: > > > > On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson <benjamin at python.org> wrote: > Antoine Pitrou <solipsis <at> ...> writes: > > I don't like it, but I also don't think it's enough of a nuisance to be > > deprecated. > > (the indexing behaviour of bytes objects is far more annoying) > I agree. We would add a method that iterates over single bytes as bytes > objects instead of integers. > > > > > Now that's making sense. Is this sarcasm about my poor writing skills ("would" should be "could") or actual support for the idea? From solipsis at pitrou.net Fri Mar 28 18:25:44 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 28 Mar 2014 18:25:44 +0100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> Message-ID: <20140328182544.62ce91ad@fsol> On Fri, 28 Mar 2014 17:06:24 +0000 (UTC) Benjamin Peterson <benjamin at python.org> wrote: > Antoine Pitrou <solipsis at ...> writes: > > I don't like it, but I also don't think it's enough of a nuisance to be > > deprecated. > > (the indexing behaviour of bytes objects is far more annoying) > > I agree. We would add a method that iterates over single bytes as bytes > objects instead of integers. In my experience, indexing is far more common than iterating over bytes. Regards Antoine. From flying-sheep at web.de Fri Mar 28 18:31:58 2014 From: flying-sheep at web.de (Philipp A.) Date: Fri, 28 Mar 2014 18:31:58 +0100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> Message-ID: <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com> 2014-03-28 18:17 GMT+01:00 Guido van Rossum <guido at python.org>: > On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson <benjamin at python.org>wrote: > >> I agree. We would add a method that iterates over single bytes as bytes >> objects instead of integers. >> > > Now that's making sense. > even if some people don?t like it, the iterating over bytes objects makes perfect sense. in every statically typed language, a byte is most commonly repesented as an integer in the range [0,255]. the annoyance has two sources: 1. python 2 byte strings (aka ?native strings?) behaved like that (and unicode strings behave like that because python has no char type) 2. byte strings are represented like strings, including ascii-compatible parts as if they were ASCII text. therefore, paople think single bytes would be something akin to chars instead of simply integers from 0 to 255. b'a'[0] == 97 looks strange, but let?s not forget that it?s actually b'\x61'[0] == 0x61 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/69b70f27/attachment-0001.html> From solipsis at pitrou.net Fri Mar 28 18:42:41 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 28 Mar 2014 18:42:41 +0100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com> Message-ID: <20140328184241.04693b7a@fsol> On Fri, 28 Mar 2014 18:31:58 +0100 "Philipp A." <flying-sheep at web.de> wrote: > even if some people don?t like it, the iterating over bytes objects makes > perfect sense. > > in every statically typed language, a byte is most commonly repesented as > an integer in the range [0,255]. > > the annoyance has two sources: > > 1. > > python 2 byte strings (aka ?native strings?) behaved like that (and > unicode strings behave like that because python has no char type) > 2. > > byte strings are represented like strings, including ascii-compatible > parts as if they were ASCII text. The actual source isn't merely cultural. It's that most of the time, bytes objects are used as containers of arbitrary binary data, not as arrays of integers (on which you would do element-wise arithmetic calculations, for instance). So, even if the current behaviour makes sense, it's not optimal for the common uses of bytes objects. Regards Antoine. From guido at python.org Fri Mar 28 18:50:34 2014 From: guido at python.org (Guido van Rossum) Date: Fri, 28 Mar 2014 10:50:34 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <loom.20140328T182140-300@post.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <loom.20140328T182140-300@post.gmane.org> Message-ID: <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com> Support. On Fri, Mar 28, 2014 at 10:23 AM, Benjamin Peterson <benjamin at python.org>wrote: > Guido van Rossum <guido at ...> writes: > > > > > > > > > On Fri, Mar 28, 2014 at 10:06 AM, Benjamin Peterson > <benjamin at python.org> wrote: > > Antoine Pitrou <solipsis <at> ...> writes: > > > I don't like it, but I also don't think it's enough of a nuisance to be > > > deprecated. > > > (the indexing behaviour of bytes objects is far more annoying) > > I agree. We would add a method that iterates over single bytes as bytes > > objects instead of integers. > > > > > > > > > > Now that's making sense. > > Is this sarcasm about my poor writing skills ("would" should be "could") or > actual support for the idea? > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/6c33b70b/attachment.html> From benjamin at python.org Fri Mar 28 18:54:04 2014 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 28 Mar 2014 17:54:04 +0000 (UTC) Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing the Python 3 bytes constructor) References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <loom.20140328T182140-300@post.gmane.org> <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com> Message-ID: <loom.20140328T185247-589@post.gmane.org> Guido van Rossum <guido at ...> writes: > > > Support. Excellent. I think the means the only thing left to do is bikeshed the name. iterbytes() maybe? That's what it's called in six. From ryan at ryanhiebert.com Fri Mar 28 18:52:37 2014 From: ryan at ryanhiebert.com (Ryan Hiebert) Date: Fri, 28 Mar 2014 12:52:37 -0500 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328184241.04693b7a@fsol> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <CAN8d9g=xLxHiSXiBzSH_BSK-QHj-Gf62JH-7oisn-kPemXXgcg@mail.gmail.com> <20140328184241.04693b7a@fsol> Message-ID: <CABpHFHRcq7k8uApDcU+VFJjL8mpgOPVBNvz2SK6FAcBy9293hQ@mail.gmail.com> On Fri, Mar 28, 2014 at 12:42 PM, Antoine Pitrou <solipsis at pitrou.net>wrote: > On Fri, 28 Mar 2014 18:31:58 +0100 > "Philipp A." <flying-sheep at web.de> wrote: > > even if some people don't like it, the iterating over bytes objects makes > > perfect sense. > > > > in every statically typed language, a byte is most commonly repesented as > > an integer in the range [0,255]. > > > > the annoyance has two sources: > > > > 1. > > > > python 2 byte strings (aka "native strings") behaved like that (and > > unicode strings behave like that because python has no char type) > > 2. > > > > byte strings are represented like strings, including ascii-compatible > > parts as if they were ASCII text. > > The actual source isn't merely cultural. It's that most of the time, > bytes objects are used as containers of arbitrary binary data, not > as arrays of integers (on which you would do element-wise arithmetic > calculations, for instance). > > So, even if the current behaviour makes sense, it's not optimal for the > common uses of bytes objects. I don't see bytes as integers, but as representations of integers, that typically come in a stream or array. Using an integer to represent that byte sometimes makes sense, but sometimes you wish to refer to the representation of the integer (the byte), not the integer itself. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140328/d2db8b2e/attachment.html> From random832 at fastmail.us Fri Mar 28 19:31:01 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 28 Mar 2014 14:31:01 -0400 Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing the Python 3 bytes constructor) In-Reply-To: <loom.20140328T185247-589@post.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <loom.20140328T182140-300@post.gmane.org> <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com> <loom.20140328T185247-589@post.gmane.org> Message-ID: <1396031461.31107.100141201.4542F733@webmail.messagingengine.com> On Fri, Mar 28, 2014, at 13:54, Benjamin Peterson wrote: > Guido van Rossum <guido at ...> writes: > > > Support. > > Excellent. I think the means the only thing left to do is bikeshed the > name. > iterbytes() maybe? That's what it's called in six. Should this be purely an iteration function, or something that can also index and slice [into objects that can subsequently be iterated or indexed]? What does six do about that? From random832 at fastmail.us Fri Mar 28 19:38:06 2014 From: random832 at fastmail.us (random832 at fastmail.us) Date: Fri, 28 Mar 2014 14:38:06 -0400 Subject: [Python-ideas] iterating over bytes as byte strings (was Fixing the Python 3 bytes constructor) In-Reply-To: <loom.20140328T185247-589@post.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> <CAP7+vJ+DT_i0_pGDRGw10ATOvPoeRzsfULONwxQt8t0PJZZo0g@mail.gmail.com> <loom.20140328T182140-300@post.gmane.org> <CAP7+vJKuRv-smYdRj2WTn+ry8Pe02d_p7Z3tE3BwEHPxeEFSrg@mail.gmail.com> <loom.20140328T185247-589@post.gmane.org> Message-ID: <1396031886.356.100142697.5C37CC66@webmail.messagingengine.com> On Fri, Mar 28, 2014, at 13:54, Benjamin Peterson wrote: > Excellent. I think the means the only thing left to do is bikeshed the > name. > iterbytes() maybe? That's what it's called in six. The thing called that in six is actually exactly the opposite of this. It doesn't seem to have functions for this. From storchaka at gmail.com Fri Mar 28 22:09:15 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 28 Mar 2014 23:09:15 +0200 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <lh4oc2$ono$1@ger.gmane.org> 28.03.14 12:27, Nick Coghlan ???????(??): > The other consequence of this is that's currently no neat way to > convert the integers produced by various bytes APIs back to a length > one bytes object - we have no binary equivalent of "chr" to convert an > integer in the range 0-255 inclusive to its bytes counterpart. The > acceptance of PEP 361 means we'll get another option (b"%c".__mod__) > but that's hardly what anyone would call obvious. AFAIK currently the fastest way is: packB = struct.Struct('B').pack But lambda x: bytes([x]) looks most obvious. > However, during a conversation today, a possible solution occurred to > me: a "bytes.chr" class method, that served as an alternate > constructor. That idea results in the following 3 part proposal: > > 1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP > 361 defined "b'%c' % x" -0. This is not very needed method and is not worth the language complication. bytes([x]) works pretty good in most cases. > 2. Add "bytearray.allnull" and "bytes.allnull" such that > "bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int > handling -1. b'\0' * x does this. > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > not only ambiguous, but actually a genuine bug magnet (it's way too > easy to accidentally pass a large integer and try to allocate a > ridiculously large bytes object) +0. From storchaka at gmail.com Fri Mar 28 22:15:35 2014 From: storchaka at gmail.com (Serhiy Storchaka) Date: Fri, 28 Mar 2014 23:15:35 +0200 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <loom.20140328T180549-288@post.gmane.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328122251.1933e4e4@fsol> <loom.20140328T180549-288@post.gmane.org> Message-ID: <lh4ons$tdr$1@ger.gmane.org> 28.03.14 19:06, Benjamin Peterson ???????(??): > Antoine Pitrou <solipsis at ...> writes: >> I don't like it, but I also don't think it's enough of a nuisance to be >> deprecated. >> (the indexing behaviour of bytes objects is far more annoying) > > I agree. We would add a method that iterates over single bytes as bytes > objects instead of integers. More general function can be used instead. def chunks(seq, size): for i in range(0, len(seq), size): yield seq[i: i + size] >>> list(chunks(b'abc', 1)) [b'a', b'b', b'c'] From cs at zip.com.au Fri Mar 28 22:59:26 2014 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 29 Mar 2014 08:59:26 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <53356810.5060602@canterbury.ac.nz> References: <53356810.5060602@canterbury.ac.nz> Message-ID: <20140328215926.GA60776@cskk.homeip.net> On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Nick Coghlan wrote: > >If I could consistently remember whether to include the "e" or not, > >"zeroes" would be my preference as well, but almost every time I go to > >type it, I have to pause... > > Whichever you choose, please make sure it's the same as > whatever numpy uses! Please get this right. Zeroes. Two 'e's. Otherwise I will feel pain every time I encounter zeros. Looks Greek. -- Cameron Simpson <cs at zip.com.au> Yes, we plot no less than the destruction of the West. Just the other day a friend and I came up with the most pernicious academic scheme to date for toppling the West: He will kneel behind the West on all fours. I will push it backwards over him. - Michael Berube From ethan at stoneleaf.us Fri Mar 28 23:33:55 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 28 Mar 2014 15:33:55 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328111347.20495c04@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <533587A5.9010701@stoneleaf.us> <20140328111347.20495c04@anarchist.wooz.org> Message-ID: <5335F8D3.5090101@stoneleaf.us> On 03/28/2014 08:13 AM, Barry Warsaw wrote: > On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote: > >> You mean like http://bugs.python.org/issue20895 ? :) > > Step *away* from the time machine. > Time machine? What time machine? *whistles innocently* -- ~Ethan~ From ncoghlan at gmail.com Sat Mar 29 00:39:33 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 29 Mar 2014 09:39:33 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328111347.20495c04@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <533587A5.9010701@stoneleaf.us> <20140328111347.20495c04@anarchist.wooz.org> Message-ID: <CADiSq7cXP6pw48Cyxbtmg8NiSwUHPxGKnmtKk_eh6dEMh4jYHg@mail.gmail.com> On 29 March 2014 01:13, Barry Warsaw <barry at python.org> wrote: > On Mar 28, 2014, at 07:31 AM, Ethan Furman wrote: > >>You mean like http://bugs.python.org/issue20895 ? :) > > Step *away* from the time machine. Hah, I *thought* there was already an issue for this, but my search-fu failed me :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg at krypto.org Sat Mar 29 08:52:42 2014 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 29 Mar 2014 00:52:42 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328215926.GA60776@cskk.homeip.net> References: <53356810.5060602@canterbury.ac.nz> <20140328215926.GA60776@cskk.homeip.net> Message-ID: <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com> On Fri, Mar 28, 2014 at 2:59 PM, Cameron Simpson <cs at zip.com.au> wrote: > On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > > Nick Coghlan wrote: > > >If I could consistently remember whether to include the "e" or not, > > >"zeroes" would be my preference as well, but almost every time I go to > > >type it, I have to pause... > > > > Whichever you choose, please make sure it's the same as > > whatever numpy uses! > > Please get this right. Zeroes. Two 'e's. > > Otherwise I will feel pain every time I encounter zeros. Looks Greek. > zeros is way more popular than zeroes: https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/965cb268/attachment.html> From ncoghlan at gmail.com Sat Mar 29 09:37:14 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 29 Mar 2014 18:37:14 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com> References: <53356810.5060602@canterbury.ac.nz> <20140328215926.GA60776@cskk.homeip.net> <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com> Message-ID: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com> On 29 March 2014 17:52, Gregory P. Smith <greg at krypto.org> wrote: > On Fri, Mar 28, 2014 at 2:59 PM, Cameron Simpson <cs at zip.com.au> wrote: >> On 29Mar2014 01:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: >> > Nick Coghlan wrote: >> > >If I could consistently remember whether to include the "e" or not, >> > >"zeroes" would be my preference as well, but almost every time I go to >> > >type it, I have to pause... >> > >> > Whichever you choose, please make sure it's the same as >> > whatever numpy uses! >> >> Please get this right. Zeroes. Two 'e's. >> >> Otherwise I will feel pain every time I encounter zeros. Looks Greek. > > zeros is way more popular than zeroes: > > https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0 This subthread is confirming my original instinct to just avoid the problem entirely because *either* choice is inevitably confusing for a non-trivial subset of users :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ron3200 at gmail.com Sat Mar 29 12:59:22 2014 From: ron3200 at gmail.com (Ron Adam) Date: Sat, 29 Mar 2014 07:59:22 -0400 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com> References: <53356810.5060602@canterbury.ac.nz> <20140328215926.GA60776@cskk.homeip.net> <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com> <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com> Message-ID: <lh6cib$1h7$1@ger.gmane.org> On 03/29/2014 04:37 AM, Nick Coghlan wrote: >>> >>Please get this right. Zeroes. Two 'e's. >>> >> >>> >>Otherwise I will feel pain every time I encounter zeros. Looks Greek. >> > >> >zeros is way more popular than zeroes: >> > >> >https://books.google.com/ngrams/graph?content=zeroes%2Czeros&year_start=1900&year_end=2013&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Czeroes%3B%2Cc0%3B.t1%3B%2Czeros%3B%2Cc0 > This subthread is confirming my original instinct to just avoid the > problem entirely because*either* choice is inevitably confusing for a > non-trivial subset of users:) Yes, and we don't have nones for lists and tuples, or spaces for strings. ;-) Ron From bcannon at gmail.com Sat Mar 29 14:49:14 2014 From: bcannon at gmail.com (Brett Cannon) Date: Sat, 29 Mar 2014 13:49:14 +0000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com> On Fri Mar 28 2014 at 10:29:36 AM, Barry Warsaw <barry at python.org> wrote: > Great idea Nick. If I may dip my brush in some paint buckets. > > On Mar 28, 2014, at 08:27 PM, Nick Coghlan wrote: > > >However, during a conversation today, a possible solution occurred to > >me: a "bytes.chr" class method, that served as an alternate > >constructor. That idea results in the following 3 part proposal: > > > >1. Add "bytes.chr" such that "bytes.chr(x)" is equivalent to the PEP > >361 defined "b'%c' % x" > > I agree with Steven that bytes.byte() is a better spelling. > If we are going to add a classmethod, then yes, otherwise just stick with byte([x]). > > >2. Add "bytearray.allnull" and "bytes.allnull" such that > >"bytearray.allnull(x)" is equivalent to the current "bytearray(x)" int > >handling > > I like bytearray.fill() for this. The first argument would be the fill > count, > but it could take an optional second argument for the byte value to fill it > with, which would of course default to zero. E.g. > > >>> bytearray.fill(5) > bytearray(b'\x00\x00\x00\x00\x00') > >>> bytearray.fill(5, 97) > bytearray(b'aaaaa') > I agree that if we are going to keep a classmethod to do this it should be like str.zfill(). Otherwise just rely on the multiply operator. > > >3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > >not only ambiguous, but actually a genuine bug magnet (it's way too > >easy to accidentally pass a large integer and try to allocate a > >ridiculously large bytes object) > > +1 > +1 > > >Does the above proposal sound like a reasonable suggestion for > improvement in > >3.5? > > Very much so. > > >Does this hit PEP territory, since it's changing the signature and API > >of a builtin? > > I don't much care either way. A PEP is not *wrong* (especially if we all > start painting), but I think a tracker issue would be fine too. > PEP wouldn't hurt since it's a core type, but I'm not going to scream for it if Guido just says "fine" at the language summit or something (should we discuss this there?). Overall I think I'm +1 on the "simplify the API" by dropping the constructor issue and relying on the multiply operator instead of coming up with a new method. Same with bytes([x]) if you need to convert a single int. I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just leaves the inability to index into bytes and get back a single as the biggest annoyance in the bytes API. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/ba76b561/attachment.html> From richard.prosser at mail.com Sat Mar 29 16:29:53 2014 From: richard.prosser at mail.com (Richard Prosser) Date: Sat, 29 Mar 2014 15:29:53 +0000 Subject: [Python-ideas] Browser for mailing lists Message-ID: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> I wanted to search this list (and other Python ones) but it is practically impossible to do so. So why not add a tool like http://markmail.org/search/?q=python to https://mail.python.org/mailman/listinfo/python-ideas or the like? I got this from http://openmeetings.apache.org/mail-lists.html Cheers, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/eb6cba7d/attachment.html> From richard.prosser at mail.com Sat Mar 29 16:36:06 2014 From: richard.prosser at mail.com (Richard Prosser) Date: Sat, 29 Mar 2014 15:36:06 +0000 Subject: [Python-ideas] Nested dictionaries Message-ID: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> Not the first time I found myself recently needing to create "multi-dimensional" dictionaries, so that I could write something like: my_dict = ['a']['b']['b']['a'] = 'fred' and once again I wonder why Python does not support that idiom directly, as it must be a fairly common requirement. I know that there are good solutions like the one at http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by using defaultdict but I feel that this really should be part of the language. So, should I propose a PEP? Thanks ... Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/4e4b24a4/attachment.html> From rosuav at gmail.com Sat Mar 29 16:46:30 2014 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 30 Mar 2014 02:46:30 +1100 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> Message-ID: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com> On Sun, Mar 30, 2014 at 2:36 AM, Richard Prosser <richard.prosser at mail.com> wrote: > Not the first time I found myself recently needing to create > "multi-dimensional" dictionaries, so that I could write something like: > > my_dict = ['a']['b']['b']['a'] = 'fred' > > and once again I wonder why Python does not support that idiom directly, as > it must be a fairly common requirement. I know that there are good solutions > like the one at > http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by > using defaultdict but I feel that this really should be part of the > language. The NestedDict at the end of that blog post looks like a nice clean solution; and it's so short that it probably doesn't need to be in the standard library. But maybe that would make a handy example, if there's a place in the docs where you'd go looking for that sort of thing. Alternatively, you may find it simpler and easier to use a non-nested dictionary. I store persistent data for my MUD client in a structure like this: persist["window/downarr"] = blah blah Of course, this assumes your keys are simple strings, and it's best for literals; you can't easily iterate over "everything under window/", for instance. But if that covers your requirements, it's probably going to be easier to read and comprehend; and it'd probably be faster, too, although that shouldn't be a primary consideration. ChrisA From richard.prosser at mail.com Sat Mar 29 16:51:20 2014 From: richard.prosser at mail.com (Richard Prosser) Date: Sat, 29 Mar 2014 15:51:20 +0000 Subject: [Python-ideas] "Read Only" namespaces Message-ID: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com> This is probably far-fetched where Python is concerned but I still think that it could be useful. It is based on an idea that I had 36 years ago! Quite simply, the principle is to allow changes to variables within a function body (for example) but not outside of it, rather like the Functional Programming paradigm. That of course excludes the use of global variables and requires functions to return whatever is needed to the outer scope. In my experience, globals are only needed to implement persistent (inter-call) states anyway, so assuming that such a feature is available in the language I believe that the "Read Only" proposal is viable. This is highly unlikely to be implemented in Python any time soon (even with an Interpreter/compiler option) and would imply some performance overhead but I still think that it is a nice idea. Any comments? Would a "Persistent Variables" PEP be interesting in its own right? Thanks ... Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/e045523b/attachment.html> From flying-sheep at web.de Sat Mar 29 17:43:31 2014 From: flying-sheep at web.de (Philipp A.) Date: Sat, 29 Mar 2014 17:43:31 +0100 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com> Message-ID: <CAN8d9g=PQE4xG4NjtWi_eGcbSsK9wQ9+2HfiPSpEreDEr=9Vgg@mail.gmail.com> 2014-03-29 16:46 GMT+01:00 Chris Angelico <rosuav at gmail.com>: Alternatively, you may find it simpler and easier to use a non-nested > dictionary. I store persistent data for my MUD client in a structure > like this: > > persist["window/downarr"] = blah blah > > Of course, this assumes your keys are simple strings, and it's best > for literals; you can't easily iterate over "everything under > window/", for instance. But if that covers your requirements, it's > probably going to be easier to read and comprehend; and it'd probably > be faster, too, although that shouldn't be a primary consideration. > > ChrisA > another possibility: use tuples as keys. persist['window', 'downarr'] = blah blah this works with all hashable types. PS: to elaborate for non-gurus: tuples in python are created by the comma, not the braces, so the above is another way to write persist[ ('window', 'downarr') ] -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/39b1ba4e/attachment.html> From steve at pearwood.info Sat Mar 29 18:29:12 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 30 Mar 2014 04:29:12 +1100 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> Message-ID: <20140329172912.GL16526@ando> On Sat, Mar 29, 2014 at 03:36:06PM +0000, Richard Prosser wrote: > Not the first time I found myself recently needing to create > "multi-dimensional" dictionaries, so that I could write something like: > > my_dict = ['a']['b']['b']['a'] = 'fred' Could you please explain what that is supposed to mean? I can't tell if that's supposed to be new syntax, or a typo. Did you mean this instead? mydict = {'a': {'b': {'b': {}}}} my_dict['a']['b']['b']['a'] = 'fred' > and once again I wonder why Python does not support that idiom directly, It does. There are no special rules to learn, no special syntax to memorise, just the same dict syntax as you already know. You create a dict with {key: value}, and if the value itself is a dict, you just repeat the process with value --> {key: value}, as deeply as you need. Not every special case needs special syntax. > as it must be a fairly common requirement. I doubt it. As the Zen of Python states, "Flat is better than nested". I think most people would try to avoid such deeply nested dicts if they can. I know I've never needed them. > I know that there are good > solutions like the one at > http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by > using defaultdict but I feel that this really should be part of the > language. I don't think something like this should be encouraged, and besides it's a four-line solution: py> class MyDict(dict): ... def __missing__(self, key): ... t = self[key] = MyDict() ... return t ... py> x = MyDict() py> x[1]['a'][None][True][23.0] = "Surprise!" py> x {1: {'a': {None: {True: {23.0: 'Surprise!'}}}}} > So, should I propose a PEP? No. Not every four-line utility class needs to be a built-in. -- Steven From ethan at stoneleaf.us Sat Mar 29 17:49:42 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 29 Mar 2014 09:49:42 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> Message-ID: <5336F9A6.2030006@stoneleaf.us> On 03/28/2014 03:27 AM, Nick Coghlan wrote: > > 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as > not only ambiguous, but actually a genuine bug magnet (it's way too > easy to accidentally pass a large integer and try to allocate a > ridiculously large bytes object) Why do 'bytes' and 'bytearray' have to have exactly the same API? Having 'bytearray(10)' return a zero filled array is fine, as bytearrays are mutable; it just doesn't make sense for 'bytes' which are immutable. -- ~Ethan~ From ericsnowcurrently at gmail.com Sat Mar 29 18:46:37 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 29 Mar 2014 11:46:37 -0600 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> Message-ID: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com> On Sat, Mar 29, 2014 at 9:29 AM, Richard Prosser <richard.prosser at mail.com> wrote: > I wanted to search this list (and other Python ones) but it is practically > impossible to do so. Google lets you search a specific URL (and its subpages): site:https://mail.python.org/mailman/listinfo/python-ideas ... That's been the most effective tool I've used. -eric From steve at pearwood.info Sat Mar 29 18:46:44 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 30 Mar 2014 04:46:44 +1100 Subject: [Python-ideas] "Read Only" namespaces In-Reply-To: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com> References: <CAJvrtnuD5_J3gP5kimmV=0SNjLCcS825dVfSpxMNG2p9tbdg2Q@mail.gmail.com> Message-ID: <20140329174644.GM16526@ando> On Sat, Mar 29, 2014 at 03:51:20PM +0000, Richard Prosser wrote: > This is probably far-fetched where Python is concerned but I still think > that it could be useful. It is based on an idea that I had 36 years ago! > > Quite simply, the principle is to allow changes to variables within a > function body (for example) but not outside of it, rather like the > Functional Programming paradigm. I'm afraid your description leaves me completely in the dark. (1) What do you mean, "changes to variables? Do you mean rebinding, or mutation, or both? e.g. "x = 1" is a binding; "x.append(1)" is a mutation. (2) What is the scope of these variables? If they are local variables, by definition they aren't visible outside of the function body, so they cannot be rebound or mutated by anything outside. If they're global variables, then by definition they're supposed to be available outside of any function for rebinding and/or mutation. If they're meant to be *constants*, not variables, then you should call them constants. (3) Under what circumstances would you use this feature? (4) What you describe doesn't sound like anything from Functional Programming that I know of. What is the connection that I am missing? (5) I'm not seeing the connection between your description and "read only namespaces", the subject of your post. Can you explain? (6) Later you describe this as "Persistent Variables". Persistent across what? [...] > Any comments? Would a "Persistent Variables" PEP be interesting in its own > right? PEP stands for Python Enhancement Proposal. From your own description, you don't seem to be proposing this as an enhancement to the language, more like just an interesting idea for discussion, so a PEP would be completely inappropriate. Do you have a blog? Perhaps this might be better started as a blog post or three. You can iron out any kinks in the idea and then reconsider whether or not it makes a serious proposal. -- Steven From greg at krypto.org Sat Mar 29 19:03:24 2014 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 29 Mar 2014 11:03:24 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <5336F9A6.2030006@stoneleaf.us> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> Message-ID: <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> On Sat, Mar 29, 2014 at 9:49 AM, Ethan Furman <ethan at stoneleaf.us> wrote: > On 03/28/2014 03:27 AM, Nick Coghlan wrote: > >> >> 3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as >> not only ambiguous, but actually a genuine bug magnet (it's way too >> easy to accidentally pass a large integer and try to allocate a >> ridiculously large bytes object) >> > > Why do 'bytes' and 'bytearray' have to have exactly the same API? Having > 'bytearray(10)' return a zero filled array is fine, as bytearrays are > mutable; it just doesn't make sense for 'bytes' which are immutable. > > I agree they didn't need to be. But that's already happened. Consistency *is* convenient when reading and re-factoring code. Regardless, the real issue is that it is easy to inadvertently pass an integer to the bytes() constructor [and thus the bytearray() constructor] when you really intended to pass a single element sequence. +1 Deprecating the single integer argument to the constructor in 3.5 (DeprecationWarning) to be considered for removal in 3.6 or 3.7 sounds like a good idea. +1 At the same time, adding class methods: bytes*.byte*(97) and bytearray *.byte*(0x65) sound great. Those are *always explicit* about the programmer's intent rather than being an overloaded constructor that does different things based on the type passed in which is more prone to bugs. +1 that *bytearray*() does need a constructor that pre-allocates a bunch of empty (zero) space as that is a common need for a mutable type. That should be .*zfill*(n) for consistency. Filling with other values is way less common and doesn't deserve a .fill(n, value) method with potentially ambiguous parameters (which one is the count and which one is the value again? that'll be a continual question just as it is for C's memset and similar functions). -0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n) class methods for the *bytes*() type. That is fine written as bytes.byte(0) * n as it is expected to be an uncommon operation. But if you want to add it for consistency, fine by me, change the sign of my preference. :) I don't think this is worthy of a PEP but won't object if you go that route. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/3aa42880/attachment.html> From python at 2sn.net Sat Mar 29 19:31:33 2014 From: python at 2sn.net (Alexander Heger) Date: Sun, 30 Mar 2014 05:31:33 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328102817.76d2b7f9@anarchist.wooz.org> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> Message-ID: <CAN3CYHy006s7nLg2kKmoBJEdgo3b1jmz5iTneSAeVPxcrVad-g@mail.gmail.com> >>>> bytearray.fill(5, 97) why would you not want to write this as bytearray(5).fill(97) ? >>3. Deprecate the current "bytes(x)" and "bytearray(x)" int handling as >>not only ambiguous, but actually a genuine bug magnet (it's way too >>easy to accidentally pass a large integer and try to allocate a >>ridiculously large bytes object) I think these are just fine. From ericsnowcurrently at gmail.com Sat Mar 29 19:39:31 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 29 Mar 2014 12:39:31 -0600 Subject: [Python-ideas] Persisting private function state between calls (was: "Read Only" namespaces) Message-ID: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com> On Sat, Mar 29, 2014 at 9:51 AM, Richard Prosser <richard.prosser at mail.com> wrote: > Quite simply, the principle is to allow changes to variables within a > function body (for example) but not outside of it, rather like the > Functional Programming paradigm. That of course excludes the use of global > variables and requires functions to return whatever is needed to the outer > scope. In my experience, globals are only needed to implement persistent > (inter-call) states anyway, so assuming that such a feature is available in > the language I believe that the "Read Only" proposal is viable. To restate, you want a way to have some variables in your function body that persist between calls without using global variables. There are already several ways to do this, though none is particularly obvious. Let's use a simple example using a global variable followed by the alternative solutions. current = 0 def adder(x): global current current += x return current 1. Still use a global variable, but make it "private" (just use the convention of a leading underscore on the name): _current = 0 def adder(x): global _current _current += x return _current 2. Use the "default argument hack": def adder(x, _ns={'current': 0}): _ns['current'] += x return _ns['current'] or class IntWrapper: def __init__(self, value=0): self.value = value def adder(x, _current=IntWrapper()): _current.value += x return _current.value 3. Use a closure (making use of the "nonlocal" keyword): def get_adder(start=0): current = start # Done for clarity. def adder(x): nonlocal current current += x return current return adder adder = get_adder() Each of the four approaches has its pros and cons. The closure approach is probably the clearest if you really want to avoid using a global. If there were agreement on a best approach, the least we could do it make it clearer in the docs, assuming you looked and didn't find anything helpful in the obvious places. :) A more explicit mechanism could be nice, but would have to pass a pretty high bar for inclusion into the language. Here are some approaches: * A new keyword, e.g. "persistent", to accompany global and nonlocal. * A special decorator that identifies persistent local variables in the function. So that you're aware, this idea came up a few years ago and involved a long discussion that eventually lost steam (don't remember why) [1][2][3]. There are also a couple existing PEPs [4][5] that would be applicable (my favorite is the given statement). -eric [1] https://mail.python.org/pipermail/python-ideas/2011-September/011805.html [2] https://mail.python.org/pipermail/python-ideas/2011-June/010479.html [3] https://mail.python.org/pipermail/python-ideas/2011-September/011750.html [4] "statement local namespaces" (given statement) http://www.python.org/dev/peps/pep-3150/ [5] "General purpose decorator clause" http://www.python.org/dev/peps/pep-0403/ From ericsnowcurrently at gmail.com Sat Mar 29 19:41:12 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 29 Mar 2014 12:41:12 -0600 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com> Message-ID: <CALFfu7A2AoqbX4gYEF7Rjhai--PfBt1FGwy3i-=kjhfg02-hpQ@mail.gmail.com> On Sat, Mar 29, 2014 at 11:46 AM, Eric Snow <ericsnowcurrently at gmail.com> wrote: > site:https://mail.python.org/mailman/listinfo/python-ideas ... Correction: site:https://mail.python.org/pipermail/python-ideas/ ... -eric From masklinn at masklinn.net Sat Mar 29 19:50:29 2014 From: masklinn at masklinn.net (Masklinn) Date: Sat, 29 Mar 2014 19:50:29 +0100 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <20140329172912.GL16526@ando> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> <20140329172912.GL16526@ando> Message-ID: <CF52F35F-4BCF-4B08-B5D4-1B41E1FD496E@masklinn.net> On 2014-03-29, at 18:29 , Steven D'Aprano <steve at pearwood.info> wrote: >> I know that there are good >> solutions like the one at >> http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html or by >> using defaultdict but I feel that this really should be part of the >> language. > > I don't think something like this should be encouraged, and besides it's > a four-line solution: > > py> class MyDict(dict): > ... def __missing__(self, key): > ... t = self[key] = MyDict() > ... return t > ... > py> x = MyDict() > py> x[1]['a'][None][True][23.0] = "Surprise!" > py> x > {1: {'a': {None: {True: {23.0: 'Surprise!'}}}}} > In most case, a single line probably suffices: MyDict = lambda: collections.defaultdict(MyDict) From steve at pearwood.info Sat Mar 29 19:58:25 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 30 Mar 2014 05:58:25 +1100 Subject: [Python-ideas] Persisting private function state between calls (was: "Read Only" namespaces) In-Reply-To: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com> References: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com> Message-ID: <20140329185824.GN16526@ando> On Sat, Mar 29, 2014 at 12:39:31PM -0600, Eric Snow wrote: > On Sat, Mar 29, 2014 at 9:51 AM, Richard Prosser > <richard.prosser at mail.com> wrote: > > Quite simply, the principle is to allow changes to variables within a > > function body (for example) but not outside of it, rather like the > > Functional Programming paradigm. That of course excludes the use of global > > variables and requires functions to return whatever is needed to the outer > > scope. In my experience, globals are only needed to implement persistent > > (inter-call) states anyway, so assuming that such a feature is available in > > the language I believe that the "Read Only" proposal is viable. > > To restate, you want a way to have some variables in your function > body that persist between calls without using global variables. Ah-ha! That makes sense now. Thank you! > There are already several ways to do this, though none is particularly > obvious. I disagree -- I think your example of an adder() function with a global is extremely obvious. It seems to be the usual solution that most beginners go for. It's not a *good* solution, but it is simple and obvious. A much better solution if you need persistent state is to encapsulate the function and state into an object: class Adder: def __init__(self): self.current = 0 def add(self, x): self.current += x return self.current Then you can have as many Adder instances as you want, without them all sharing state. adder1 = Adder().add adder2 = Adder().add [...] > A more explicit mechanism could be nice, but would have to pass a > pretty high bar for inclusion into the language. Here are some > approaches: > > * A new keyword, e.g. "persistent", to accompany global and nonlocal. > * A special decorator that identifies persistent local variables in > the function. -1 to both of those. Not every idiom needs a keyword. -- Steven From ericsnowcurrently at gmail.com Sat Mar 29 20:14:22 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 29 Mar 2014 13:14:22 -0600 Subject: [Python-ideas] Persisting private function state between calls (was: "Read Only" namespaces) In-Reply-To: <20140329185824.GN16526@ando> References: <CALFfu7Az7FXDKyHBtQuB2C+=AiwAO6B+jwzXxxynosDdTmvwgQ@mail.gmail.com> <20140329185824.GN16526@ando> Message-ID: <CALFfu7Ddk0Yi5R7-gp1Txa5dSu=UcmJ3+V7mP86_7esTti3fUQ@mail.gmail.com> On Sat, Mar 29, 2014 at 12:58 PM, Steven D'Aprano <steve at pearwood.info> wrote: > On Sat, Mar 29, 2014 at 12:39:31PM -0600, Eric Snow wrote: >> There are already several ways to do this, though none is particularly >> obvious. > > I disagree -- I think your example of an adder() function with a global > is extremely obvious. It seems to be the usual solution that most > beginners go for. It's not a *good* solution, but it is simple and > obvious. Agreed. It was the example that mapped most closely to how the OP described his problem. :) [...] >> A more explicit mechanism could be nice, but would have to pass a >> pretty high bar for inclusion into the language. Here are some >> approaches: >> >> * A new keyword, e.g. "persistent", to accompany global and nonlocal. >> * A special decorator that identifies persistent local variables in >> the function. > > -1 to both of those. Not every idiom needs a keyword. Oh, I agree*. I should have made it clear. :) I would advocate instead for a more general solution such as that provided by statement local namespaces (e.g. PEP 3150/403). -eric * I do think it may be worth exploring the idea of special decorators that set code/function object flags that influence special-case behaviors. However, the performance implications of handling such flags at call-time *may* render the idea impractical. From masklinn at masklinn.net Sat Mar 29 21:23:12 2014 From: masklinn at masklinn.net (Masklinn) Date: Sat, 29 Mar 2014 21:23:12 +0100 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAJvrtnuHTW7sO8Kr1xy0kUyLGm3W5zWXZct0jPzZZMZYH9qA4Q@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> <20140329172912.GL16526@ando> <CF52F35F-4BCF-4B08-B5D4-1B41E1FD496E@masklinn.net> <CAJvrtnuHTW7sO8Kr1xy0kUyLGm3W5zWXZct0jPzZZMZYH9qA4Q@mail.gmail.com> Message-ID: <A6CCAE20-621E-494C-9D97-D58AB1D106BE@masklinn.net> On 2014-03-29, at 20:43 , Richard Prosser <richard.prosser at mail.com> wrote: > Maybe so but it is not straightforward and therefore IMO not "Pythonic". I don't believe autovivification to be pythonic in the first place[0]. Python making you roll your own thus feels sensible. > I really don't understand the objections that are being raised. What is wrong with being able to use a simple expression? > > > Richard > PS Thanks for the responses everyone but it would nice if someone agreed with my proposal! [0] Steven already mentioned "flat is better than nested", but "explicit is better than implicit" also applies. To the surprise of no one, using autovivification is idiomatic perl. From ben+python at benfinney.id.au Sat Mar 29 21:54:38 2014 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 30 Mar 2014 07:54:38 +1100 Subject: [Python-ideas] Browser for mailing lists References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> Message-ID: <85d2h4ohdd.fsf@benfinney.id.au> Richard Prosser <richard.prosser at mail.com> writes: > I wanted to search this list (and other Python ones) but it is practically > impossible to do so. The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> has all its messages archived online for web browsing at <URL:http://mail.python.org/pipermail/python-ideas/>. So you can use any web search tool to search it. A good search engine like DuckDuckGo will let you constrain the search to a particular subset of the web: site:http://mail.python.org/pipermail/python-ideas foo bar baz > So why not add a tool like http://markmail.org/search/?q=python to > https://mail.python.org/mailman/listinfo/python-ideas or the like? We have generally-appicable services that index the web and make much better search tools. I don't often find a site's own custom search tool to be sufficiently better to use it, when DuckDuckGo is available. -- \ ?For every complex problem, there is a solution that is simple, | `\ neat, and wrong.? ?Henry L. Mencken | _o__) | Ben Finney From python at 2sn.net Sat Mar 29 21:59:13 2014 From: python at 2sn.net (Alexander Heger) Date: Sun, 30 Mar 2014 07:59:13 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com> Message-ID: <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com> > I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just > leaves the inability to index into bytes and get back a single as the > biggest annoyance in the bytes API. The issue remains that there is no single byte type, but what is returned is bytes type over which, again, you can iterate. This is different from a tuple of integers, when you iterate over that, you get integers not length-one tuples of integers. The issue, I think, originates from the string concept where iteration also return a string not a character - which python does not have. A logical extension would be to have to have character and byte types that are returned when iterating of strings and bytes. This is like taking a giraffe out of a zoo and still calling the result a (small) zoo. Or calling a can of tuna a grocery store. From greg.ewing at canterbury.ac.nz Sat Mar 29 22:06:29 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 30 Mar 2014 10:06:29 +1300 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <20140328215926.GA60776@cskk.homeip.net> References: <53356810.5060602@canterbury.ac.nz> <20140328215926.GA60776@cskk.homeip.net> Message-ID: <533735D5.1000901@canterbury.ac.nz> Cameron Simpson wrote: > Please get this right. Zeroes. Two 'e's. > > Otherwise I will feel pain every time I encounter zeros. Looks Greek. I sympathi{s,z}e, but I feel that having it inconsistent with with numpy would cause more pain overall. Ah, I know -- Python should treat identifiers ending in "-os" and "-oes" as equivalent! That would solve all problems of this kind once and for all. "import oes"-ly y'rs, Greg From ncoghlan at gmail.com Sat Mar 29 22:07:45 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 07:07:45 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> Message-ID: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> On 30 March 2014 04:03, Gregory P. Smith <greg at krypto.org> wrote: > > +1 that bytearray() does need a constructor that pre-allocates a bunch of > empty (zero) space as that is a common need for a mutable type. That should > be .zfill(n) for consistency. Filling with other values is way less common > and doesn't deserve a .fill(n, value) method with potentially ambiguous > parameters (which one is the count and which one is the value again? that'll > be a continual question just as it is for C's memset and similar functions). We can't use .zfill(), as that is already used for the same purposes that it is used for with str and bytes objects (i.e. an ASCII zero-fill). I'm currently leaning towards the more explicit "from_len()" (with the fill value being optional, and defaulting to zero). > -0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n) > class methods for the bytes() type. That is fine written as bytes.byte(0) * > n as it is expected to be an uncommon operation. But if you want to add it > for consistency, fine by me, change the sign of my preference. :) > > I don't think this is worthy of a PEP but won't object if you go that route. I already have a draft PEP written that covers the constructor issue, iteration and adding acceptance of integer inputs to the remaining methods that don't currently handle them. There was some background explanation of the text/binary domain split in the Python 2->3 transition that I wanted Guido's feedback on before posting, but I just realised I can cut that out for now, and then add it back after Guido has had a chance to review it. So I'll tidy that up and get the draft posted later today. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ethan at stoneleaf.us Sat Mar 29 22:09:36 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 29 Mar 2014 14:09:36 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com> <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com> Message-ID: <53373690.1060209@stoneleaf.us> On 03/29/2014 01:59 PM, Alexander Heger wrote: >> >> I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That just >> leaves the inability to index into bytes and get back a single as the >> biggest annoyance in the bytes API. > > The issue remains that there is no single byte type, but what is > returned is bytes type over which, again, you can iterate. This is > different from a tuple of integers, when you iterate over that, you > get integers not length-one tuples of integers. The issue, I think, > originates from the string concept where iteration also return a > string not a character - which python does not have. A logical > extension would be to have to have character and byte types that are > returned when iterating of strings and bytes. This is like taking a > giraffe out of a zoo and still calling the result a (small) zoo. Or > calling a can of tuna a grocery store. Nice examples. The problem is that one *can* say: b'a' and get a 'bytes' object of length one, and then quite naturally one would then try something like b'abc'[0] == b'a' and that will fail, even though it *looks* correct. -- ~Ethan~ From greg.ewing at canterbury.ac.nz Sat Mar 29 22:31:35 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 30 Mar 2014 10:31:35 +1300 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com> References: <53356810.5060602@canterbury.ac.nz> <20140328215926.GA60776@cskk.homeip.net> <CAGE7PNJush+LMBFk+7fvTMxNstT5K9YAj7nhvcKoWKUZA2B8gg@mail.gmail.com> <CADiSq7feeQTM5XAP-S8DU74Y50+xM+KZ51MPn2uysMW3yZ1WHA@mail.gmail.com> Message-ID: <53373BB7.2050208@canterbury.ac.nz> Nick Coghlan wrote: > This subthread is confirming my original instinct to just avoid the > problem entirely because *either* choice is inevitably confusing for a > non-trivial subset of users :) How about just calling it "zero"? There's really no need for it to be plural. After all, we refer to int(0) as just "zero", even though there may be more than one zero bit in its representation. I really don't like "allnull", it just sounds wrong, too ascii-centric. -- Greg From ncoghlan at gmail.com Sat Mar 29 22:34:44 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 07:34:44 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <53373690.1060209@stoneleaf.us> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <20140328102817.76d2b7f9@anarchist.wooz.org> <CAP1=2W5z2JZwpFqvxqWo5LaSvyPTPXkvwJhnj+Nx9VqFaqqrFw@mail.gmail.com> <CAN3CYHwsJGBH8pUhwTqPrt=XS0hM0B4gdeaX5VtAe2Zih6dhcQ@mail.gmail.com> <53373690.1060209@stoneleaf.us> Message-ID: <CADiSq7fD7Ax8B4d=VKKPpiRu2r2Wa5yqk_qW9_fAkg9YxhR0jw@mail.gmail.com> On 30 March 2014 07:09, Ethan Furman <ethan at stoneleaf.us> wrote: > On 03/29/2014 01:59 PM, Alexander Heger wrote: >>> >>> >>> I'm also +1 on Benjamin's idea of the iterate-as-bytes addition. That >>> just >>> leaves the inability to index into bytes and get back a single as the >>> biggest annoyance in the bytes API. >> >> >> The issue remains that there is no single byte type, but what is >> returned is bytes type over which, again, you can iterate. This is >> different from a tuple of integers, when you iterate over that, you >> get integers not length-one tuples of integers. The issue, I think, >> originates from the string concept where iteration also return a >> string not a character - which python does not have. A logical >> extension would be to have to have character and byte types that are >> returned when iterating of strings and bytes. This is like taking a >> giraffe out of a zoo and still calling the result a (small) zoo. Or >> calling a can of tuna a grocery store. > > > Nice examples. The problem is that one *can* say: > > b'a' > > and get a 'bytes' object of length one, and then quite naturally one would > then try something like > > b'abc'[0] == b'a' > > and that will fail, even though it *looks* correct. FWIW, this is part of why Python 3 *didn't* originally have a bytes literal - the affordances of the str-like literal syntax are wrong for a tuple-of-integers type. We only added it back (pre 3.0) because people wanted it for working with wire protocols that contain ASCII segments (and I think that was a reasonable decision, even though it has contributed directly to people several years later still expecting them to behave more like Python 2 strings) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg.ewing at canterbury.ac.nz Sat Mar 29 22:37:43 2014 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 30 Mar 2014 10:37:43 +1300 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> <CAPTjJmoHYkfxzXHdPZ6=MMkHPUWX7uG36fbVr+ngJmhJHSWWYg@mail.gmail.com> Message-ID: <53373D27.2020700@canterbury.ac.nz> Chris Angelico wrote: > Alternatively, you may find it simpler and easier to use a non-nested > dictionary. > > persist["window/downarr"] = blah blah Also keep in mind that you can use tuples as keys: mydict['a', 'b', 'b', 'a'] = fred -- Greg From ncoghlan at gmail.com Sat Mar 29 23:05:45 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 08:05:45 +1000 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <85d2h4ohdd.fsf@benfinney.id.au> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> Message-ID: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote: > Richard Prosser <richard.prosser at mail.com> writes: > > We have generally-appicable services that index the web and make much > better search tools. I don't often find a site's own custom search tool > to be sufficiently better to use it, when DuckDuckGo is available. That said, migrating to Mailman3 + HyperKitty is definitely in the longer term plans for the python.org mailing list infrastructure, and that includes integrated search on the HyperKitty side of things. While Mailman3 has been an ongoing project for quite some time (I believe the remaining blockers mostly relate to handling migrations of existing Mailman 2 installations), the HyperKitty work is mostly being driven by some Fedora folks in order to upgrade Fedora's own infrastructure. You can see the current state of the prototype here: https://lists.stg.fedoraproject.org/archives/ Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Sat Mar 29 23:08:55 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 29 Mar 2014 18:08:55 -0400 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <85d2h4ohdd.fsf@benfinney.id.au> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> Message-ID: <lh7ga5$c77$2@ger.gmane.org> On 3/29/2014 4:54 PM, Ben Finney wrote: > Richard Prosser <richard.prosser at mail.com> writes: > >> I wanted to search this list (and other Python ones) but it is practically >> impossible to do so. > > The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> > has all its messages archived online for web browsing at > <URL:http://mail.python.org/pipermail/python-ideas/>. > > So you can use any web search tool to search it. A good search engine > like DuckDuckGo will let you constrain the search to a particular subset > of the web: > > site:http://mail.python.org/pipermail/python-ideas foo bar baz > >> So why not add a tool like http://markmail.org/search/?q=python to >> https://mail.python.org/mailman/listinfo/python-ideas or the like? > > We have generally-appicable services that index the web and make much > better search tools. I don't often find a site's own custom search tool > to be sufficiently better to use it, when DuckDuckGo is available. This list and many others are also available through, archived at, and searchable at news.gmane.org. -- Terry Jan Reedy From tjreedy at udel.edu Sat Mar 29 23:20:16 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 29 Mar 2014 18:20:16 -0400 Subject: [Python-ideas] Nested dictionaries In-Reply-To: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> References: <CAJvrtns6yWa5HUG1JOuwDyo8cFzKS4v=g+0=xLW3=OQN+0TR6g@mail.gmail.com> Message-ID: <lh7gvf$le5$1@ger.gmane.org> On 3/29/2014 11:36 AM, Richard Prosser wrote: > I know that there are good solutions like the one at > http://ohuiginn.net/mt/2010/07/nested_dictionaries_in_python.html You can file a request on the tracker asking that this simple 4-line example be added to the doc. >>> class NestedDict(dict): ... def __getitem__(self, key): ... if key in self: return self.get(key) ... return self.setdefault(key, NestedDict()) > or by using defaultdict but I feel that this really > should be part of the language. It is. You just have to copy and paste ;-). Actually, I am being serious. The example is an application of dict.setdefault, which *is* in the language. -- Terry Jan Reedy From greg at krypto.org Sat Mar 29 23:30:26 2014 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 29 Mar 2014 15:30:26 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> Message-ID: <CAGE7PNK9a9E3xs8KE6n7z0hihjakRwUabCXAGS+8YT2SowpUvA@mail.gmail.com> On Sat, Mar 29, 2014 at 2:07 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 30 March 2014 04:03, Gregory P. Smith <greg at krypto.org> wrote: > > > > +1 that bytearray() does need a constructor that pre-allocates a bunch of > > empty (zero) space as that is a common need for a mutable type. That > should > > be .zfill(n) for consistency. Filling with other values is way less > common > > and doesn't deserve a .fill(n, value) method with potentially ambiguous > > parameters (which one is the count and which one is the value again? > that'll > > be a continual question just as it is for C's memset and similar > functions). > > We can't use .zfill(), as that is already used for the same purposes > that it is used for with str and bytes objects (i.e. an ASCII > zero-fill). > > I'm currently leaning towards the more explicit "from_len()" (with the > fill value being optional, and defaulting to zero). > makes sense to me. > > > -0 on the idea of a .zeros(n), .zeroes(n), .fill(n, value) or .zfill(n) > > class methods for the bytes() type. That is fine written as > bytes.byte(0) * > > n as it is expected to be an uncommon operation. But if you want to add > it > > for consistency, fine by me, change the sign of my preference. :) > > > > I don't think this is worthy of a PEP but won't object if you go that > route. > > I already have a draft PEP written that covers the constructor issue, > iteration and adding acceptance of integer inputs to the remaining > methods that don't currently handle them. There was some background > explanation of the text/binary domain split in the Python 2->3 > transition that I wanted Guido's feedback on before posting, but I > just realised I can cut that out for now, and then add it back after > Guido has had a chance to review it. > > So I'll tidy that up and get the draft posted later today. > Thanks Nick! > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/6c8a9854/attachment-0001.html> From cs at zip.com.au Sat Mar 29 23:47:16 2014 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 30 Mar 2014 09:47:16 +1100 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <53373BB7.2050208@canterbury.ac.nz> References: <53373BB7.2050208@canterbury.ac.nz> Message-ID: <20140329224716.GA61813@cskk.homeip.net> On 30Mar2014 10:31, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote: > Nick Coghlan wrote: > >This subthread is confirming my original instinct to just avoid the > >problem entirely because *either* choice is inevitably confusing for a > >non-trivial subset of users :) > > How about just calling it "zero"? There's really no need > for it to be plural. > > After all, we refer to int(0) as just "zero", even though > there may be more than one zero bit in its representation. Unfortunately, "zero" reads like a verb. One might zero an array, for example. As a noun it would imply exactly one. It's a good idea though; I guess I'm -0 purely for verb/noun reasons. If numpy already has zeros() with a very similar meaning I would live with the misspelling for the sake of consistency. If the numpy zeros does something different then it would carry no weight with me. Hoping for something more betterer, -- Cameron Simpson <cs at zip.com.au> For those who understand, NO explanation is needed, for those who don't understand, NO explanation will be given! - Davey D <decoster at vnet.ibm.com> From ncoghlan at gmail.com Sun Mar 30 04:17:08 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 12:17:08 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> Message-ID: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote: > I already have a draft PEP written that covers the constructor issue, > iteration and adding acceptance of integer inputs to the remaining > methods that don't currently handle them. There was some background > explanation of the text/binary domain split in the Python 2->3 > transition that I wanted Guido's feedback on before posting, but I > just realised I can cut that out for now, and then add it back after > Guido has had a chance to review it. > > So I'll tidy that up and get the draft posted later today. Guido pointed out most of the stuff I had asked him to look at wasn't actually relevant to the PEP, so I just cut most of it entirely. Suffice to say, after stepping back and reviewing them systematically for the first time in years, I believe the APIs for the core binary data types in Python 3 could do with a little sprucing up :) Web version: http://www.python.org/dev/peps/pep-0467/ ====================================== PEP: 467 Title: Improved API consistency for bytes and bytearray Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan at gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2014-03-30 Python-Version: 3.5 Post-History: 2014-03-30 Abstract ======== During the initial development of the Python 3 language specification, the core ``bytes`` type for arbitrary binary data started as the mutable type that is now referred to as ``bytearray``. Other aspects of operating in the binary domain in Python have also evolved over the course of the Python 3 series. This PEP proposes a number of small adjustments to the APIs of the ``bytes`` and ``bytearray`` types to make their behaviour more internally consistent and to make it easier to operate entirely in the binary domain for use cases that actually involve manipulating binary data directly, rather than converting it to a more structured form with additional modelling semantics (such as ``str``) and then converting back to binary format after processing. Background ========== Over the course of Python 3's evolution, a number of adjustments have been made to the core ``bytes`` and ``bytearray`` types as additional practical experience was gained with using them in code beyond the Python 3 standard library and test suite. However, to date, these changes have been made on a relatively ad hoc tactical basis as specific issues were identified, rather than as part of a systematic review of the APIs of these types. This approach has allowed inconsistencies to creep into the API design as to which input types are accepted by different methods. Additional inconsistencies linger from an earlier pre-release design where there was *no* separate ``bytearray`` type, and instead the core ``bytes`` type was mutable (with no immutable counterpart), as well as from the origins of these types in the text-like behaviour of the Python 2 ``str`` type. This PEP aims to provide the missing systematic review, with the goal of ensuring that wherever feasible (given backwards compatibility constraints) these current inconsistencies are addressed for the Python 3.5 release. Proposals ========= As a "consistency improvement" proposal, this PEP is actually about a number of smaller micro-proposals, each aimed at improving the self-consistency of the binary data model in Python 3. Proposals are motivated by one of three factors: * removing remnants of the original design of ``bytes`` as a mutable type * more consistently accepting length 1 ``bytes`` objects as input where an integer between ``0`` and ``255`` inclusive is expected, and vice-versa * allowing users to easily convert integer output to a length 1 ``bytes`` object Alternate Constructors ---------------------- The ``bytes`` and ``bytearray`` constructors currently accept an integer argument, but interpret it to mean a zero-filled object of the given length. This is a legacy of the original design of ``bytes`` as a mutable type, rather than a particularly intuitive behaviour for users. It has become especially confusing now that other ``bytes`` interfaces treat integers and the corresponding length 1 bytes instances as equivalent input. Compare:: >>> b"\x03" in bytes([1, 2, 3]) True >>> 3 in bytes([1, 2, 3]) True >>> bytes(b"\x03") b'\x03' >>> bytes(3) b'\x00\x00\x00' This PEP proposes that the current handling of integers in the bytes and bytearray constructors by deprecated in Python 3.5 and removed in Python 3.6, being replaced by two more type appropriate alternate constructors provided as class methods. The initial python-ideas thread [ideas-thread1]_ that spawned this PEP was specifically aimed at deprecating this constructor behaviour. For ``bytes``, a ``byte`` constructor is proposed that converts integers (as indicated by ``operator.index``) in the appropriate range to a ``bytes`` object, converts objects that support the buffer API to bytes, and also passes through length 1 byte strings unchanged:: >>> bytes.byte(3) b'\x03' >>> bytes.byte(bytearray(bytes([3]))) b'\x03' >>> bytes.byte(memoryview(bytes([3]))) b'\x03' >>> bytes.byte(bytes([3])) b'\x03' >>> bytes.byte(512) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: bytes must be in range(0, 256) >>> bytes.byte(b"ab") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: bytes.byte() expected a byte, but buffer of length 2 found One specific use case for this alternate constructor is to easily convert the result of indexing operations on ``bytes`` and other binary sequences from an integer to a ``bytes`` object. The documentation for this API should note that its counterpart for the reverse conversion is ``ord()``. For ``bytearray``, a ``from_len`` constructor is proposed that preallocates the buffer filled with a particular value (default to ``0``) as a direct replacement for the current constructor behaviour, rather than having to use sequence repetition to achieve the same effect in a less intuitive way:: >>> bytearray.from_len(3) bytearray(b'\x00\x00\x00') >>> bytearray.from_len(3, 6) bytearray(b'\x06\x06\x06') This part of the proposal was covered by an existing issue [empty-buffer-issue]_ and a variety of names have been proposed (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The specific name currently proposed was chosen by analogy with ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely explicit that it is an alternate constructor rather than an in-place mutation, as well as how it differs from the standard constructor. Open questions ^^^^^^^^^^^^^^ * Should ``bytearray.byte()`` also be added? Or is ``bytearray(bytes.byte(x))`` sufficient for that case? * Should ``bytes.from_len()`` also be added? Or is sequence repetition sufficient for that case? * Should ``bytearray.from_len()`` use a different name? * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary sequences with more than one element? The ``TypeError`` currently proposed is copied (with slightly improved wording) from the behaviour of ``ord()`` with sequences containing more than one code point, while ``ValueError`` would be more consistent with the existing handling of out-of-range integer values. * ``bytes.byte()`` is defined above as accepting length 1 binary sequences as individual bytes, but this is currently inconsistent with the main ``bytes`` constructor:: >>> bytes([b"a", b"b", b"c"]) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'bytes' object cannot be interpreted as an integer Should the ``bytes`` constructor be changed to accept iterables of length 1 bytes objects in addition to iterables of integers? If so, should it allow a mixture of the two in a single iterable? Iteration --------- Iteration over ``bytes`` objects and other binary sequences produces integers. Rather than proposing a new method that would need to be added not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially to third party types as well, this PEP proposes that iteration to produce length 1 ``bytes`` objects instead be handled by combining ``map`` with the new ``bytes.byte()`` alternate constructor proposed above:: for x in map(bytes.byte, data): # x is a length 1 ``bytes`` object, rather than an integer # This works with *any* container of integers in the range # 0 to 255 inclusive Consistent support for different input types -------------------------------------------- In Python 3.3, the binary search operations (``in``, ``count()``, ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to accept integers in the range 0 to 255 (inclusive) as their first argument (in addition to the existing support for binary sequences). This PEP proposes extending that behaviour of accepting integers as being equivalent to the corresponding length 1 binary sequence to several other ``bytes`` and ``bytearray`` methods that currently expect a ``bytes`` object for certain parameters. In essence, if a value is an acceptable input to the new ``bytes.byte`` constructor defined above, then it would be acceptable in the roles defined here (in addition to any other already supported inputs): * ``startswith()`` prefix(es) * ``endswith()`` suffix(es) * ``center()`` fill character * ``ljust()`` fill character * ``rjust()`` fill character * ``strip()`` character to strip * ``lstrip()`` character to strip * ``rstrip()`` character to strip * ``partition()`` separator argument * ``rpartition()`` separator argument * ``split()`` separator argument * ``rsplit()`` separator argument * ``replace()`` old value and new value In addition to the consistency motive, this approach also makes it easier to work with the indexing behaviour , as the result of an indexing operation can more easily be fed back in to other methods. For ``bytearray``, some additional changes are proposed to the current integer based operations to ensure they remain consistent with the proposed constructor changes:: * ``append()``: updated to be consistent with ``bytes.byte()`` * ``remove()``: updated to be consistent with ``bytes.byte()`` * ``+=``: updated to be consistent with ``bytes()`` changes (if any) * ``extend()``: updated to be consistent with ``bytes()`` changes (if any) Acknowledgement of surprising behaviour of some ``bytearray`` methods --------------------------------------------------------------------- Several of the ``bytes`` and ``bytearray`` methods have their origins in the Python 2 ``str`` API. As ``str`` is an immutable type, all of these operations are defined as returning a *new* instance, rather than operating in place. This contrasts with methods on other mutable types like ``list``, where ``list.sort()`` and ``list.reverse()`` operate in-place and return ``None``, rather than creating a new object. Backwards compatibility constraints make it impractical to change this behaviour at this point, but it may be appropriate to explicitly call out this quirk in the documentation for the ``bytearray`` type. It affects the following methods that could reasonably be expected to operate in-place on a mutable type: * ``center()`` * ``ljust()`` * ``rjust()`` * ``strip()`` * ``lstrip()`` * ``rstrip()`` * ``replace()`` * ``lower()`` * ``upper()`` * ``swapcase()`` * ``title()`` * ``capitalize()`` * ``translate()`` * ``expandtabs()`` * ``zfill()`` Note that the following ``bytearray`` operations *do* operate in place, as they're part of the mutable sequence API in ``bytearray``, rather than being inspired by the immutable Python 2 ``str`` API: * ``+=`` * ``append()`` * ``extend()`` * ``reverse()`` * ``remove()`` * ``pop()`` References ========== .. [ideas-thread1] https://mail.python.org/pipermail/python-ideas/2014-March/027295.html .. [empty-buffer-issue] http://bugs.python.org/issue20895 Copyright ========= This document has been placed in the public domain. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From abarnert at yahoo.com Sun Mar 30 06:02:33 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sat, 29 Mar 2014 21:02:33 -0700 (PDT) Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <lh7ga5$c77$2@ger.gmane.org> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> Message-ID: <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Terry Reedy <tjreedy at udel.edu> Sent: Saturday, March 29, 2014 3:08 PM > On 3/29/2014 4:54 PM, Ben Finney wrote: >> Richard Prosser <richard.prosser at mail.com> writes: >> >>> I wanted to search this list (and other Python ones) but it is > practically >>> impossible to do so. >> >> The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> >> has all its messages archived online for web browsing at >> <URL:http://mail.python.org/pipermail/python-ideas/>. >> >> So you can use any web search tool to search it. A good search engine >> like DuckDuckGo will let you constrain the search to a particular subset >> of the web: >> >> ? ? ? site:http://mail.python.org/pipermail/python-ideas foo bar baz >> >>> So why not add a tool like http://markmail.org/search/?q=python to >>> https://mail.python.org/mailman/listinfo/python-ideas or the like? >> >> We have generally-appicable services that index the web and make much >> better search tools. I don't often find a site's own custom search > tool >> to be sufficiently better to use it, when DuckDuckGo is available. > > This list and many others are also available through, archived at, and > searchable at news.gmane.org. How hard would it be to add appropriate links?(or, better, custom search boxes, or, worse but still helpful, a paragraph explaining how to search each of those manually) to the mailing list archive pages? For example, if you link to this: http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas ? then people just need to fill in their search terms on the Advanced Search page that comes up. From greg at krypto.org Sun Mar 30 08:10:53 2014 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 29 Mar 2014 23:10:53 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> Message-ID: <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com> On Sat, Mar 29, 2014 at 7:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote: > > I already have a draft PEP written that covers the constructor issue, > > iteration and adding acceptance of integer inputs to the remaining > > methods that don't currently handle them. There was some background > > explanation of the text/binary domain split in the Python 2->3 > > transition that I wanted Guido's feedback on before posting, but I > > just realised I can cut that out for now, and then add it back after > > Guido has had a chance to review it. > > > > So I'll tidy that up and get the draft posted later today. > > Guido pointed out most of the stuff I had asked him to look at wasn't > actually relevant to the PEP, so I just cut most of it entirely. > Suffice to say, after stepping back and reviewing them systematically > for the first time in years, I believe the APIs for the core binary > data types in Python 3 could do with a little sprucing up :) > > Web version: http://www.python.org/dev/peps/pep-0467/ > > ====================================== > PEP: 467 > Title: Improved API consistency for bytes and bytearray > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan <ncoghlan at gmail.com> > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 2014-03-30 > Python-Version: 3.5 > Post-History: 2014-03-30 > > > Abstract > ======== > > During the initial development of the Python 3 language specification, the > core ``bytes`` type for arbitrary binary data started as the mutable type > that is now referred to as ``bytearray``. Other aspects of operating in > the binary domain in Python have also evolved over the course of the Python > 3 series. > > This PEP proposes a number of small adjustments to the APIs of the > ``bytes`` > and ``bytearray`` types to make their behaviour more internally consistent > and to make it easier to operate entirely in the binary domain for use > cases > that actually involve manipulating binary data directly, rather than > converting it to a more structured form with additional modelling > semantics (such as ``str``) and then converting back to binary format after > processing. > > > Background > ========== > > Over the course of Python 3's evolution, a number of adjustments have been > made to the core ``bytes`` and ``bytearray`` types as additional practical > experience was gained with using them in code beyond the Python 3 standard > library and test suite. However, to date, these changes have been made > on a relatively ad hoc tactical basis as specific issues were identified, > rather than as part of a systematic review of the APIs of these types. This > approach has allowed inconsistencies to creep into the API design as to > which > input types are accepted by different methods. Additional inconsistencies > linger from an earlier pre-release design where there was *no* separate > ``bytearray`` type, and instead the core ``bytes`` type was mutable (with > no immutable counterpart), as well as from the origins of these types in > the text-like behaviour of the Python 2 ``str`` type. > > This PEP aims to provide the missing systematic review, with the goal of > ensuring that wherever feasible (given backwards compatibility constraints) > these current inconsistencies are addressed for the Python 3.5 release. > > > Proposals > ========= > > As a "consistency improvement" proposal, this PEP is actually about a > number > of smaller micro-proposals, each aimed at improving the self-consistency of > the binary data model in Python 3. Proposals are motivated by one of three > factors: > > * removing remnants of the original design of ``bytes`` as a mutable type > * more consistently accepting length 1 ``bytes`` objects as input where an > integer between ``0`` and ``255`` inclusive is expected, and vice-versa > * allowing users to easily convert integer output to a length 1 ``bytes`` > object > > > Alternate Constructors > ---------------------- > > The ``bytes`` and ``bytearray`` constructors currently accept an integer > argument, but interpret it to mean a zero-filled object of the given > length. > This is a legacy of the original design of ``bytes`` as a mutable type, > rather than a particularly intuitive behaviour for users. It has become > especially confusing now that other ``bytes`` interfaces treat integers > and the corresponding length 1 bytes instances as equivalent input. > Compare:: > > >>> b"\x03" in bytes([1, 2, 3]) > True > >>> 3 in bytes([1, 2, 3]) > True > > >>> bytes(b"\x03") > b'\x03' > >>> bytes(3) > b'\x00\x00\x00' > > This PEP proposes that the current handling of integers in the bytes and > bytearray constructors by deprecated in Python 3.5 and removed in Python > 3.6, being replaced by two more type appropriate alternate constructors > provided as class methods. The initial python-ideas thread [ideas-thread1]_ > that spawned this PEP was specifically aimed at deprecating this > constructor > behaviour. > > For ``bytes``, a ``byte`` constructor is proposed that converts integers > (as indicated by ``operator.index``) in the appropriate range to a > ``bytes`` > object, converts objects that support the buffer API to bytes, and also > passes through length 1 byte strings unchanged:: > > >>> bytes.byte(3) > b'\x03' > >>> bytes.byte(bytearray(bytes([3]))) > b'\x03' > >>> bytes.byte(memoryview(bytes([3]))) > b'\x03' > >>> bytes.byte(bytes([3])) > b'\x03' > >>> bytes.byte(512) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: bytes must be in range(0, 256) > >>> bytes.byte(b"ab") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: bytes.byte() expected a byte, but buffer of length 2 found > > One specific use case for this alternate constructor is to easily convert > the result of indexing operations on ``bytes`` and other binary sequences > from an integer to a ``bytes`` object. The documentation for this API > should note that its counterpart for the reverse conversion is ``ord()``. > > For ``bytearray``, a ``from_len`` constructor is proposed that preallocates > the buffer filled with a particular value (default to ``0``) as a direct > replacement for the current constructor behaviour, rather than having to > use > sequence repetition to achieve the same effect in a less intuitive way:: > > >>> bytearray.from_len(3) > bytearray(b'\x00\x00\x00') > >>> bytearray.from_len(3, 6) > bytearray(b'\x06\x06\x06') > > This part of the proposal was covered by an existing issue > [empty-buffer-issue]_ and a variety of names have been proposed > (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The > specific name currently proposed was chosen by analogy with > ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely > explicit that it is an alternate constructor rather than an in-place > mutation, as well as how it differs from the standard constructor. > > > Open questions > ^^^^^^^^^^^^^^ > > * Should ``bytearray.byte()`` also be added? Or is > ``bytearray(bytes.byte(x))`` sufficient for that case? > * Should ``bytes.from_len()`` also be added? Or is sequence repetition > sufficient for that case? > I prefer keeping them consistent across the types myself. * Should ``bytearray.from_len()`` use a different name? > This name works for me. > * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary > sequences with more than one element? The ``TypeError`` currently > proposed > is copied (with slightly improved wording) from the behaviour of > ``ord()`` > with sequences containing more than one code point, while ``ValueError`` > would be more consistent with the existing handling of out-of-range > integer values. > * ``bytes.byte()`` is defined above as accepting length 1 binary sequences > as individual bytes, but this is currently inconsistent with the main > ``bytes`` constructor:: > I don't like that bytes.byte() would accept anything other than an int. It should not accept length 1 binary sequences at all. I'd prefer to see bytes.byte(b"X") raise a TypeError. > >>> bytes([b"a", b"b", b"c"]) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: 'bytes' object cannot be interpreted as an integer > > Should the ``bytes`` constructor be changed to accept iterables of > length 1 > bytes objects in addition to iterables of integers? If so, should it > allow a mixture of the two in a single iterable? > > > Iteration > --------- > > Iteration over ``bytes`` objects and other binary sequences produces > integers. Rather than proposing a new method that would need to be added > not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially > to third party types as well, this PEP proposes that iteration to produce > length 1 ``bytes`` objects instead be handled by combining ``map`` with > the new ``bytes.byte()`` alternate constructor proposed above:: > > for x in map(bytes.byte, data): > # x is a length 1 ``bytes`` object, rather than an integer > # This works with *any* container of integers in the range > # 0 to 255 inclusive > > > Consistent support for different input types > -------------------------------------------- > > In Python 3.3, the binary search operations (``in``, ``count()``, > ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to > accept integers in the range 0 to 255 (inclusive) as their first argument > (in addition to the existing support for binary sequences). > > This PEP proposes extending that behaviour of accepting integers as being > equivalent to the corresponding length 1 binary sequence to several other > ``bytes`` and ``bytearray`` methods that currently expect a ``bytes`` > object for certain parameters. In essence, if a value is an acceptable > input to the new ``bytes.byte`` constructor defined above, then it would > be acceptable in the roles defined here (in addition to any other already > supported inputs): > > * ``startswith()`` prefix(es) > * ``endswith()`` suffix(es) > > * ``center()`` fill character > * ``ljust()`` fill character > * ``rjust()`` fill character > > * ``strip()`` character to strip > * ``lstrip()`` character to strip > * ``rstrip()`` character to strip > > * ``partition()`` separator argument > * ``rpartition()`` separator argument > > * ``split()`` separator argument > * ``rsplit()`` separator argument > > * ``replace()`` old value and new value > > In addition to the consistency motive, this approach also makes it easier > to work with the indexing behaviour , as the result of an indexing > operation > can more easily be fed back in to other methods. > > For ``bytearray``, some additional changes are proposed to the current > integer based operations to ensure they remain consistent with the proposed > constructor changes:: > > * ``append()``: updated to be consistent with ``bytes.byte()`` > * ``remove()``: updated to be consistent with ``bytes.byte()`` > * ``+=``: updated to be consistent with ``bytes()`` changes (if any) > Where was a change to += behavior mentioned? I don't see that above (or did I miss something?). > * ``extend()``: updated to be consistent with ``bytes()`` changes (if any) > > > Acknowledgement of surprising behaviour of some ``bytearray`` methods > --------------------------------------------------------------------- > > Several of the ``bytes`` and ``bytearray`` methods have their origins in > the > Python 2 ``str`` API. As ``str`` is an immutable type, all of these > operations are defined as returning a *new* instance, rather than operating > in place. This contrasts with methods on other mutable types like ``list``, > where ``list.sort()`` and ``list.reverse()`` operate in-place and return > ``None``, rather than creating a new object. > > Backwards compatibility constraints make it impractical to change this > behaviour at this point, but it may be appropriate to explicitly call out > this quirk in the documentation for the ``bytearray`` type. It affects the > following methods that could reasonably be expected to operate in-place on > a mutable type: > > * ``center()`` > * ``ljust()`` > * ``rjust()`` > * ``strip()`` > * ``lstrip()`` > * ``rstrip()`` > * ``replace()`` > * ``lower()`` > * ``upper()`` > * ``swapcase()`` > * ``title()`` > * ``capitalize()`` > * ``translate()`` > * ``expandtabs()`` > * ``zfill()`` > > Note that the following ``bytearray`` operations *do* operate in place, as > they're part of the mutable sequence API in ``bytearray``, rather than > being > inspired by the immutable Python 2 ``str`` API: > > * ``+=`` > * ``append()`` > * ``extend()`` > * ``reverse()`` > * ``remove()`` > * ``pop()`` > > > References > ========== > > .. [ideas-thread1] > https://mail.python.org/pipermail/python-ideas/2014-March/027295.html > .. [empty-buffer-issue] http://bugs.python.org/issue20895 > > > Copyright > ========= > > This document has been placed in the public domain. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140329/4575b9d0/attachment-0001.html> From ncoghlan at gmail.com Sun Mar 30 08:31:37 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 16:31:37 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com> Message-ID: <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com> On 30 March 2014 16:10, Gregory P. Smith <greg at krypto.org> wrote: >> Open questions >> ^^^^^^^^^^^^^^ >> >> * Should ``bytearray.byte()`` also be added? Or is >> ``bytearray(bytes.byte(x))`` sufficient for that case? >> * Should ``bytes.from_len()`` also be added? Or is sequence repetition >> sufficient for that case? > > I prefer keeping them consistent across the types myself. > >> * Should ``bytearray.from_len()`` use a different name? > > This name works for me. > >> >> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary >> sequences with more than one element? The ``TypeError`` currently >> proposed >> is copied (with slightly improved wording) from the behaviour of >> ``ord()`` >> with sequences containing more than one code point, while ``ValueError`` >> would be more consistent with the existing handling of out-of-range >> integer values. >> * ``bytes.byte()`` is defined above as accepting length 1 binary sequences >> as individual bytes, but this is currently inconsistent with the main >> ``bytes`` constructor:: > > > I don't like that bytes.byte() would accept anything other than an int. It > should not accept length 1 binary sequences at all. I'd prefer to see > bytes.byte(b"X") raise a TypeError. Unfortunately, it's not that simple, because accepting both is the only way I see of rendering the current APIs coherent. The problem is that the str-derived APIs expect bytes objects, the bytearray mutating methods expect integers, and in Python 3.3, the substring search APIs were updated to accept both. This means we currently have: >>> data = bytes([1, 2, 3, 4]) >>> 3 in data True >>> b"\x03" in data True >>> data.count(3) 1 >>> data.count(b"\x03") 1 >>> data.replace(3, 4) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: expected bytes, bytearray or buffer compatible object >>> data.replace(b"\x03", b"\x04") b'\x01\x02\x04\x04' >>> mutable = bytearray(data) >>> mutable bytearray(b'\x01\x02\x03\x04') >>> mutable.append(b"\x05") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: an integer is required >>> mutable.append(5) >>> mutable bytearray(b'\x01\x02\x03\x04\x05') Since some APIs work one way, some work the other, the only backwards compatible path I see to consistency is to always treat a length 1 byte string as an acceptable input for the APIs that currently accept an integer and vice-versa. That said, I think this hybrid nature accurately reflects the fact that indexing and slicing bytes objects in Python 3 return different types - the individual elements are integers, but the subsequences are bytes objects, and several of these APIs are either "element-or-subsequence" APIs (in which case they should accept both), or else they *should* have been element APIs, but currently expect a subsequence due to their Python 2 str heritage. If we had the opportunity to redesign these APIs from scratch, we'd likely make a much clearer distinction between element based APIs (that would use integers) and subsequence APIs (that would accept buffer implementing objects). As it is, I think the situation is inherently ambiguous, and providing hybrid APIs to help deal with that ambiguity is our best available option. >> For ``bytearray``, some additional changes are proposed to the current >> integer based operations to ensure they remain consistent with the >> proposed >> constructor changes:: >> >> * ``append()``: updated to be consistent with ``bytes.byte()`` >> * ``remove()``: updated to be consistent with ``bytes.byte()`` >> * ``+=``: updated to be consistent with ``bytes()`` changes (if any) > > > Where was a change to += behavior mentioned? I don't see that above (or did > I miss something?). It was an open question against the constructors - if bytes.byte() is defined as the PEP suggests, then the case can be made that the iterables accepted by the bytes() constructor should also be made more permissive in terms of the contents of the iterables it accepts. If *that* happens, then extending an existing bytearray should also become more permissive. Note that I'm not sold on actually changing that - that's why it's an open question, rather than something the PEP is currently proposing. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg at krypto.org Sun Mar 30 09:03:32 2014 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 30 Mar 2014 00:03:32 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com> <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com> Message-ID: <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com> On Sat, Mar 29, 2014 at 11:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 30 March 2014 16:10, Gregory P. Smith <greg at krypto.org> wrote: > >> Open questions > >> ^^^^^^^^^^^^^^ > >> > >> * Should ``bytearray.byte()`` also be added? Or is > >> ``bytearray(bytes.byte(x))`` sufficient for that case? > >> * Should ``bytes.from_len()`` also be added? Or is sequence repetition > >> sufficient for that case? > > > > I prefer keeping them consistent across the types myself. > > > >> * Should ``bytearray.from_len()`` use a different name? > > > > This name works for me. > > > >> > >> * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for > binary > >> sequences with more than one element? The ``TypeError`` currently > >> proposed > >> is copied (with slightly improved wording) from the behaviour of > >> ``ord()`` > >> with sequences containing more than one code point, while > ``ValueError`` > >> would be more consistent with the existing handling of out-of-range > >> integer values. > >> * ``bytes.byte()`` is defined above as accepting length 1 binary > sequences > >> as individual bytes, but this is currently inconsistent with the main > >> ``bytes`` constructor:: > > > > > > I don't like that bytes.byte() would accept anything other than an int. > It > > should not accept length 1 binary sequences at all. I'd prefer to see > > bytes.byte(b"X") raise a TypeError. > > Unfortunately, it's not that simple, because accepting both is the > only way I see of rendering the current APIs coherent. The problem is > that the str-derived APIs expect bytes objects, the bytearray mutating > methods expect integers, and in Python 3.3, the substring search APIs > were updated to accept both. This means we currently have: > > >>> data = bytes([1, 2, 3, 4]) > >>> 3 in data > True > >>> b"\x03" in data > True > >>> data.count(3) > 1 > >>> data.count(b"\x03") > 1 > >>> data.replace(3, 4) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: expected bytes, bytearray or buffer compatible object > >>> data.replace(b"\x03", b"\x04") > b'\x01\x02\x04\x04' > >>> mutable = bytearray(data) > >>> mutable > bytearray(b'\x01\x02\x03\x04') > >>> mutable.append(b"\x05") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: an integer is required > >>> mutable.append(5) > >>> mutable > bytearray(b'\x01\x02\x03\x04\x05') > > Since some APIs work one way, some work the other, the only backwards > compatible path I see to consistency is to always treat a length 1 > byte string as an acceptable input for the APIs that currently accept > an integer and vice-versa. > > That said, I think this hybrid nature accurately reflects the fact > that indexing and slicing bytes objects in Python 3 return different > types - the individual elements are integers, but the subsequences are > bytes objects, and several of these APIs are either > "element-or-subsequence" APIs (in which case they should accept both), > or else they *should* have been element APIs, but currently expect a > subsequence due to their Python 2 str heritage. > > If we had the opportunity to redesign these APIs from scratch, we'd > likely make a much clearer distinction between element based APIs > (that would use integers) and subsequence APIs (that would accept > buffer implementing objects). As it is, I think the situation is > inherently ambiguous, and providing hybrid APIs to help deal with that > ambiguity is our best available option. > Okay I see where you're going with this. So long as we limit this to APIs specifically surrounding bytes and bytearray I guess I'm "fine" with it (given the status quo and existing mess of never knowing which behaviors will be allowed where). Other APIs that accept numbers outside of bytes() and bytearray() related methods should *never* accept a bytes() of any length as valid numeric input. Thanks for exploring all of the APIs, we do have quite a mess that can be made better. > >> For ``bytearray``, some additional changes are proposed to the current > >> integer based operations to ensure they remain consistent with the > >> proposed > >> constructor changes:: > >> > >> * ``append()``: updated to be consistent with ``bytes.byte()`` > >> * ``remove()``: updated to be consistent with ``bytes.byte()`` > >> * ``+=``: updated to be consistent with ``bytes()`` changes (if any) > > > > > > Where was a change to += behavior mentioned? I don't see that above (or > did > > I miss something?). > > It was an open question against the constructors - if bytes.byte() is > defined as the PEP suggests, then the case can be made that the > iterables accepted by the bytes() constructor should also be made more > permissive in terms of the contents of the iterables it accepts. If > *that* happens, then extending an existing bytearray should also > become more permissive. > > Note that I'm not sold on actually changing that - that's why it's an > open question, rather than something the PEP is currently proposing. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/a373f86a/attachment.html> From ncoghlan at gmail.com Sun Mar 30 09:10:46 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 30 Mar 2014 17:10:46 +1000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> <CAGE7PN++cSZiK1OBXOhwPD9Oo6i6TDPNB_FNE9_z-SYXzthwjA@mail.gmail.com> <CADiSq7d_A4tojMPRY=qkTF4OUcOvQYS_tUztWE90r11SCBqqZg@mail.gmail.com> <CAGE7PNJAK-3dE+g+D4ifz_-CL4F_rMy-X1t8GPGo=MoKJMRFgA@mail.gmail.com> Message-ID: <CADiSq7d8vsGcTW5M+N5e6RFUh+dEWEipRcGqk9R8TdOrxW2GYA@mail.gmail.com> On 30 March 2014 17:03, Gregory P. Smith <greg at krypto.org> wrote: > Okay I see where you're going with this. So long as we limit this to APIs > specifically surrounding bytes and bytearray I guess I'm "fine" with it > (given the status quo and existing mess of never knowing which behaviors > will be allowed where). > > Other APIs that accept numbers outside of bytes() and bytearray() related > methods should *never* accept a bytes() of any length as valid numeric > input. Yeah, agreed. I've also reworked the relevant section of the PEP to give the examples I posted in my reply to you - I think they do a good job of showing why the current behaviour is problematic, and why "accept both" is the most plausible backwards compatible remedy available to us. > Thanks for exploring all of the APIs, we do have quite a mess that can be > made better. Thank Brandon Rhodes, too - he recently pointed me to bytes.replace() not accepting integers as a specific example of the current behaviour being confusing for users, and after I started down that rabbithole... well, this thread and the PEP were the end result :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From skip at pobox.com Sun Mar 30 13:35:26 2014 From: skip at pobox.com (Skip Montanaro) Date: Sun, 30 Mar 2014 06:35:26 -0500 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <CALFfu7AbxzusT8g=OOLEowu2r=knLEyt_J1AoRNp4rmbEKakAA@mail.gmail.com> Message-ID: <CANc-5Uxogk=FME-Ryz+2naE2eRg=hdba7PPhfuaNKbGDzt31Rg@mail.gmail.com> > Google lets you search a specific URL (and its subpages): > > site:https://mail.python.org/mailman/listinfo/python-ideas/ ... > > That's been the most effective tool I've used. +1. So effective, that I am almost completely hamstrung searching mailing list archives, web sites or Google Groups that are hidden from Google's view behind firewalls or login barriers. Skip From cjwelborn at live.com Sun Mar 30 17:20:45 2014 From: cjwelborn at live.com (Christopher Welborn) Date: Sun, 30 Mar 2014 10:20:45 -0500 Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com> References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando> <lg228f$d92$1@ger.gmane.org> <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com> Message-ID: <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl> On 03/30/2014 05:44 AM, Liam Marsh wrote: > hello, > is it possible to accept this syntax? > it is about the same subject, but an other proposition... > so, here it is: > > >>>for x in xs , y in ys: > ... ? ?***instructions*** > thank you! > > I think you meant to send this to the list, instead of my personal email. I use thunderbird and news.gmane.org to send things to python-ideas, but I think you can just send a mail to: Python-ideas at python.org with 'Re: <the original subject line>' such as: Re: [Python-ideas] for-loop-if like list comps have? -- Christopher Welborn (cj) cjwelborn at live.com http://welbornprod.com From bcannon at gmail.com Sun Mar 30 18:54:38 2014 From: bcannon at gmail.com (Dr. Brett Cannon) Date: Sun, 30 Mar 2014 16:54:38 +0000 Subject: [Python-ideas] Fixing the Python 3 bytes constructor References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> Message-ID: <CAP1=2W7ka+HgqfdKMJ7-75mApM8s260MwZeZH21YvddrxFBo5A@mail.gmail.com> Can we make bytes.from_len's second argument be keyword-only? Passing in two integer literals looks ambiguous if you don't know what the second argument is for. On Saturday, March 29, 2014 10:18:04 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: On 30 March 2014 07:07, Nick Coghlan <ncoghlan <ncoghlan at gmail.com>@<ncoghlan at gmail.com> gmail.com <ncoghlan at gmail.com>> wrote: > I already have a draft PEP written that covers the constructor issue, > iteration and adding acceptance of integer inputs to the remaining > methods that don't currently handle them. There was some background > explanation of the text/binary domain split in the Python 2->3 > transition that I wanted Guido's feedback on before posting, but I > just realised I can cut that out for now, and then add it back after > Guido has had a chance to review it. > > So I'll tidy that up and get the draft posted later today. Guido pointed out most of the stuff I had asked him to look at wasn't actually relevant to the PEP, so I just cut most of it entirely. Suffice to say, after stepping back and reviewing them systematically for the first time in years, I believe the APIs for the core binary data types in Python 3 could do with a little sprucing up :) Web version: http:// <http://www.python.org/dev/peps/pep-0467/> www.python.org <http://www.python.org/dev/peps/pep-0467/>/dev/<http://www.python.org/dev/peps/pep-0467/> peps <http://www.python.org/dev/peps/pep-0467/>/pep-0467/<http://www.python.org/dev/peps/pep-0467/> ====================================== PEP: 467 Title: Improved API consistency for bytes and bytearray Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan <ncoghlan at gmail.com>@ <ncoghlan at gmail.com> gmail.com <ncoghlan at gmail.com>> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2014-03-30 Python-Version: 3.5 Post-History: 2014-03-30 Abstract ======== During the initial development of the Python 3 language specification, the core ``bytes`` type for arbitrary binary data started as the mutable type that is now referred to as ``bytearray``. Other aspects of operating in the binary domain in Python have also evolved over the course of the Python 3 series. This PEP proposes a number of small adjustments to the APIs of the ``bytes`` and ``bytearray`` types to make their behaviour more internally consistent and to make it easier to operate entirely in the binary domain for use cases that actually involve manipulating binary data directly, rather than converting it to a more structured form with additional modelling semantics (such as ``str``) and then converting back to binary format after processing. Background ========== Over the course of Python 3's evolution, a number of adjustments have been made to the core ``bytes`` and ``bytearray`` types as additional practical experience was gained with using them in code beyond the Python 3 standard library and test suite. However, to date, these changes have been made on a relatively ad hoc tactical basis as specific issues were identified, rather than as part of a systematic review of the APIs of these types. This approach has allowed inconsistencies to creep into the API design as to which input types are accepted by different methods. Additional inconsistencies linger from an earlier pre-release design where there was *no* separate ``bytearray`` type, and instead the core ``bytes`` type was mutable (with no immutable counterpart), as well as from the origins of these types in the text-like behaviour of the Python 2 ``str`` type. This PEP aims to provide the missing systematic review, with the goal of ensuring that wherever feasible (given backwards compatibility constraints) these current inconsistencies are addressed for the Python 3.5 release. Proposals ========= As a "consistency improvement" proposal, this PEP is actually about a number of smaller micro-proposals, each aimed at improving the self-consistency of the binary data model in Python 3. Proposals are motivated by one of three factors: * removing remnants of the original design of ``bytes`` as a mutable type * more consistently accepting length 1 ``bytes`` objects as input where an integer between ``0`` and ``255`` inclusive is expected, and vice-versa * allowing users to easily convert integer output to a length 1 ``bytes`` object Alternate Constructors ---------------------- The ``bytes`` and ``bytearray`` constructors currently accept an integer argument, but interpret it to mean a zero-filled object of the given length. This is a legacy of the original design of ``bytes`` as a mutable type, rather than a particularly intuitive behaviour for users. It has become especially confusing now that other ``bytes`` interfaces treat integers and the corresponding length 1 bytes instances as equivalent input. Compare:: >>> b"\x03" in bytes([1, 2, 3]) True >>> 3 in bytes([1, 2, 3]) True >>> bytes(b"\x03") b'\x03' >>> bytes(3) b'\x00\x00\x00' This PEP proposes that the current handling of integers in the bytes and bytearray constructors by deprecated in Python 3.5 and removed in Python 3.6, being replaced by two more type appropriate alternate constructors provided as class methods. The initial python-ideas thread [ideas-thread1]_ that spawned this PEP was specifically aimed at deprecating this constructor behaviour. For ``bytes``, a ``byte`` constructor is proposed that converts integers (as indicated by ``operator.index``) in the appropriate range to a ``bytes`` object, converts objects that support the buffer API to bytes, and also passes through length 1 byte strings unchanged:: >>> bytes.byte(3) b'\x03' >>> bytes.byte(bytearray(bytes([3]))) b'\x03' >>> bytes.byte(memoryview(bytes([3]))) b'\x03' >>> bytes.byte(bytes([3])) b'\x03' >>> bytes.byte(512) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: bytes must be in range(0, 256) >>> bytes.byte(b"ab") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: bytes.byte() expected a byte, but buffer of length 2 found One specific use case for this alternate constructor is to easily convert the result of indexing operations on ``bytes`` and other binary sequences from an integer to a ``bytes`` object. The documentation for this API should note that its counterpart for the reverse conversion is ``ord()``. For ``bytearray``, a ``from_len`` constructor is proposed that preallocates the buffer filled with a particular value (default to ``0``) as a direct replacement for the current constructor behaviour, rather than having to use sequence repetition to achieve the same effect in a less intuitive way:: >>> bytearray.from_len(3) bytearray(b'\x00\x00\x00') >>> bytearray.from_len(3, 6) bytearray(b'\x06\x06\x06' This part of the proposal was covered by an existing issue [empty-buffer-issue]_ and a variety of names have been proposed (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The specific name currently proposed was chosen by analogy with ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely explicit that it is an alternate constructor rather than an in-place mutation, as well as how it differs from the standard constructor. Open questions ^^^^^^^^^^^^^^ * Should ``bytearray.byte()`` also be added? Or is ``bytearray(bytes.byte(x))`` sufficient for that case? * Should ``bytes.from_len()`` also be added? Or is sequence repetition sufficient for that case? * Should ``bytearray.from_len()`` use a different name? * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary sequences with more than one element? The ``TypeError`` currently proposed is copied (with slightly improved wording) from the behaviour of ``ord()`` with sequences containing more than one code point, while ``ValueError`` would be more consistent with the existing handling of out-of-range integer values. * ``bytes.byte()`` is defined above as accepting length 1 binary sequences as individual bytes, but this is currently inconsistent with the main ``bytes`` constructor:: >>> bytes([b"a", b"b", b"c"]) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'bytes' object cannot be interpreted as an integer Should the ``bytes`` constructor be changed to accept iterables of length 1 bytes objects in addition to iterables of integers? If so, should it allow a mixture of the two in a single iterable? Iteration --------- Iteration over ``bytes`` objects and other binary sequences produces integers. Rather than proposing a new method that would need to be added not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially to third party types as well, this PEP proposes that iteration to produce length 1 ``bytes`` objects instead be handled by combining ``map`` with the new ``bytes.byte()`` alternate constructor proposed above:: for x in map(bytes.byte, data): # x is a length 1 ``bytes`` object, rather than an integer # This works with *any* container of integers in the range # 0 to 255 inclusive Consistent support for different input types -------------------------------------------- In Python 3.3, the binary search operations (``in``, ``count()``, ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to accept integers in the range 0 to 255 (inclusive) as their first argument (in addition to the existing support for binary sequences). This PEP proposes extending that behaviour of accepting integers as being equivalent to the corresponding length 1 binary sequence to several other ``bytes`` and ``bytearray`` methods that currently expect a ``bytes`` object for certain parameters. In essence, if a value is an acceptable input to the new ``bytes.byte`` constructor defined above, then it would be acceptable in the roles defined here (in addition to any other already supported inputs): * ``startswith()`` prefix(es) * ``endswith()`` suffix(es) * ``center()`` fill character * ``ljust()`` fill character * ``rjust()`` fill character * ``strip()`` character to strip * ``lstrip()`` character to strip * ``rstrip()`` character to strip * ``partition()`` separator argument * ``rpartition()`` separator argument * ``split()`` separator argument * ``rsplit()`` separator argument * ``replace()`` old value and new value In addition to the consistency motive, this approach also makes it easier to work with the indexing behaviour , as the result of an indexing operation can more easily be fed back in to other methods. For ``bytearray``, some additional changes are proposed to the current integer based operations to ensure they remain consistent with the proposed constructor changes:: * ``append()``: updated to be consistent with ``bytes.byte()`` * ``remove()``: updated to be consistent with ``bytes.byte()`` * ``+=``: updated to be consistent with ``bytes()`` changes (if any) * ``extend()``: updated to be consistent with ``bytes()`` changes (if any) Acknowledgement of surprising behaviour of some ``bytearray`` methods --------------------------------------------------------------------- Several of the ``bytes`` and ``bytearray`` methods have their origins in the Python 2 ``str`` API. As ``str`` is an immutable type, all of these operations are defined as returning a *new* instance, rather than operating in place. This contrasts with methods on other mutable types like ``list``, where ``list.sort()`` and ``list.reverse()`` operate in-place and return ``None``, rather than creating a new object. Backwards compatibility constraints make it impractical to change this behaviour at this point, but it may be appropriate to explicitly call out this quirk in the documentation for the ``bytearray`` type. It affects the following methods that could reasonably be expected to operate in-place on a mutable type: * ``center()`` * ``ljust()`` * ``rjust()`` * ``strip()`` * ``lstrip()`` * ``rstrip()`` * ``replace()`` * ``lower()`` * ``upper()`` * ``swapcase()`` * ``title()`` * ``capitalize()`` * ``translate()`` * ``expandtabs()`` * ``zfill()`` Note that the following ``bytearray`` operations *do* operate in place, as they're part of the mutable sequence API in ``bytearray``, rather than being inspired by the immutable Python 2 ``str`` API: * ``+=`` * ``append()`` * ``extend()`` * ``reverse()`` * ``remove()`` * ``pop()`` References ========== .. [ideas-thread1] https<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> :// <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> mail.python.org<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> / <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> pipermail<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> /python-ideas/2014-March/027295.<https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> html <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html> .. [empty-buffer-issue] http:// <http://bugs.python.org/issue20895> bugs.python.org <http://bugs.python.org/issue20895>/issue20895<http://bugs.python.org/issue20895> Copyright ========= This document has been placed in the public domain. -- Nick Coghlan | ncoghlan <ncoghlan at gmail.com>@ <ncoghlan at gmail.com> gmail.com <ncoghlan at gmail.com> | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@ <Python-ideas at python.org>python.org <Python-ideas at python.org> https <https://mail.python.org/mailman/listinfo/python-ideas>://<https://mail.python.org/mailman/listinfo/python-ideas> mail.python.org <https://mail.python.org/mailman/listinfo/python-ideas> /mailman/ <https://mail.python.org/mailman/listinfo/python-ideas>listinfo<https://mail.python.org/mailman/listinfo/python-ideas> /python-ideas <https://mail.python.org/mailman/listinfo/python-ideas> Code of Conduct: http:// <http://python.org/psf/codeofconduct/>python.org<http://python.org/psf/codeofconduct/> / <http://python.org/psf/codeofconduct/>psf<http://python.org/psf/codeofconduct/> / <http://python.org/psf/codeofconduct/>codeofconduct<http://python.org/psf/codeofconduct/> / <http://python.org/psf/codeofconduct/> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/5ed046dc/attachment-0001.html> From guido at python.org Sun Mar 30 18:05:44 2014 From: guido at python.org (Guido van Rossum) Date: Sun, 30 Mar 2014 09:05:44 -0700 Subject: [Python-ideas] Fixing the Python 3 bytes constructor In-Reply-To: <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> References: <CADiSq7cbE6MSOUKLw3GeG6tetWhvJT_iv8iJhrDx7FfYtVodbg@mail.gmail.com> <5336F9A6.2030006@stoneleaf.us> <CAGE7PNJSL17RXheGTGo3JywBUTJww92grNqWYn036w8cou0Mkg@mail.gmail.com> <CADiSq7fndDrNNxW9a-t_zLGkRXJYFRTRahBN3=LdwZczp73BQA@mail.gmail.com> <CADiSq7fNFyD57ZPxhqDT8dkZhK5HyWDeQ1akev636ptoy=UYwg@mail.gmail.com> Message-ID: <CAP7+vJL4dQ3z9XMR6=iUgcsPK0kZDA=s16ty-ZoScA6AkDjKag@mail.gmail.com> On Sat, Mar 29, 2014 at 7:17 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 30 March 2014 07:07, Nick Coghlan <ncoghlan at gmail.com> wrote: > > I already have a draft PEP written that covers the constructor issue, > > iteration and adding acceptance of integer inputs to the remaining > > methods that don't currently handle them. There was some background > > explanation of the text/binary domain split in the Python 2->3 > > transition that I wanted Guido's feedback on before posting, but I > > just realised I can cut that out for now, and then add it back after > > Guido has had a chance to review it. > > > > So I'll tidy that up and get the draft posted later today. > > Guido pointed out most of the stuff I had asked him to look at wasn't > actually relevant to the PEP, so I just cut most of it entirely. > Suffice to say, after stepping back and reviewing them systematically > for the first time in years, I believe the APIs for the core binary > data types in Python 3 could do with a little sprucing up :) > Thanks for cutting it down, it's easier to concentrate on the essentials now. > Web version: http://www.python.org/dev/peps/pep-0467/ > > ====================================== > PEP: 467 > Title: Improved API consistency for bytes and bytearray > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan <ncoghlan at gmail.com> > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 2014-03-30 > Python-Version: 3.5 > Post-History: 2014-03-30 > > > Abstract > ======== > > During the initial development of the Python 3 language specification, the > core ``bytes`` type for arbitrary binary data started as the mutable type > that is now referred to as ``bytearray``. Other aspects of operating in > the binary domain in Python have also evolved over the course of the Python > 3 series. > > This PEP proposes a number of small adjustments to the APIs of the > ``bytes`` > and ``bytearray`` types to make their behaviour more internally consistent > and to make it easier to operate entirely in the binary domain for use > cases > that actually involve manipulating binary data directly, rather than > converting it to a more structured form with additional modelling > semantics (such as ``str``) and then converting back to binary format after > processing. > I hope you don't mind I cut the last 60% of this sentence (everything after "binary domain"). > > > Background > ========== > > Over the course of Python 3's evolution, a number of adjustments have been > made to the core ``bytes`` and ``bytearray`` types as additional practical > experience was gained with using them in code beyond the Python 3 standard > library and test suite. However, to date, these changes have been made > on a relatively ad hoc tactical basis as specific issues were identified, > rather than as part of a systematic review of the APIs of these types. I'm not sure you can claim that. We probably have more information based on experience now than when we did the redesign. (At that time most experience was based on using str() for binary data.) > This > approach has allowed inconsistencies to creep into the API design as to > which > input types are accepted by different methods. Additional inconsistencies > linger from an earlier pre-release design where there was *no* separate > ``bytearray`` type, and instead the core ``bytes`` type was mutable (with > no immutable counterpart), as well as from the origins of these types in > the text-like behaviour of the Python 2 ``str`` type. > You make it sound as if modeling bytes() after Python 2's str() was an accident. It wasn't. > This PEP aims to provide the missing systematic review, with the goal of > ensuring that wherever feasible (given backwards compatibility constraints) > these current inconsistencies are addressed for the Python 3.5 release. > I would like to convince you to aim lower, drop the "systematic review", and just focus on some changes that are likely to improve users' experience (which includes porting Python 2 code). > > > Proposals > ========= > > As a "consistency improvement" proposal, this PEP is actually about a > number > of smaller micro-proposals, each aimed at improving the self-consistency of > the binary data model in Python 3. Proposals are motivated by one of three > factors: > > * removing remnants of the original design of ``bytes`` as a mutable type > Yes. * more consistently accepting length 1 ``bytes`` objects as input where an > integer between ``0`` and ``255`` inclusive is expected, and vice-versa > Not sure I like this as a goal. OK, stronger: I don't like this goal. > * allowing users to easily convert integer output to a length 1 ``bytes`` > object > I think you meant integer values instead of output? In Python 2 we did this with the global function chr(), but in Python 3 that creates a str(). (The history of chr() and ord() sa built-in functions is that they long predates the notion of methods (class- or otherwise), and their naming comes straight from Pascal.) Anyway, I don't know that the use case is so common that it needs more than bytes([i]) or bytearray([i]) -- if there is an argument to be made for bytes.byte(i) and bytearray.byte(i) it would be that the [i] in the constructor is somewhat hard to grasp. > > > Alternate Constructors > ---------------------- > > The ``bytes`` and ``bytearray`` constructors currently accept an integer > argument, but interpret it to mean a zero-filled object of the given > length. > This is one of the two legacies of the original "mutable bytes" design, and I agree we should strive to replace it -- although I think one round of deprecation may be too quick. (The other legacy is of course that b[i] is an int, not a bytes -- it's the worse problem, but I don't think we can fix it without breaking more than the fix would be worth.) > This is a legacy of the original design of ``bytes`` as a mutable type, > rather than a particularly intuitive behaviour for users. It has become > especially confusing now that other ``bytes`` interfaces treat integers > and the corresponding length 1 bytes instances as equivalent input. > Compare:: > > >>> b"\x03" in bytes([1, 2, 3]) > True > >>> 3 in bytes([1, 2, 3]) > True > > >>> bytes(b"\x03") > b'\x03' > >>> bytes(3) > b'\x00\x00\x00' > > This PEP proposes that the current handling of integers in the bytes and > bytearray constructors by deprecated in Python 3.5 and removed in Python > 3.6, being replaced by two more type appropriate alternate constructors > provided as class methods. The initial python-ideas thread [ideas-thread1]_ > that spawned this PEP was specifically aimed at deprecating this > constructor > behaviour. > > For ``bytes``, a ``byte`` constructor is proposed that converts integers > (as indicated by ``operator.index``) I know why you reference this, but it feels confusing to me. At this point in the narrative it's better to just say "integer" and explain how it decides "integer-ness" later. > in the appropriate range to a ``bytes`` > object, converts objects that support the buffer API to bytes, and also > passes through length 1 byte strings unchanged:: > I think the second half (accepting bytes instances of length 1) is wrong here and doesn't actually have a practical use case. I'll say more below. > > >>> bytes.byte(3) > b'\x03' > >>> bytes.byte(bytearray(bytes([3]))) > b'\x03' > >>> bytes.byte(memoryview(bytes([3]))) > b'\x03' > >>> bytes.byte(bytes([3])) > b'\x03' > >>> bytes.byte(512) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ValueError: bytes must be in range(0, 256) > >>> bytes.byte(b"ab") > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: bytes.byte() expected a byte, but buffer of length 2 found > > One specific use case for this alternate constructor is to easily convert > the result of indexing operations on ``bytes`` and other binary sequences > from an integer to a ``bytes`` object. The documentation for this API > should note that its counterpart for the reverse conversion is ``ord()``. > However, in a pinch, b[0] will do as well, assuming you don't need the length check implied by ord(). > > For ``bytearray``, a ``from_len`` constructor is proposed that preallocates > the buffer filled with a particular value (default to ``0``) as a direct > replacement for the current constructor behaviour, rather than having to > use > sequence repetition to achieve the same effect in a less intuitive way:: > > >>> bytearray.from_len(3) > bytearray(b'\x00\x00\x00') > >>> bytearray.from_len(3, 6) > bytearray(b'\x06\x06\x06') > > This part of the proposal was covered by an existing issue > [empty-buffer-issue]_ and a variety of names have been proposed > (``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The > specific name currently proposed was chosen by analogy with > ``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely > explicit that it is an alternate constructor rather than an in-place > mutation, as well as how it differs from the standard constructor. > I think you need to brainstorm more on the name; from_len() looks pretty awkward. And I think it's better to add it to bytes() as well, since the two classes intentionally try to be as similar as possible. > > > Open questions > ^^^^^^^^^^^^^^ > > * Should ``bytearray.byte()`` also be added? Or is > ``bytearray(bytes.byte(x))`` sufficient for that case? > It should be added. > * Should ``bytes.from_len()`` also be added? Or is sequence repetition > sufficient for that case? > It should be added. > * Should ``bytearray.from_len()`` use a different name? > Yes. > * Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary > sequences with more than one element? The ``TypeError`` currently > proposed > is copied (with slightly improved wording) from the behaviour of > ``ord()`` > with sequences containing more than one code point, while ``ValueError`` > would be more consistent with the existing handling of out-of-range > integer values. > It should not accept any bytes arguments. But if somehow you convince me otherwise, it should be ValueError (and honestly, ord() is wrong there). > * ``bytes.byte()`` is defined above as accepting length 1 binary sequences > as individual bytes, but this is currently inconsistent with the main > ``bytes`` constructor:: > > >>> bytes([b"a", b"b", b"c"]) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > TypeError: 'bytes' object cannot be interpreted as an integer > > Should the ``bytes`` constructor be changed to accept iterables of > length 1 > bytes objects in addition to iterables of integers? If so, should it > allow a mixture of the two in a single iterable? > Noooooooooooooooooooooooooo!!!!! > > > Iteration > --------- > > Iteration over ``bytes`` objects and other binary sequences produces > integers. Rather than proposing a new method that would need to be added > not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially > to third party types as well, this PEP proposes that iteration to produce > length 1 ``bytes`` objects instead be handled by combining ``map`` with > the new ``bytes.byte()`` alternate constructor proposed above:: > > for x in map(bytes.byte, data): > # x is a length 1 ``bytes`` object, rather than an integer > # This works with *any* container of integers in the range > # 0 to 255 inclusive > I can see why you don't like a new method, but this idiom is way too verbose and unintuitive to ever gain traction. Let's just add a new method to all three types, 3rd party types will get the message. > > > Consistent support for different input types > -------------------------------------------- > > In Python 3.3, the binary search operations (``in``, ``count()``, > ``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to > accept integers in the range 0 to 255 (inclusive) as their first argument > (in addition to the existing support for binary sequences). > I wonder if that wasn't a bit over-zealous. While 'in', count() and index() are sequence methods (looking for elements) that have an extended meaning (looking for substrings) for string types, the find() and r*() variants are only defined for strings. > This PEP proposes extending that behaviour of accepting integers as being > equivalent to the corresponding length 1 binary sequence to several other > ``bytes`` and ``bytearray`` methods that currently expect a ``bytes`` > object for certain parameters. In essence, if a value is an acceptable > input to the new ``bytes.byte`` constructor defined above, then it would > be acceptable in the roles defined here (in addition to any other already > supported inputs): > > * ``startswith()`` prefix(es) > * ``endswith()`` suffix(es) > > * ``center()`` fill character > * ``ljust()`` fill character > * ``rjust()`` fill character > > * ``strip()`` character to strip > * ``lstrip()`` character to strip > * ``rstrip()`` character to strip > > * ``partition()`` separator argument > * ``rpartition()`` separator argument > > * ``split()`` separator argument > * ``rsplit()`` separator argument > > * ``replace()`` old value and new value > > In addition to the consistency motive, this approach also makes it easier > to work with the indexing behaviour , as the result of an indexing > operation > can more easily be fed back in to other methods. > I think herein lies madness. The intention seems to be to paper over as much as possible the unfortunate behavior of b[i]. But how often does any of these methods get called with such a construct? And how often will that be in a context where this is the *only* thing that is affected by b[i] returning an int in Python 3 but a string in Python 2? (In my experience these are mostly called with literal arguments, except inside wrapper functions that are themselves intended to be called with a literal argument.) Weakening the type checking here seems a bad idea -- it would accept integers in *any* context, and that would just cause more nasty debugging issues. > For ``bytearray``, some additional changes are proposed to the current > integer based operations to ensure they remain consistent with the proposed > constructor changes:: > > * ``append()``: updated to be consistent with ``bytes.byte()`` > * ``remove()``: updated to be consistent with ``bytes.byte()`` > * ``+=``: updated to be consistent with ``bytes()`` changes (if any) > * ``extend()``: updated to be consistent with ``bytes()`` changes (if any) > Eew again. These are operations from the MutableSequence ABC and there is no reason to make their signatures fuzzier. > > > Acknowledgement of surprising behaviour of some ``bytearray`` methods > --------------------------------------------------------------------- > > Several of the ``bytes`` and ``bytearray`` methods have their origins in > the > Python 2 ``str`` API. You make it sound as if this is a bad thing or an accident. > As ``str`` is an immutable type, all of these > operations are defined as returning a *new* instance, rather than operating > in place. This contrasts with methods on other mutable types like ``list``, > where ``list.sort()`` and ``list.reverse()`` operate in-place and return > ``None``, rather than creating a new object. > So does bytestring.reverse(). And if you really insist we can add bytestring.sort(). :-) > > Backwards compatibility constraints make it impractical to change this > behaviour at this point, but it may be appropriate to explicitly call out > this quirk in the documentation for the ``bytearray`` type. It affects the > following methods that could reasonably be expected to operate in-place on > a mutable type: > > * ``center()`` > * ``ljust()`` > * ``rjust()`` > * ``strip()`` > * ``lstrip()`` > * ``rstrip()`` > * ``replace()`` > * ``lower()`` > * ``upper()`` > * ``swapcase()`` > * ``title()`` > * ``capitalize()`` > * ``translate()`` > * ``expandtabs()`` > * ``zfill()`` > That all feels like hypercorrection. These are string methods and it would be completely wrong if bytearray changed them to modify the object in-place. I also don't see why anyone would think these would modify the object, given that everybody encounters these first for the str() type, then for bytes(), then finally (by extension) for bytearray(). The *only* place where there should be any confusion about whether the value is mutated or the variable is updated with a new object would be the += operator (and *=) but that's due to that operator's ambiguity. > Note that the following ``bytearray`` operations *do* operate in place, as > they're part of the mutable sequence API in ``bytearray``, rather than > being > inspired by the immutable Python 2 ``str`` API: > > * ``+=`` > * ``append()`` > * ``extend()`` > * ``reverse()`` > * ``remove()`` > * ``pop()`` > Right. And there's nothing wrong with this. > > > References > ========== > > .. [ideas-thread1] > https://mail.python.org/pipermail/python-ideas/2014-March/027295.html > .. [empty-buffer-issue] http://bugs.python.org/issue20895 > > > Copyright > ========= > > This document has been placed in the public domain. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/06a5ffd0/attachment-0001.html> From ram.rachum at gmail.com Sun Mar 30 19:59:27 2014 From: ram.rachum at gmail.com (Ram Rachum) Date: Sun, 30 Mar 2014 10:59:27 -0700 (PDT) Subject: [Python-ideas] Allow specifying list of functions to `map` Message-ID: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> If `map` accepted a tuple for its first argument, things like this: stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate())) Could become this: stdout, stderr = map((bytes.decode, str.strip), popen.communicate()) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140330/f844b68c/attachment.html> From python at mrabarnett.plus.com Sun Mar 30 21:06:25 2014 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 30 Mar 2014 20:06:25 +0100 Subject: [Python-ideas] Allow specifying list of functions to `map` In-Reply-To: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> Message-ID: <53386B31.8040505@mrabarnett.plus.com> On 2014-03-30 18:59, Ram Rachum wrote: > If `map` accepted a tuple for its first argument, things like this: > > stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate())) > > Could become this: > > stdout, stderr = map((bytes.decode, str.strip), popen.communicate()) > I think that the problem there is that it's not obvious in what order they are applied. You could, for example, argue that map((f, g) x) should do map(f, map(g, x) rather than map(g, map(f, x)). From abarnert at yahoo.com Sun Mar 30 23:12:42 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 30 Mar 2014 14:12:42 -0700 (PDT) Subject: [Python-ideas] for-loop-if like list comps have? In-Reply-To: <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl> References: <lg0ck8$nnh$1@ger.gmane.org> <20140315023747.GA16526@ando> <lg228f$d92$1@ger.gmane.org> <CACPPHzvMLmx4WefY+W9eX1pRb4neyCA2wGizu-Bw-_Tu1BwNGA@mail.gmail.com> <BLU0-SMTP1091872A6EEF58BB8D8EB27C5600@phx.gbl> Message-ID: <1396213962.86004.YahooMailNeo@web181001.mail.ne1.yahoo.com> From: Christopher Welborn <cjwelborn at live.com> > On 03/30/2014 05:44 AM, Liam Marsh wrote: >> hello, >> is it possible to accept this syntax? >> it is about the same subject, but an other proposition... >> so, here it is: >> >> ? >>>for x in xs , y in ys: >> ... ? ?***instructions*** >> thank you! (This is obviously a reply to Liam, not Christopher.) It's not clear which of these you want that to mean: ? ? for x, y in zip(xs, ys): ? ? ? ? do_stuff(x, y) ? ? for x, y in product(xs, ys): ? ? ? ? do_stuff(x, y) And the existing versions are already pretty clear and readable. In a listcomp, it's often clearer to use two for clauses than product? but usually in that case you write them on separate lines, and in a statement, it would be even clearer to use nested statements. Compare this somewhat realistic use case: ? ? transformed_values = (long expression with value and sublist ? ? ? ? ? ? ? ? ? ? ? ? ? for sublist in?long_function_that_returns_a_list(various, arguments) ? ? ? ? ? ? ? ? ? ? ? ? ? for value in sublist) ? ? for sublist in long_function_that_returns_a_list(various, arguments): ? ? ? ? for value in sublist: ? ? ? ? ? ? yield long expression with value and sublist Would you really want to write the second one on one line, even if you could? ? ? for sublist in?long_function_that_returns_a_list(various, arguments), value in sublist: ? ? ? ? yield long expression with value and sublist From rosuav at gmail.com Mon Mar 31 00:00:47 2014 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 31 Mar 2014 09:00:47 +1100 Subject: [Python-ideas] Allow specifying list of functions to `map` In-Reply-To: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> Message-ID: <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com> On Mon, Mar 31, 2014 at 4:59 AM, Ram Rachum <ram.rachum at gmail.com> wrote: > If `map` accepted a tuple for its first argument, things like this: > > stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate())) > > Could become this: > > stdout, stderr = map((bytes.decode, str.strip), popen.communicate()) stdout, stderr = map(lambda x: x.decode().strip(), popen.communicate()) Or: stdout, stderr = popen.communicate() stdout = stdout.decode().strip() stderr = stderr.decode().strip() Yes, the latter is a bit of duplication, but it's clear what's going on. (Obviously appropriate only where you know exactly how many elements there'll be, as in your example.) ChrisA From cs at zip.com.au Mon Mar 31 00:22:34 2014 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 31 Mar 2014 09:22:34 +1100 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> Message-ID: <20140330222234.GA14300@cskk.homeip.net> On 30Mar2014 08:05, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote: > > Richard Prosser <richard.prosser at mail.com> writes: > > We have generally-appicable services that index the web and make much > > better search tools. I don't often find a site's own custom search tool > > to be sufficiently better to use it, when DuckDuckGo is available. > > That said, migrating to Mailman3 + HyperKitty is definitely in the > longer term plans for the python.org mailing list infrastructure, and > that includes integrated search on the HyperKitty side of things. > > While Mailman3 has been an ongoing project for quite some time (I > believe the remaining blockers mostly relate to handling migrations of > existing Mailman 2 installations), the HyperKitty work is mostly being > driven by some Fedora folks in order to upgrade Fedora's own > infrastructure. You can see the current state of the prototype here: > https://lists.stg.fedoraproject.org/archives/ Please tell me Mailman3 still includes the pipermail stuff still. Looking at the Fedora HyperKitty page you cite, is it just me or is this really slow and clunky? [Ten minutes later...] It is slightly quicker now, but that first page load took minutes. The main page search (multilist) doesn't show the list name in the search results, making it harder to gauge relevance. I think this is an argument for the OP's "convenient search of relevant python lists" request, even if that were a quick hack that readirected to a general purpose search engine with a funky search criterion. But a search engine covers multiple sites; this can only search its own lists. Also, there are no archive download links. Nor can I see the original text of any message (no headers or anything). This cripples personal searching. When I join a list my first act, if possible, is to download the pipermail archives and unpack them into the mail folder in which future lists posts will go. That gets me: - local search of local data access when offline, privacy, speed, whatever tools or indices I prefer - search tools of my choosing (eg mairix or mu), including the search facilities of my mail reader - using my mail reader search lets me read messages in my _preferred_ form, as it would for anyone else using their preferrer mail reader Without an archive download, this is all far far less useful. HyperKitty looks... pretty... I suppose (for some definition of the term; I'm no far of the great big blocks of whitespace on either side, etc), for someone who enjoys web forums. Disgruntledly yours, -- Cameron Simpson <cs at zip.com.au> From python at mrabarnett.plus.com Mon Mar 31 00:24:30 2014 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 30 Mar 2014 23:24:30 +0100 Subject: [Python-ideas] Allow specifying list of functions to `map` In-Reply-To: <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com> References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> <CAPTjJmq8csRUU4jb_SmDFOu+idFkYAOsT_9O=4EK=3svo6v+Cg@mail.gmail.com> Message-ID: <5338999E.3050206@mrabarnett.plus.com> On 2014-03-30 23:00, Chris Angelico wrote: > On Mon, Mar 31, 2014 at 4:59 AM, Ram Rachum <ram.rachum at gmail.com> wrote: >> If `map` accepted a tuple for its first argument, things like this: >> >> stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate())) >> >> Could become this: >> >> stdout, stderr = map((bytes.decode, str.strip), popen.communicate()) > > stdout, stderr = map(lambda x: x.decode().strip(), popen.communicate()) > > Or: > > stdout, stderr = popen.communicate() > stdout = stdout.decode().strip() > stderr = stderr.decode().strip() > Or: stdout, stderr = (x.decode().strip() for x in popen.communicate()) for that matter. > Yes, the latter is a bit of duplication, but it's clear what's going > on. (Obviously appropriate only where you know exactly how many > elements there'll be, as in your example.) > From ncoghlan at gmail.com Mon Mar 31 00:37:02 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 31 Mar 2014 08:37:02 +1000 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <20140330222234.GA14300@cskk.homeip.net> References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> <20140330222234.GA14300@cskk.homeip.net> Message-ID: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> On 31 Mar 2014 08:22, "Cameron Simpson" <cs at zip.com.au> wrote: > > On 30Mar2014 08:05, Nick Coghlan <ncoghlan at gmail.com> wrote: > > On 30 March 2014 06:54, Ben Finney <ben+python at benfinney.id.au> wrote: > > > Richard Prosser <richard.prosser at mail.com> writes: > > > We have generally-appicable services that index the web and make much > > > better search tools. I don't often find a site's own custom search tool > > > to be sufficiently better to use it, when DuckDuckGo is available. > > > > That said, migrating to Mailman3 + HyperKitty is definitely in the > > longer term plans for the python.org mailing list infrastructure, and > > that includes integrated search on the HyperKitty side of things. > > > > While Mailman3 has been an ongoing project for quite some time (I > > believe the remaining blockers mostly relate to handling migrations of > > existing Mailman 2 installations), the HyperKitty work is mostly being > > driven by some Fedora folks in order to upgrade Fedora's own > > infrastructure. You can see the current state of the prototype here: > > https://lists.stg.fedoraproject.org/archives/ > > Please tell me Mailman3 still includes the pipermail stuff still. Mailman3 includes a pluggable archiver model - that's how HyperKitty can be a separate project, specifically aimed at tackling the "seamless web gateway" problem. > That gets me: > > - local search of local data > access when offline, privacy, speed, whatever tools or indices I prefer > > - search tools of my choosing (eg mairix or mu), including the > search facilities of my mail reader > > - using my mail reader search lets me read messages in my _preferred_ form, > as it would for anyone else using their preferrer mail reader > > Without an archive download, this is all far far less useful. The pluggable archiver model in MM3 means HyperKitty itself doesn't need to handle that usage model. However, I'm not aware of any current efforts to create an MM3 archiver that is closer to a pure pipermail replacement (the workflow you describe strikes me as being incredibly unusual these days, as it requires either running your own mail server, or only accessing your email from one device) Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/fbeaa6fa/attachment.html> From abarnert at yahoo.com Mon Mar 31 00:53:31 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Sun, 30 Mar 2014 15:53:31 -0700 (PDT) Subject: [Python-ideas] Fw: Allow specifying list of functions to `map` In-Reply-To: <1396218299.38518.YahooMailNeo@web181005.mail.ne1.yahoo.com> References: <26375700-d2c5-4f18-b248-bd409fdd62c5@googlegroups.com> <1396218299.38518.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <1396220011.5869.YahooMailNeo@web181002.mail.ne1.yahoo.com> Sorry, google-grouped, so I'm not sure if this went out to the list or not; apologies if it's a dup? From: Ram Rachum <ram.rachum at gmail.com> Sent: Sunday, March 30, 2014 10:59 AM > If `map` accepted a tuple for its first argument, things like this:? > > ? ? stdout, stderr = map(str.strip, map(bytes.decode, popen.communicate())) > > Could become this:? > > ? ? stdout, stderr = map((bytes.decode, str.strip), popen.communicate()) I'm not sure whether to read that as returning: ? ? map(bytes.decode, x), map(str.strip, x) ? ? map(compose(bytes.decode, str.strip), x) ? ? map(rcompose(bytes.decode, str.strip), x) None of the functional languages that Python borrowed map from have any of? these, but that's because they all have compose as a builtin, or even an operator: ? ? map(bytes.decode @ str.strip, popen.communicate()) But I think in Python this is much more readable as an expression on each value rather than a higher-order function: ? ? (text.decode().strip() for text in popen.communicate()) Even in Haskell, a listcomp is often more readable than mapping a compose or other higher-order function, but that's much more true when you're using Python-style method calls. From cs at zip.com.au Mon Mar 31 02:47:33 2014 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 31 Mar 2014 11:47:33 +1100 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> References: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> Message-ID: <20140331004733.GA73427@cskk.homeip.net> On 31Mar2014 08:37, Nick Coghlan <ncoghlan at gmail.com> wrote: > On 31 Mar 2014 08:22, "Cameron Simpson" <cs at zip.com.au> wrote: > > Please tell me Mailman3 still includes the pipermail stuff still. > > Mailman3 includes a pluggable archiver model - that's how HyperKitty can be > a separate project, specifically aimed at tackling the "seamless web > gateway" problem. If it can be run in parallel, fine. If it replaces pipermail, it looks like a step backwards. > > That gets me: > > - local search of local data > > access when offline, privacy, speed, whatever tools or indices I > prefer > > > > - search tools of my choosing (eg mairix or mu), including the > > search facilities of my mail reader > > > > - using my mail reader search lets me read messages in my _preferred_ > form, > > as it would for anyone else using their preferrer mail reader > > > > Without an archive download, this is all far far less useful. > > The pluggable archiver model in MM3 means HyperKitty itself doesn't need to > handle that usage model. However, I'm not aware of any current efforts to > create an MM3 archiver that is closer to a pure pipermail replacement (the > workflow you describe strikes me as being incredibly unusual these days, as > it requires either running your own mail server, or only accessing your > email from one device) My workflow is unusual only because most people don't think about it and because the existing MUAs do not make it easy. There _should_ be a "join a mailing list" button with optional archive suck. Most modern MUAs have a "search my mail" facility. Making the raw list archive hard to obtain gets in the way of offering the user's own search field for search. _Nothing_ in my workflow requires running my own mail server, it merely requires having local folders; which is entirely reasonable with an IMAP service, and entirely reasonable with multiple devices. Using one's own search tools is only unusual because unless you're a nerd it is hard. Likewise, sucking down the archives is difficult for end users for various reasons: - even pipemail makes you do that in pieces - the mbox archive URLs (often gzipped) don't come down with a cool MIME type meaning "UNIX mailbox format" - the pipermail archives need some unobfuscation before use as pure mbox files - most MUAs do not install a platform hook to handle an mbox MIME type (if there even is one), thus making an mbox download that much harder on the user. None of these is hard to address. I do it with a tiny shell script, but that is not for most people. I still content that any change that removes the simple ability to fetch the list archive for use by the end user is a huge step backwards, regardless of the prettiness of whatever replaces it. Cheers, -- Cameron Simpson <cs at zip.com.au> Early to bed and early to rise makes a man healthy and wealthy with bags under his eyes. - Dave Cochran, <cochran at dg-rtp.dg.com> From turnbull at sk.tsukuba.ac.jp Mon Mar 31 07:07:24 2014 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 31 Mar 2014 14:07:24 +0900 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <20140331004733.GA73427@cskk.homeip.net> References: <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> <20140331004733.GA73427@cskk.homeip.net> Message-ID: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp> N.B. This is off-topic. If you really care that much, people who want to hear about it are over on mailman-developers at python.org. Cameron Simpson writes: > - the mbox archive URLs (often gzipped) don't come down with a > cool MIME type meaning "UNIX mailbox format" There isn't one, can't be one at present, and likely won't be one in the future: http://www.jwz.org/doc/content-length.html. This still applies to some extent today despite Jamie's rant being almost 20 years old -- Mailman can't choose the mbox spec being used, that depends on the MTA. If you want something that could actually have a MIME-Type, there's Babyl, MMDF, or (file-per-message) maildir, all of which have reasonably precise specs -- but aren't in anywhere near as wide use. Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's far more robust and easier to handle than mbox. > I still content that any change that removes the simple ability to > fetch the list archive for use by the end user is a huge step > backwards, regardless of the prettiness of whatever replaces it. It's not a question of "removing". This has always been a user choice, and many lists use third-party services (mail-archive.com) or non-pipermail archivers (MHonArc). It's just that Mailman 2 with its bundled mbox-format archiver made it easy to provide by default. If lots of users really prefer pipermail, for whatever reason, it's not that hard to add by hand, and could be packaged on PyPI. From cs at zip.com.au Mon Mar 31 07:52:14 2014 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 31 Mar 2014 16:52:14 +1100 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20140331055214.GA3830@cskk.homeip.net> On 31Mar2014 14:07, Stephen J. Turnbull <turnbull at sk.tsukuba.ac.jp> wrote: > N.B. This is off-topic. If you really care that much, people who want > to hear about it are over on mailman-developers at python.org. I may join that list, though I'd be an informed commenter (== whinger) rather than a dev. > Cameron Simpson writes: > > - the mbox archive URLs (often gzipped) don't come down with a > > cool MIME type meaning "UNIX mailbox format" > > There isn't one, can't be one at present, and likely won't be one in > the future: http://www.jwz.org/doc/content-length.html. This still > applies to some extent today despite Jamie's rant being almost 20 > years old -- Mailman can't choose the mbox spec being used, that > depends on the MTA. It can cheerfully choose the mbox spec for delivery of an archive. It is very simple and is a single flat file. As a bonus, many MUAs can read it directly. I've seen JMZ's content-length rant in the past, or something equivalent; if one documents what is being done an mbox can still be delivered usefully to the end user. If one doesn't want Content-Length headers in the archive format, the ">From" hack works quite well. Conversely, if is perfectly possible to create valid Content-Length heders when constructing the archive. We're talking about an archive/transfer format here, not whatever raw format one works with locally. > If you want something that could actually have a MIME-Type, there's > Babyl, MMDF, or (file-per-message) maildir, all of which have > reasonably precise specs -- but aren't in anywhere near as wide use. I meant the BSD mailbox format, often called mbox. I know there are several minor variants, but if mailman (or whatever subcomponent) documents its chosen flavour that's perfectly tractable. > Maildir is what Mailman 3's bundled "trivial archiver" uses, as it's > far more robust and easier to handle than mbox. Maildir is great as a storage platform; I use it myself for many folders. But as a delivery format for end users one then has to pick some wrapper. Zip or tar or... mbox! > > I still content that any change that removes the simple ability to > > fetch the list archive for use by the end user is a huge step > > backwards, regardless of the prettiness of whatever replaces it. > > It's not a question of "removing". This has always been a user > choice, and many lists use third-party services (mail-archive.com) or > non-pipermail archivers (MHonArc). It's just that Mailman 2 with its > bundled mbox-format archiver made it easy to provide by default. Effectively it is removal. If the python lists moved to mailman 3 and suddenly didn't have a "download the list archive", from an end user point of view that is removal. > If lots of users really prefer pipermail, for whatever reason, it's > not that hard to add by hand, and could be packaged on PyPI. Pipermail has two sides to it: the archive bundles and the message browser. Plenty of things do a richer message browse (MHonArc seemed quite nice when last I looked, many years ago). I'm not wedded to pipermail; from a browsing point of view this new thing may do well. And I've seen plenty of other simple things on the net that expose the discussion threads nicely. I'm wedded to having the archives available in a simple and easy to fetch form, in a format that is easily imported by many MUAs. Therefore: mbox. When Google Groups became a popular site to host mailing lists I became aghast at how crude and forumlike the interface is/was, and how the historical list archive was effectively unavailable. When Google Groups sucked in the usenet archive tapes I was full of hope, but I don't see those archives available for download either. We have all chosen our preferred tool to read lists (be it in "mail" or "usenet" forms); not being able to access old messages the same way as current messages feels... shortsighted. Cheers, -- Cameron Simpson <cs at zip.com.au> Archiving the net is like washing toilet paper! - Leader Kibo From richard.prosser at mail.com Mon Mar 31 14:09:48 2014 From: richard.prosser at mail.com (Richard Prosser) Date: Mon, 31 Mar 2014 13:09:48 +0100 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> Message-ID: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> I agree - links to search tools or at least explanations of how to use them would be very welcome. Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. Richard On 30 March 2014 05:02, Andrew Barnert <abarnert at yahoo.com> wrote: > From: Terry Reedy <tjreedy at udel.edu> > > Sent: Saturday, March 29, 2014 3:08 PM > > > > On 3/29/2014 4:54 PM, Ben Finney wrote: > >> Richard Prosser <richard.prosser at mail.com> writes: > >> > >>> I wanted to search this list (and other Python ones) but it is > > practically > >>> impossible to do so. > >> > >> The forum <URL:https://mail.python.org/mailman/listinfo/python-ideas> > >> has all its messages archived online for web browsing at > >> <URL:http://mail.python.org/pipermail/python-ideas/>. > >> > >> So you can use any web search tool to search it. A good search engine > >> like DuckDuckGo will let you constrain the search to a particular > subset > >> of the web: > >> > >> site:http://mail.python.org/pipermail/python-ideas foo bar baz > >> > >>> So why not add a tool like http://markmail.org/search/?q=python to > >>> https://mail.python.org/mailman/listinfo/python-ideas or the like? > >> > >> We have generally-appicable services that index the web and make much > >> better search tools. I don't often find a site's own custom search > > tool > >> to be sufficiently better to use it, when DuckDuckGo is available. > > > > This list and many others are also available through, archived at, and > > searchable at news.gmane.org. > > > How hard would it be to add appropriate links (or, better, custom search > boxes, or, worse but still helpful, a paragraph explaining how to search > each of those manually) to the mailing list archive pages? > > For example, if you link to this: > > > http://www.google.com/advanced_search?hl=en&fg=1&as_sitesearch=http%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas > > > ... then people just need to fill in their search terms on the Advanced > Search page that comes up. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/3c5e4303/attachment.html> From phd at phdru.name Mon Mar 31 14:23:56 2014 From: phd at phdru.name (Oleg Broytman) Date: Mon, 31 Mar 2014 14:23:56 +0200 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> Message-ID: <20140331122356.GA8976@phdru.name> On Mon, Mar 31, 2014 at 01:09:48PM +0100, Richard Prosser <richard.prosser at mail.com> wrote: > Better still I believe, abandon mailing lists altogether (because they can > clog up mail clients) and use forum/BB instead. OMG, please, no! Mailing lists are certainly much better than any forum. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From breamoreboy at yahoo.co.uk Mon Mar 31 14:51:05 2014 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 31 Mar 2014 13:51:05 +0100 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> Message-ID: <lhbobr$2mm$1@ger.gmane.org> On 31/03/2014 13:09, Richard Prosser wrote: > > Better still I believe, abandon mailing lists altogether (because they > can clog up mail clients) and use forum/BB instead. > To avoid clogging up your mail client I suggest gmane.comp.python.xyz, where in this case xyz is ideas. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com From barry at python.org Mon Mar 31 19:35:57 2014 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2014 13:35:57 -0400 Subject: [Python-ideas] Browser for mailing lists References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> Message-ID: <20140331133557.7508e3d3@anarchist.wooz.org> On Mar 29, 2014, at 03:29 PM, Richard Prosser wrote: >I wanted to search this list (and other Python ones) but it is practically >impossible to do so. Gmane is my personal favorite. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/0404385b/attachment.sig> From barry at python.org Mon Mar 31 19:44:52 2014 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2014 13:44:52 -0400 Subject: [Python-ideas] Browser for mailing lists References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> <20140330222234.GA14300@cskk.homeip.net> <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> Message-ID: <20140331134452.55a67550@anarchist.wooz.org> Discussing specifics of Mailman 3 and archives is off-topic for this list, so I won't post a huge amount of detail. Everyone is welcome to come to the mailman-developers list to further interact with us. On Mar 31, 2014, at 08:37 AM, Nick Coghlan wrote: >The pluggable archiver model in MM3 means HyperKitty itself doesn't need to >handle that usage model. However, I'm not aware of any current efforts to >create an MM3 archiver that is closer to a pure pipermail replacement (the >workflow you describe strikes me as being incredibly unusual these days, as >it requires either running your own mail server, or only accessing your >email from one device) I would certainly not start from the existing Pipermail code, unless you pine for the late '90s. While the possibility exists to allow for downloading the raw archives, we disable that in Mailman 2 by default because of the feature's abuse for email harvesters. My personal favorite combination for drive-by engagements, historical archive access, convenience, and inbox sanity is the NNTP API for Gmane. My primary MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have to explicitly opt-in to Gmane, once they do, it can import the archive. Countless times I've subscribed to a mailing list's newsgroup gateway, interacted with that community for a short time, and then unsubscribed. The architecture of MM3 allows, and I would someday love, to be able to offer an NNTP interface to a mailing list's archives. (Gmane does a lot more and we don't need to duplicate all of it.) If that's something you'd be interested in working on, come talk to us! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/617c4d5c/attachment-0001.sig> From barry at python.org Mon Mar 31 19:48:58 2014 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2014 13:48:58 -0400 Subject: [Python-ideas] Browser for mailing lists References: <87y4zrlzw3.fsf@uwakimon.sk.tsukuba.ac.jp> <20140331055214.GA3830@cskk.homeip.net> Message-ID: <20140331134858.4781ff08@anarchist.wooz.org> On Mar 31, 2014, at 04:52 PM, Cameron Simpson wrote: >I'm wedded to having the archives available in a simple and easy >to fetch form, in a format that is easily imported by many MUAs. >Therefore: mbox. I'm not sure that's really an effective use case. For example, python-list raw mbox archives are 156GB. I'm positive it's highly compressable, but still, why would you want to download some huge mbox like that? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/b95463fa/attachment.sig> From ericsnowcurrently at gmail.com Mon Mar 31 20:22:57 2014 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Mon, 31 Mar 2014 12:22:57 -0600 Subject: [Python-ideas] Making sys.std* thread-local In-Reply-To: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com> References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com> Message-ID: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com> On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee at berkeley.edu> wrote: > The recent discussion about lexical vs. dynamic scoping reminded me of the > following issue, that plagues e.g. the new contextlib.redirect_stdout > context manager: the standard file-like objects sys.std* are global instead > of thread-local, making their manipulation unsafe in multithreaded programs > (this issue is related to lexical vs dynamic scoping because in a language > with (optional) dynamic scoping, these objects can be made dynamically > scoped, thus solving the issue). How much of a problem is this in practice? What's your specific use case? I can't say that's I've ever had such a need, but then I haven't used contextlib.redirect_stdout() in a multi-threaded context. At that point I'd probably opt for using Python's logging system rather than sys.std*. > Of course, changing sys.std* to being thread-local would be backwards > incompatible, but perhaps some new objects, e.g. sys.thread_local_std*, > could be added to sys, with print() and related functions using > sys.thread_local_std* if it is set and sys.std* otherwise. Regardless of the validity of the proposal, coming up with an API to support thread-local sys.std* should be doable in a backward-compatible way, as you imply. It's just a question of how much complexity we can justify in the underlying implementation and in changes to sys.std* (if any). FYI, for 3.5 I plan on working on two things related to your proposal: * restructuring the sys module (or something like it); * adding an API for managing thread-local contexts (a la decimal). That's not to say that either will find enough support to actually land, but I will be working on them. :) -eric From solipsis at pitrou.net Mon Mar 31 20:27:03 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 31 Mar 2014 20:27:03 +0200 Subject: [Python-ideas] Browser for mailing lists References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> <20140330222234.GA14300@cskk.homeip.net> <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> <20140331134452.55a67550@anarchist.wooz.org> Message-ID: <20140331202703.36e33eb3@fsol> On Mon, 31 Mar 2014 13:44:52 -0400 Barry Warsaw <barry at python.org> wrote: > Discussing specifics of Mailman 3 and archives is off-topic for this list, so > I won't post a huge amount of detail. Everyone is welcome to come to the > mailman-developers list to further interact with us. I understand that, but is it actually required for simple feedback as a mere Web user? > My personal favorite combination for drive-by engagements, historical archive > access, convenience, and inbox sanity is the NNTP API for Gmane. My primary > MUA (claws-mail) is perfectly at home with NNTP and while mailing lists have > to explicitly opt-in to Gmane, once they do, it can import the archive. I also use claws-mail with Gmane NNTP, and I like it. However, it's not really adequate for casual browsing of mailing-lists you don't usually read, or for browsing of private mailing-lists with protected archives, or for giving out hyperlinks to individual mailing-list messages. About Hyperkitty, I have to say that the following kind of UI, while seemingly "nifty", is IMHO a massive step back from pipermail's ease of use: https://lists.stg.fedoraproject.org/archives/list/389-devel at lists.fedoraproject.org/ pipermail's dead simple threaded view is, ironically, the best UI I've ever seen for *browsing* mailing-lists (Google Groups are a disaster in that regard, and Hyperkitty seems to be heading in the same direction as Google Groups). I would really hate to have to deal with Hyperkitty's current UI on a daily basis. Regards Antoine. From njs at pobox.com Mon Mar 31 20:46:11 2014 From: njs at pobox.com (Nathaniel Smith) Date: Mon, 31 Mar 2014 19:46:11 +0100 Subject: [Python-ideas] Making sys.std* thread-local In-Reply-To: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com> References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com> <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com> Message-ID: <CAPJVwBm5FT998o3JBOdsOBJtsP89sZSF0am0iTd861xR4M37pA@mail.gmail.com> On Mon, Mar 31, 2014 at 7:22 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote: > * adding an API for managing thread-local contexts (a la decimal). Numpy might be another consumer of such an API, for similar reasons to decimal: http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html#numpy.errstate An interesting constraint from numpy's point of view is that we want to be able to read the TLS context very quickly (e.g., no allocs) from C code. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From barry at python.org Mon Mar 31 20:52:11 2014 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2014 14:52:11 -0400 Subject: [Python-ideas] Browser for mailing lists References: <CADiSq7f1cH+3pvZK+EShzQdVMT74pyTGFmMWbPapycCVAS7-AA@mail.gmail.com> <20140330222234.GA14300@cskk.homeip.net> <CADiSq7cq_DYxZvCb0soQUxqnhpHQS8GYYshMDLod2yxCKyNQYw@mail.gmail.com> <20140331134452.55a67550@anarchist.wooz.org> <20140331202703.36e33eb3@fsol> Message-ID: <20140331145211.32787335@anarchist.wooz.org> On Mar 31, 2014, at 08:27 PM, Antoine Pitrou wrote: >I understand that, but is it actually required for simple feedback as >a mere Web user? No, but your feedback won't necessarily reach the developers who can do something about it. >I also use claws-mail with Gmane NNTP, and I like it. However, it's not >really adequate for casual browsing of mailing-lists you don't usually read, >or for browsing of private mailing-lists with protected archives, or for >giving out hyperlinks to individual mailing-list messages. I like it for casual browsing, but general use the web ui for searching. It is a little tricky to then refer back to the NNTP interface for (IMO) more easily catching up on a thread, or responding. >pipermail's dead simple threaded view is, ironically, the best UI I've >ever seen for *browsing* mailing-lists (Google Groups are a disaster in >that regard, and Hyperkitty seems to be heading in the same direction >as Google Groups). I would really hate to have to deal with >Hyperkitty's current UI on a daily basis. I think Pipermail's ui has problems, not the least of which is the arbitrary splitting of threads across months, f.e. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/ccfe722b/attachment.sig> From ethan at stoneleaf.us Mon Mar 31 20:51:21 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 31 Mar 2014 11:51:21 -0700 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> Message-ID: <5339B929.1040000@stoneleaf.us> On 03/31/2014 05:09 AM, Richard Prosser wrote: > > I agree - links to search tools or at least explanations of how to use them would be very welcome. +1 > Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. Even better than that: use a decent mail client, and exercise one's mouse and keyboard to delete unnecessary context when replying to posts. -- ~Ethan~ From phd at phdru.name Mon Mar 31 23:05:27 2014 From: phd at phdru.name (Oleg Broytman) Date: Mon, 31 Mar 2014 23:05:27 +0200 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <5339B929.1040000@stoneleaf.us> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> <5339B929.1040000@stoneleaf.us> Message-ID: <20140331210527.GA23523@phdru.name> On Mon, Mar 31, 2014 at 11:51:21AM -0700, Ethan Furman <ethan at stoneleaf.us> wrote: > On 03/31/2014 05:09 AM, Richard Prosser wrote: > >Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. > > Even better than that: use a decent mail client, and exercise one's > mouse and keyboard to delete unnecessary context when replying to > posts. +many Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From antony.lee at berkeley.edu Mon Mar 31 23:15:16 2014 From: antony.lee at berkeley.edu (Antony Lee) Date: Mon, 31 Mar 2014 14:15:16 -0700 Subject: [Python-ideas] Making sys.std* thread-local In-Reply-To: <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com> References: <CAGRr6BFUifENjUVhFcLdJbp-KyhaqX9DwCHmmFbfAZ00wKsASQ@mail.gmail.com> <CALFfu7BM6KKh1WuOaG=16hMurBX2+KheQR8+R5RZgi+NXP1GUA@mail.gmail.com> Message-ID: <CAGRr6BE9VjXB68QvoOrOjfC0XfBtbuA-2A-qCZH9m2R65mojJA@mail.gmail.com> My specific use case came from embedding the IPython QtConsole ( http://ipython.org/ipython-doc/stable/interactive/qtconsole.html) into a Qt app. Specifically, when a user types "print('foo')" into the QtConsole, I expect 'foo' to be printed in the QtConsole; however when "print('foo')" is called from the application code, I expect 'foo' to be printed in __stdout__ (the original stdout, i.e. the terminal). An in fact, IPython achieves this through temporary stdout redirection while exec'ing user code ( https://github.com/ipython/ipython/blob/master/IPython/kernel/inprocess/ipkernel.py#L113 ). Obviously, this breaks down if application code tries to call print while redirection is taking place; more subtly, if user code (that is, typed by the user at the QtConsole) starts a new thread, then it would be nice if that new thread could inherit the current settings (that is, stdout being redirected to print to the QtConsole), which is not the case right now (delayed prints go to the terminal as the global stdout gets restored to __stdout__, not to the QtConsole). Perhaps a bit specific, but that was the original motivation for this request. Antony 2014-03-31 11:22 GMT-07:00 Eric Snow <ericsnowcurrently at gmail.com>: > On Thu, Feb 6, 2014 at 2:37 PM, Antony Lee <antony.lee at berkeley.edu> > wrote: > > The recent discussion about lexical vs. dynamic scoping reminded me of > the > > following issue, that plagues e.g. the new contextlib.redirect_stdout > > context manager: the standard file-like objects sys.std* are global > instead > > of thread-local, making their manipulation unsafe in multithreaded > programs > > (this issue is related to lexical vs dynamic scoping because in a > language > > with (optional) dynamic scoping, these objects can be made dynamically > > scoped, thus solving the issue). > > How much of a problem is this in practice? What's your specific use > case? I can't say that's I've ever had such a need, but then I > haven't used contextlib.redirect_stdout() in a multi-threaded context. > At that point I'd probably opt for using Python's logging system > rather than sys.std*. > > > Of course, changing sys.std* to being thread-local would be backwards > > incompatible, but perhaps some new objects, e.g. sys.thread_local_std*, > > could be added to sys, with print() and related functions using > > sys.thread_local_std* if it is set and sys.std* otherwise. > > Regardless of the validity of the proposal, coming up with an API to > support thread-local sys.std* should be doable in a > backward-compatible way, as you imply. It's just a question of how > much complexity we can justify in the underlying implementation and in > changes to sys.std* (if any). > > FYI, for 3.5 I plan on working on two things related to your proposal: > > * restructuring the sys module (or something like it); > * adding an API for managing thread-local contexts (a la decimal). > > That's not to say that either will find enough support to actually > land, but I will be working on them. :) > > -eric > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140331/6674e395/attachment.html> From ncoghlan at gmail.com Mon Mar 31 23:51:39 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 1 Apr 2014 07:51:39 +1000 Subject: [Python-ideas] Browser for mailing lists In-Reply-To: <5339B929.1040000@stoneleaf.us> References: <CAJvrtnsOFk1YCr1xO7dP7Rys7wYBn0UKPFMviegkF+1KHoNvZA@mail.gmail.com> <85d2h4ohdd.fsf@benfinney.id.au> <lh7ga5$c77$2@ger.gmane.org> <1396152153.12277.YahooMailNeo@web181001.mail.ne1.yahoo.com> <CAJvrtnv6cw5qscp=sh0TV1=DUK6WAbp=LFH6v-=riT-VQMZopQ@mail.gmail.com> <5339B929.1040000@stoneleaf.us> Message-ID: <CADiSq7fLMKVrAVQgd-Py320=QRZ=mE8eEW74+6SfYAWxHEMhHA@mail.gmail.com> On 1 Apr 2014 05:40, "Ethan Furman" <ethan at stoneleaf.us> wrote: > > On 03/31/2014 05:09 AM, Richard Prosser wrote: >> Better still I believe, abandon mailing lists altogether (because they can clog up mail clients) and use forum/BB instead. > > > Even better than that: use a decent mail client, and exercise one's mouse and keyboard to delete unnecessary context when replying to posts. No need to get personal folks. I use a lot of both mailing lists *and* web based communication interfaces, and see a place for both of them. Mailing lists, I prefer when I'm highly invested. Just as an RSS reader brings blogs I care about into a single interface, mailing lists do the same for collaborative communication. Appropriate use of filters and "mark all as read" then deals with the volume of traffic. However, mailing lists are *terrible* if you just want to drop in, ask one question & leave again. One possible answer to that is AskBot (a self-hosted site-specific, Stack Overflow style Q&A forum), but the MM3/HyperKitty web gateway idea is aimed at making the existing *already invested* list participants more readily available to new users without having to ask those existing participants to change their reading habits and look in a new place for questions. In the meantime, making GMane more discoverable by suggesting it as an alternative to subscription in the various list descriptions sounds reasonable to me. Cheers, Nick. > > -- > ~Ethan~ > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140401/8dcc5a60/attachment.html>