Mailman 3 January 2015 - Python-ideas

Re: [Python-ideas] PEP 485: A Function for testing approximate equality
by Chris Barker Jan. 27, 2015

Jan. 27, 2015

Sorry, This slipped off list -- bringin it back. On Mon, Jan 26, 2015 at 12:40 PM, Paul Moore <p.f.moore(a)gmail.com> wrote: > > Any of the approaches on the table will do something reasonable in this > > case: > > > > In [4]: is_close_to.is_close_to(sum([0.1]*10), 1) > > testing: 0.9999999999999999 1 > > Out[4]: True > > Yes, but that's not my point. I was responding to Steven's comment > that having 2 different types of tolerance isn't "arcana", by pointing > out that I find even stuff as simple as multiplication vs cumulative > addition confusing. And I should note that I was (many years ago!) a > maths graduate and did some numerical maths courses, so this stuff > isn't completely unknown to me. Right it can be arcane -- which is why I want this function, and why we want it to do something "sane" most of the time, be default. > Note that the 1e-8 default I chose (which I am not committed to) is not > > ENTIRELY arbitrary -- it's about half the digits carried by a python > float > > (double) -- essentially saying the values are close to about half of the > > precision available. And we are constrained here, the options are between > > 0.1 (which would be crazy, if you ask me!) and 1e-14 -- any larger an it > > would meaningless, and any smaller, and it would surpass the precision > of a > > python float. PIcking a default near the middle of that range seems quite > > sane to me. > > Sorry, that means nothing to me. Head exploding time again :-) Darn -- I'll try again -- with a relative tolerence, two values are only going to be close if their exponent is within one of each-other. So what you are setting is how many digits of the mantisa you care about. a toleranc eof 0.1 would be about one digit, and a tolerance of 1e-15 would be 15 digits. Python floats carry about 15 digits -- so the relative tolerance has to be betwwen 1e-1 and 1e-15 -- nothign else is useful or makes sense. So I put it in the middle: 1e-8 > > This is quite different than setting a value for an absolute tolerance -- > > saying something is close to another number if the difference is less > than > > 1e-8 would be wildly inappropriate when the smallest numbers a float can > > hold are on order of 1e-300! > > On the other hand, I find this completely obvious. (Well, mostly - > don't the gaps between the representable floats increase as the > magnitude gets bigger, so an absolute tolerance of 1e-8 might be > entirely reasonable when the numbers are sufficiently high? sure it would -- that's the point -- what makes sense as an absolute tolerance depends entirely on the magnitude of the numbers -- since we don't know the magnitude of the numbers someone may use, we can't set a reasonable default. > arcana, maybe, not it's not a floating point issue -- X% of zero is zero > > absolutely precisely. > > But the "arcana" I was talking about is that a relative error of X% > could be X% of the value under test, of the expected value, of their > average, or something else. Ahh! -- which is exactly the point I think some of us are making -- defining X% error relative to the "expected" value is the simplest and most straightforward to explain. That's the primary reason I prefer it. And only *one* of those values is zero, so > whether X% is a useful value is entirely dependent on the definition. > not sure what you meant here, but actually relative error goes to heck if either value is zero, and with any of the definitions we are working with. So X% is useful for any value except if one of the values is zero. > And how relative errors are defined *is* floating point arcana (I can > picture the text book page now, and it wasn't simple...) > semantics here -- defining a realtive error can be done with pure real numbers -- computing it can get complex with floating point. > But back to a point made earlier -- the idea here is to provide something > > better than naive use of > > > > x == y > <snip> > I still wonder whether "naive use of equality" is much of a target, > though. There are only two use cases that have been mentioned so far. > Testing is not about equality, because we're replacing > assertAlmostEqual. And when someone is doing an iterative algorithm, > they are looking for convergence, i.e. within a given range of the > answer. So neither use case would be using an equality test anyway. > well, the secondary target is a better (or more flexible) assertAlmostEqual. It is not suitable for arbitrarily large or small numbers, and particularly not numbers with a range of magnitudes -- a relative difference test is much needed. I'm not sure I follow your point, but I will say that if Nathaniel has > seen a lot of use cases for assertAlmostEqual that can't be easily > handled with the new function, then something is badly wrong. Well, I"m not suggesting that we replace assertAlmostEqual -- but rather augment it. IN fact, assertAlmostEqual is actually a an absolute tolerance test (expressed in terms f decimal places). That is the right thing, and the only right thing to use when you want to compare to zero. What I'm proposing a relative tolerance test, which is not the right thing to use for comparing to zero, but is the right thing to use when comparing numbers of varying magnitude. > There > aren't enough good use cases that we can reasonably decide to reject > any of them as out of scope, I've lost track of what we might be rejecting. The whole symmetric -- non symmetric argument really is bike shedding -- in the context of "better than ==" or "different but as good as assetAlmostEqual" -- any of them are just fine. so really all wer aare left with is defaults -- also bike-shedding, except for the default for the zero test, and there are really two options there: use 0.0 for abs_tolerance, and have it fail for any test against zero unless the user specifies something approporate for their use case. or use a SOME default, either for abs_tolerance or zero_tolerance, and make an assumption about the ofder of magnitide of the lielky results, so that it will "jsut work" for tests against zero. Maybe something small relative to one (like 1e-8) would be OK, but that concerns me -- I think you'd get false positives for small numbers which is worse that false negatives for all comparisons to zero. > 1e-8 -- but you already know that ;-) -- anything between 1e-8 and 1e-12 > > would be fine with me. > > TBH all I care about in this context is that there must be 2 values x > and y for which is_close(x,y) == True and x != y. everything on the table will do that. I'm tempted to > strengthen that to "for all y there must be at least 1 x such that..." > but it's possible that's *too* strong and it can't be achieved. > I think we could say that for all y except 0.0 -- and even zero if an abs_tolerance is greater than zero is set. > Basically, the default behaviour needs to encompass what I believe is > most people's intuition - that "close" is a proper superset of > "equal". > A good reason not to have all defaults be zero -- I don't think we need a function that doesn't work at all with default values. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

Syntax: 'return: ...' expressions
by Yawar Amin Jan. 25, 2015

Jan. 25, 2015

Note: I'm reposting this here after posting on the /r/Python reddit as I've realised this is a better venue: http://www.reddit.com/r/Python/comments/2ra8yc/return_expressions/ I have an idea for a new syntax element that combines any number of statements and expressions into a single expression. I'm partial to calling it a 'return' expression because that should be compatible with existing code--there's guaranteed to be no existing code of the form 'return:'. But we could call it 'do' or 'val' or 'expr' or whatever. expression ::= ... | return_expr return_expr ::= "return" ":" statement* expression+ We can define all expression-oriented syntax in terms of return expressions. I.e., wrap up all statement-oriented syntax inside an expression. E.g., x = ( return: if y == 1: z = 'a' elif y == 2: z = 'b' elif y == 3: z = 'c' else: z = 'd' z ) Or a single-line example, x = return: print "Blah"; 5 I know, it's the Python programmer's dreaded multi-line expression. Note here that I'm not proposing any change to indentation rules. I'm relying on parens to relax the rules. There's precedent for using parens in new kinds of expressions--e.g. generator expressions. So the usage shouldn't look alien in Python code. Now the controversy. We can use a return expression to get a multi-line, multi-statement, lambda, as long as lambda only cares that its body is a single exprssion. Which I believe is the case, e.g. this is valid Python today: f = lambda x: ( x, x + 1, x + 2 ) Anyway, an imaginary lambda example: txtName.text.signal.map( # Need to wrap the return expr in parens to separate the lambdas. lambda s: ( return: l = len(s) l >= 5 ), # The next argument, a lambda without a return expr, happily lives # alongside. lambda err: notifications.send(err) ).subscribe( # Parens not needed here, only a single argument. lambda flag: return: pnlMain.back_color = "green" if flag else "red" # Must end with an expr, unlike a normal function. Think of it # this way: we're 'inside' a return, we _have to_ return # some value. None ) I believe 'return: ...' is an unintrusive and versatile solution. It's _not_ meant to just forcefully shoehorn full functionality into lambdas as I believe I show above; it doesn't break compatibility; it doesn't require any indent/whitespace rule changes; and it's guaranteed to not affect _any_ existing code. Regards, Yawar P.S. Relevant: https://groups.google.com/d/msg/python-ideas/kYQbvsmyM-4/ufU26RPTjLQJ 'A much better idea would be to find a way to make all compound statements into expressions, future-proofing the decision and avoiding the redundancy between "compound statement statement" and "compound statement expression".' https://groups.google.com/d/msg/python-ideas/EQQq3--DDu0/UTcfI34sVwwJ 'Something that has started to annoy me in the last couple of years is the fact that most Python control statements cannot be used as expressions. I feel this is a pretty deep limitation and personally I don't feel it's well-justified.'

10 41

PEP 484 (Type Hints) -- first draft round
by Guido van Rossum Jan. 24, 2015

Jan. 24, 2015

After a long editing process we've got PEP 484 <https://www.python.org/dev/peps/pep-0484/> (Type Hints) ready for your review. This is by no means final, and several areas are either open (to be resolved in a later draft) or postponed (to a different PEP altogether). But there's enough meat that I think we can start having the discussion. Please also see PEP 483 <https://www.python.org/dev/peps/pep-0483/> (The Theory of Type Hint; copied and reformatted from the original Quip document that I posted just before last Christmas) and PEP 482 <https://www.python.org/dev/peps/pep-0482/> (Literature Overview for Type Hints, by Łukasz). Those are informational PEPs though; the actual spec is focused in PEP 484 (the only one on the Standards Track). As I said earlier, I hope to have a rough consensus before PyCon <https://us.pycon.org/2015/> and working code (just the typing.py module <https://github.com/ambv/typehinting/tree/master/prototyping>, really) committed to CPython before the last 3.5 alpha <https://www.python.org/dev/peps/pep-0478/>. Here is the raw text of PEP 484. Fire away!! PEP: 484 Title: Type Hints Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum <guido(a)python.org>, Jukka Lehtosalo < jukka.lehtosalo(a)iki.fi>, Łukasz Langa <lukasz(a)langa.pl> Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Sep-2014 Post-History: 16-Jan-2015 Resolution: Abstract ======== This PEP introduces a standard syntax for type hints using annotations on function definitions. The proposal is strongly inspired by mypy [mypy]_. The theory behind type hints and gradual typing is explained in PEP 483. Rationale and Goals =================== PEP 3107 added support for arbitrary annotations on parts of a function definition. Although no meaning was assigned to annotations then, there has always been an implicit goal to use them for type hinting, which is listed as the first possible use case in said PEP. This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and performance optimizations utilizing type information. Type Definition Syntax ====================== The syntax leverages PEP 3107-style annotations with a number of extensions described in sections below. In its basic form, type hinting is used by filling function annotations with classes:: def greeting(name: str) -> str: return 'Hello ' + name This denotes that the expected type of the ``name`` argument is ``str``. Analogically, the expected return type is ``str``. Subclasses of a specified argument type are also accepted as valid types for that argument. Abstract base classes, types available in the ``types`` module, and user-defined classes may be used as type hints as well. Annotations must be valid expressions that evaluate without raising exceptions at the time the function is defined. In addition, the needs of static analysis require that annotations must be simple enough to be interpreted by static analysis tools. (This is an intentionally somewhat vague requirement.) .. FIXME: Define rigorously what is/isn't supported. When used as an annotation, the expression ``None`` is considered equivalent to ``NoneType`` (i.e., ``type(None)`` for type hinting purposes. Type aliases are also valid type hints:: integer = int def retry(url: str, retry_count: integer): ... New names that are added to support features described in following sections are available in the ``typing`` package. Callbacks --------- Frameworks expecting callback functions of specific signatures might be type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. Examples:: from typing import Any, AnyArgs, Callable def feeder(get_next_item: Callable[[], Item]): ... def async_query(on_success: Callable[[int], None], on_error: Callable[[int, Exception], None]): ... def partial(func: Callable[AnyArgs, Any], *args): ... Since using callbacks with keyword arguments is not perceived as a common use case, there is currently no support for specifying keyword arguments with ``Callable``. Generics -------- Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. Example:: from typing import Mapping, Set def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]): ... Generics can be parametrized by using a new factory available in ``typing`` called ``TypeVar``. Example:: from typing import Sequence, TypeVar T = TypeVar('T') # Declare type variable def first(l: Sequence[T]) -> T: # Generic function return l[0] In this case the contract is that the returning value is consistent with the elements held by the collection. ``TypeVar`` supports constraining parametric types to classes with any of the specified bases. Example:: from typing import Iterable X = TypeVar('X') Y = TypeVar('Y', Iterable[X]) def filter(rule: Callable[[X], bool], input: Y) -> Y: ... .. FIXME: Add an example with multiple bases defined. In the example above we specify that ``Y`` can be any subclass of Iterable with elements of type ``X``, as long as the return type of ``filter()`` will be the same as the type of the ``input`` argument. .. FIXME: Explain more about how this works. Forward references ------------------ When a type hint contains names that have not been defined yet, that definition may be expressed as a string, to be resolved later. For example, instead of writing:: def notify_by_email(employees: Set[Employee]): ... one might write:: def notify_by_email(employees: 'Set[Employee]'): ... .. FIXME: Rigorously define this. Defend it, or find an alternative. Union types ----------- Since accepting a small, limited set of expected types for a single argument is common, there is a new special factory called ``Union``. Example:: from typing import Union def handle_employees(e: Union[Employee, Sequence[Employee]]): if isinstance(e, Employee): e = [e] ... A type factored by ``Union[T1, T2, ...]`` responds ``True`` to ``issubclass`` checks for ``T1`` and any of its subclasses, ``T2`` and any of its subclasses, and so on. One common case of union types are *optional* types. By default, ``None`` is an invalid value for any type, unless a default value of ``None`` has been provided in the function definition. Examples:: def handle_employee(e: Union[Employee, None]): ... As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``; for example, the above is equivalent to:: from typing import Optional def handle_employee(e: Optional[Employee]): ... An optional type is also automatically assumed when the default value is ``None``, for example:: def handle_employee(e: Employee = None): ... This is equivalent to:: def handle_employee(e: Optional[Employee] = None): ... .. FIXME: Is this really a good idea? A special kind of union type is ``Any``, a class that responds ``True`` to ``issubclass`` of any class. This lets the user explicitly state that there are no constraints on the type of a specific argument or return value. Platform-specific type checking ------------------------------- In some cases the typing information will depend on the platform that the program is being executed on. To enable specifying those differences, simple conditionals can be used:: from typing import PY2, WINDOWS if PY2: text = unicode else: text = str def f() -> text: ... if WINDOWS: loop = ProactorEventLoop else: loop = UnixSelectorEventLoop Arbitrary literals defined in the form of ``NAME = True`` will also be accepted by the type checker to differentiate type resolution:: DEBUG = False ... if DEBUG: class Tracer: <verbose implementation> else: class Tracer: <dummy implementation> For the purposes of type hinting, the type checker assumes ``__debug__`` is set to ``True``, in other words the ``-O`` command-line option is not used while type checking. Compatibility with other uses of function annotations ----------------------------------------------------- A number of existing or potential use cases for function annotations exist, which are incompatible with type hinting. These may confuse a static type checker. However, since type hinting annotations have no run time behavior (other than evaluation of the annotation expression and storing annotations in the ``__annotations__`` attribute of the function object), this does not make the program incorrect -- it just makes it issue warnings when a static analyzer is used. To mark portions of the program that should not be covered by type hinting, use the following: * a ``@no_type_checks`` decorator on classes and functions * a ``# type: ignore`` comment on arbitrary lines .. FIXME: should we have a module-wide comment as well? Type Hints on Local and Global Variables ======================================== No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP. To help with type inference in complex cases, a comment of the following format may be used:: x = [] # type: List[Employee] In the case where type information for a local variable is needed before if was declared, an ``Undefined`` placeholder might be used:: from typing import Undefined x = Undefined # type: List[Employee] y = Undefined(int) If type hinting proves useful in general, a syntax for typing variables may be provided in a future Python version. Explicit raised exceptions ========================== No support for listing explicitly raised exceptions is being defined by this PEP. Currently the only known use case for this feature is documentational, in which case the recommendation is to put this information in a docstring. The ``typing`` package ====================== To open the usage of static type checking to Python 3.5 as well as older versions, a uniform namespace is required. For this purpose, a new package in the standard library is introduced called ``typing``. It holds a set of classes representing builtin types with generics, namely: * Dict, used as ``Dict[key_type, value_type]`` * List, used as ``List[element_type]`` * Set, used as ``Set[element_type]``. See remark for ``AbstractSet`` below. * FrozenSet, used as ``FrozenSet[element_type]`` * Tuple, used as ``Tuple[index0_type, index1_type, ...]``. Arbitrary-length tuples might be expressed using ellipsis, in which case the following arguments are considered the same type as the last defined type on the tuple. It also introduces factories and helper members needed to express generics and union types: * Any, used as ``def get(key: str) -> Any: ...`` * Union, used as ``Union[Type1, Type2, Type3]`` * TypeVar, used as ``X = TypeVar('X', Type1, Type2, Type3)`` or simply ``Y = TypeVar('Y')`` * Undefined, used as ``local_variable = Undefined # type: List[int]`` or ``local_variable = Undefined(List[int])`` (the latter being slower during runtime) * Callable, used as ``Callable[[Arg1Type, Arg2Type], ReturnType]`` * AnyArgs, used as ``Callable[AnyArgs, ReturnType]`` * AnyStr, equivalent to ``TypeVar('AnyStr', str, bytes)`` All abstract base classes available in ``collections.abc`` are importable from the ``typing`` package, with added generics support: * ByteString * Callable * Container * Hashable * ItemsView * Iterable * Iterator * KeysView * Mapping * MappingView * MutableMapping * MutableSequence * MutableSet * Sequence * Set as ``AbstractSet``. This name change was required because ``Set`` in the ``typing`` module means ``set()`` with generics. * Sized * ValuesView * Mapping The library includes literals for platform-specific type hinting: * PY2 * PY3, equivalent to ``not PY2`` * WINDOWS * UNIXOID, equivalent to ``not WINDOWS`` The following types are available in the ``typing.io`` module: * IO * BinaryIO * TextIO The following types are provided by the ``typing.re`` module: * Match and Pattern, types of ``re.match()`` and ``re.compile()`` results As a convenience measure, types from ``typing.io`` and ``typing.re`` are also available in ``typing`` (quoting Guido, "There's a reason those modules have two-letter names."). The place of the ``typing`` module in the standard library ---------------------------------------------------------- .. FIXME: complete this section Usage Patterns ============== The main use case of type hinting is static analysis using an external tool without executing the analyzed program. Existing tools used for that purpose like ``pyflakes`` [pyflakes]_ or ``pylint`` [pylint]_ might be extended to support type checking. New tools, like mypy's ``mypy -S`` mode, can be adopted specifically for this purpose. Type checking based on type hints is understood as a best-effort mechanism. In other words, whenever types are not annotated and cannot be inferred, the type checker considers such code valid. Type errors are only reported in case of explicit or inferred conflict. Moreover, as a mechanism that is not tied to execution of the code, it does not affect runtime behaviour. In other words, even in the case of a typing error, the program will continue running. The implementation of a type checker, whether linting source files or enforcing type information during runtime, is out of scope for this PEP. .. FIXME: Describe stub modules. .. FIXME: Describe run-time behavior of generic types. Existing Approaches =================== PEP 482 lists existing approaches in Python and other languages. Is type hinting Pythonic? ========================= Type annotations provide important documentation for how a unit of code should be used. Programmers should therefore provide type hints on public APIs, namely argument and return types on functions and methods considered public. However, because types of local and global variables can be often inferred, they are rarely necessary. The kind of information that type hints hold has always been possible to achieve by means of docstrings. In fact, a number of formalized mini-languages for describing accepted arguments have evolved. Moving this information to the function declaration makes it more visible and easier to access both at runtime and by static analysis. Adding to that the notion that “explicit is better than implicit”, type hints are indeed *Pythonic*. Acknowledgements ================ This document could not be completed without valuable input, encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson Vitousek, Andrey Vlasovskikh, and Radomir Dopieralski. Influences include existing languages, libraries and frameworks mentioned in PEP 482. Many thanks to their creators, in alphabetical order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer, Raoul-Gabriel Urma, and Julien Verlaguet. References ========== .. [mypy] http://mypy-lang.org .. [pyflakes] https://github.com/pyflakes/pyflakes/ .. [pylint] http://www.pylint.org Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- --Guido van Rossum (python.org/~guido)

33 155

math.nexttoward, math.ulp, etc.
by Andrew Barnert Jan. 24, 2015

Jan. 24, 2015

Some of this has come up multiple times in the discussion of the is_close_to PEP and the math.nan idea, but I think it's actually a separate issue. I think it might be worth adding some of the following to math: * isnormal (like C) * nextafter/nexttoward (like C) and/or nextplus/nextminus (like decimal) * bitdifference (inverse of nexttoward) * ulp (like Java) * signbit (like C) * isqnan, issnan (like decimal) * makenan(payload, quiet=True, sign=False) and nanpayload(f) * as_tuple and from_tuple (like decimal--although of course the tuple isn't quite the same) In some cases, it would be perfectly reasonable for these functions to only be available on, e.g., Windows (maybe even only NT, not CE) plus any platform that confirms to either C99 or POSIX-2001 (which includes almost all non-Windows platforms you're likely to run into nowadays), or only on platforms that use IEEE 754-2008 or Intel-style IEEE 754-1985 (again, almost everything), etc., as appropriate; no need to try to implement them all fully generally from scratch for any C89 platform just in case someone decides to build Python 3.5 for 68k A/UX or something.

1 0

OT or going meta: How this mailing list is perceived
by Chris Angelico Jan. 24, 2015

Jan. 24, 2015

Some of you here probably saw/heard David Beazley at PyCon 2012, but I'm only just now finding out about this thanks to the magic of the internet. The talk is long, but I just want to highlight this one little sequence (about 20 seconds)... and how python-ideas is depicted. http://youtu.be/6_-5XZzJyt0?t=3m40s :) ChrisA

1 0

Should bool continue to inherit from int?
by Michael Mitchell Jan. 23, 2015

Jan. 23, 2015

PEP 285 <http://legacy.python.org/dev/peps/pep-0285> provides some justification for why arithmetic operations should be valid on bools and why bool should inherit from int (see points (4) and (6) respectively). Since it's been 12 years (it's possible this has been brought up again between now and then), I thought it could be worthwhile to take another look. I am mostly interested in a resurveying of the questions: 1) Would it still be very inconvenient to implement bools separately from ints? 2) Do most Python users still agree that arithmetic operations should be supported on booleans? Follow-up thought: Something I noticed is that with PEP 484 <https://www.python.org/dev/peps/pep-0484/> (Type Hints) specified as is, there would be no way to statically verify that a function will only operate on ints and not bools. An example would be a function that can only operate on integer values in a JSON dict created by the builtin `json` module (using the default decoder) cannot exist, as that function could operate on the boolean values of the dict.

14 18

Keyword/Symbol literals
by Matthew Rocklin Jan. 22, 2015

Jan. 22, 2015

It would be nice to have literal keywords/symbols. By this I mean, terms that look like words but are not previously defined variables. *Prior art* Some other languages prefix by colon, :foo, or use a backtick, `foo. julia> :foo :foo julia> typeof(:foo) Symbol user=> :foo :foo user=> (type :foo) clojure.lang.Keyword user=> `foo user/foo user=> (type `foo) clojure.lang.Symbol *Why is this useful?* One use case in NumPy/Pandas use is to specify columns or fields of data without resorting to strings. E.g. df = pandas.load(...) df.sort(:name) *What do people do now?* Currently people use auto-generated attributes df.sort(df.name) or strings df.sort('name') auto-generated attributes work great until you want to use chained expressions df.change_dataframe().sort(some_new_column_not_in_df) strings work but feel unpleasant *Prior discussion?* This is a common language construct so my guess is that it has come up before. Sadly Google searching the terms *keywords* and *symbols* results it a lot of unrelated material. Can anyone point me to previous discussion? There are clearly issues with using :foo in that it overlaps with slice syntax, presumably some other character could be pressed into service if this was found worthwhile. I can come up with more motivating use cases if desired. Best, -Matthew

17 29

Floating point "closeness" Proposal Outline
by Chris Barker Jan. 21, 2015

Jan. 21, 2015

OK folks, There has been a lot of chatter about this, which I think has served to provide some clarity, at least to me. However, I'm concerned that the upshot, at least for folks not deep into the discussion, will be: clearly there are too many use-case specific details to put any one thing in the std lib. But I still think we can provide something that is useful for most use-cases, and would like to propose what that is, and what the decision points are: A function for the math module, called somethign like "is_close", "approx_equal", etc. It will compute a relative tolerance, with a default maybe around 1-e12, with the user able to specify the tolerance they want. Optionally, the user can specify an "minimum absolute tolerance", it will default to zero, but can be set so that comparisons to zero can be handled gracefully. The relative tolerance will be computed from the smallest of the two input values, so as to get symmetry : is_close(a,b) == is_close(b,a). (this is the Boost "strong" definition, and what is used by Steven D'Aprano's code in the statistics test module) Alternatively, the relative error could be computed against a particular one of the input values (the second one?). This would be asymmetric, but be more clear exactly how "relative" is defined, and be closer to what people may expect when using it as a "actual vs expected" test. --- "expected" would be the scaling value. If the tolerance is small, it makes very little difference anyway, so I'm happy with whatever consensus moves us to. Note that if we go this way, then the parameter names should make it at least a little more clear -- maybe "actual" and "expected", rather than x and y or a and b or... and the function name should be something like is_close_to, rather than just is_close. It will be designed for floating point numbers, and handle inf, -inf, and NaN "properly". But is will also work with other numeric types, to the extent that duck typing "just works" (i.e. division and comparisons all work). complex numbers will be handled by: is_close(x.real, y.real) and is_close(x.imag, y.imag) (but i haven't written any code for that yet) It will not do a simple absolute comparison -- that is the job of a different function, or, better yet, folks just write it themselves: abs(x - y) <= delta really isn't much harder to write than a function call: absolute_diff(x,y,delta) Here is a gist with a sample implementation: https://gist.github.com/PythonCHB/6e9ef7732a9074d9337a I need to add more tests, and make the test proper unit tests, but it's a start. I also need to see how it does with other data types than float -- hopefully, it will "just work" with the core set. I hope we can come to some consensus that something like this is the way to go. -Chris On Sun, Jan 18, 2015 at 11:27 AM, Ron Adam <ron3200(a)gmail.com> wrote: > > > On 01/17/2015 11:37 PM, Chris Barker wrote: > >> (Someone claimed that 'nothing is close to zero'. This is >> nonsensical both in applied math and everyday life.) >> >> >> I'm pretty sure someone (more than one of use) asserted that "nothing is >> *relatively* close to zero -- very different. >> > > Yes, that is the case. > > > And I really wanted a way to have a default behavior that would do a >> reasonable transition to an absolute tolerance near zero, but I no longer >> thing that's possible. (numpy's implimentaion kind of does that, but it is >> really wrong for small numbers, and if you made the default min_tolerance >> the smallest possible representable number, it really wouldn't be useful. >> > > I'm going to try to summarise what I got out of this discussion. Maybe it > will help bring some focus to the topic. > > I think there are two case's to consider. > > # The most common case. > rel_is_good(actual, expected, delta) # value +- %delta. > > # Testing for possible equivalence? > rel_is_close(value1, value2, delta) # %delta close to each other. > > I don't think they are quite the same thing. > > rel_is_good(9, 10, .1) --> True > rel_is_good(10, 9, .1) --> False > > rel_is_close(9, 10, .1) --> True > rel_is_close(10, 9, .1) --> True > > > In the "is close" case, it shouldn't matter what order the arguments are > given. The delta is the distance from the larger number the smaller number > is. (of the same sign) > > So when calculating the relative error from two values, you want it to be > consistent with the rel_is_close function. > > rel_is_close(a, b, delta) <---> rel_err(a, b) <= delta > > And you should not use the rel_err function in the rel_is_good function. > > > > The next issue is, where does the numeric accuracy of the data, > significant digits, and the languages accuracy (ULPs), come into the > picture. > > My intuition.. I need to test the idea to make a firmer claim.. is that in > the case of is_good, you want to exclude the uncertain parts, but with > is_close, you want to include the uncertain parts. > > Two values "are close" if you can't tell one from the other with > certainty. The is_close range includes any uncertainty. > > A value is good if it's within a range with certainty. And this excludes > any uncertainty. > > This is where taking in consideration of an absolute delta comes in. The > minimum range for both is the uncertainty of the data. But is_close and > is_good do different things with it. > > Of course all of this only applies if you agree with these definitions of > is_close, and is_good. ;) > > Cheers, > Ron > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

13 34

Re: [Python-ideas] What's going on with PEP 448 - Additional Unpacking Generalizations ?
by Neil Girdhar Jan. 20, 2015

Jan. 20, 2015

Fixed it on my system! (OS X Yosemite) http://hamelot.co.uk/programming/osx-gcc-dispatch_block_t-has-not-been-decl… Okay, I'll get working on this patch soon. Thanks, Neil On Mon, Jan 19, 2015 at 5:39 PM, Chris Angelico <rosuav(a)gmail.com> wrote: > On Tue, Jan 20, 2015 at 9:22 AM, Chris Angelico <rosuav(a)gmail.com> wrote: > > Skipped tests are normal; if you're building on a system that doesn't > > have GDB, it'll skip test_gdb, because there's nothing to test. The > > failing tests, though, are likely to be a problem. What environment > > are you building on? Are you experienced with building Python from > > source? > > FWIW, here's my 'make test' summary. Debian Wheezy, Python 3.5 (tip), > amd64 architecture. > > 379 tests OK. > 1 test altered the execution environment: > test_warnings > 11 tests skipped: > test_devpoll test_gdb test_kqueue test_msilib test_ossaudiodev > test_startfile test_tk test_ttk_guionly test_winreg test_winsound > test_zipfile64 > > ChrisA >

3 14

What's going on with PEP 448 - Additional Unpacking Generalizations ?
by Neil Girdhar Jan. 19, 2015

Jan. 19, 2015

It says "standards track", but it doesn't seem to work in Python 3.4. I am really looking forward to this! Any chance we can get a from __future__ for inclusion in 3.4? Or maybe it can make it into 3.5? Best, Neil

3 3