Mailman 3 March 2020 - Python-ideas

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 17, 2022

Feb. 17, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, … [View More]

17 32

Make fnmatch.filter accept a tuple of patterns
by Andre Delfino March 26, 2021

March 26, 2021

Frequently, while globbing, one needs to work with multiple extensions. I’d like to propose for fnmatch.filter to handle a tuple of patterns (while preserving the single str argument functionality, alas str.endswith), as a first step for glob.i?glob to accept multiple patterns as well. Here is the implementation I came up with: https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-mu… If this is deemed reasonable, I’ll write tests and documentation updates. Any opinion?

9 15

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman March 11, 2021

March 11, 2021

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/… [View More]clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman [View Less]

12 35

Add command-line option to unittest for enabling post-mortem debugging
by Dominik Vilsmeier Jan. 9, 2021

Jan. 9, 2021

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-… [View More]

1 1

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Compound with .. else statement
by Jimmy Thrasibule Sept. 9, 2020

Sept. 9, 2020

Hi, In Python, there are multiple [compound statements](https://docs.python.org/3/reference/compound_stmts.html) with the `else` keyword. For example: ``` for x in iterable: if x == sentinel: break else: print("Sentinel not found.") ``` or: ``` try: do_something_sensitive() except MyError: print("Oops!") else: print("We're all safe.") ``` In my situation, I would like to mix the `with` statement with `else`. In this case I would like that if no exception is … [View More]

6 10

Make `del x` an expression evaluating to `x`
by Eric Wieser June 12, 2020

June 12, 2020

TL;DR: should we make `del x` an expression that returns the value of `x`. ## Motivation I noticed yesterday that `itertools.combinations` has an optimization for when the returned tuple has no remaining ref-counts, and reuses it - namely, the following code: >>> for v in itertools.combinations([1, 2, 3], 1): ... print(id(v)) ... del v # without this, the optimization can't take place 2500926199840 2500926199840 2500926199840 will print the same id … [View More]

15 50

PYTHONLOGGING env variable
by Bar Harel June 6, 2020

June 6, 2020

Another idea I've had that may be of use: PYTHONLOGGING environment variable. Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level. Another option would be that -x dev or a different -x logging will initialize basic config. Will be useful mostly for debugging purposes instead of temporarily modifying the code. Kinda surprised it doesn't exist tbh. Bar Harel

9 22

Explicitly defining a string buffer object (aka StringIO += operator)
by Paul Sokolovsky April 11, 2020

April 11, 2020

Hello, 1. Intro -------- It is a well-known anti-pattern to use a string as a string buffer, to construct a long (perhaps very long) string piece-wise. A running example is: buf = "" for i in range(50000): buf += "foo" print(buf) An alternative is to use a buffer-like object explicitly designed for incremental updates, which for Python is io.StringIO: buf = io.StringIO() for i in range(50000): buf.write("foo") print(buf.getvalue()) As can be seen, this requires changing the way … [View More]buffer is constructed (usually in one place), the way buffer value is taken (usually in one place), but more importantly, it requires changing each line which adds content to a buffer, and there can be many of those for more complex algorithms, leading to a code less clear than the original code, requiring noise-like changes, and complicating updates to 3rd-party code which needs optimization. To address this, this RFC proposes to add an __iadd__ method (i.e. implementing "+=" operator) to io.StringIO and io.BytesIO objects, making it the exact alias of .write() method. This will allow for the code very parallel to the original str-using code: buf = io.StringIO() for i in range(50000): buf += "foo" print(buf.getvalue()) This will still require updates for buffer construction/getting value, but that's usually 2 lines. But it will leave the rest of code intact, and not obfuscate the original content construction algorithm. 2. Performance Discussion ------------------------- The motivation for this change (of promoting usage of io.StringIO, by making it look&feel more like str) is performance. But is it really a problem? Turns out, it is such a pervasive anti-pattern, that recent versions on CPython3 have a special optimization for it. Let's use following script for testing: --------- import timeit import io def string(): sb = u"" for i in range(50000): sb += u"a" def strio(): sb = io.StringIO() for i in range(50000): sb.write(u"a") print(timeit.timeit(string, number=10)) print(timeit.timeit(strio, number=10)) --------- With CPython3.6 the result is: $ python3.6 str_iadd-vs-StringIO_write.py 0.03350826998939738 0.033480543992482126 In other words, there's no difference between usage of str vs StringIO. But it wasn't always like that, with CPython2.7.17: $ python2.7 str_iadd-vs-StringIO_write.py 2.10510993004 0.0399420261383 But Python2 is dead, right? Ok, let's see how Jython3 and IronPython3 fair. To my surprise, there're no (public releases of) such. Both projects sit firmly in the Python2 territory. So, let's try them: $ java -jar jython-standalone-2.7.2.jar str_iadd-vs-StringIO_write.py 10.8869998455 1.74700021744 Sadly, I wasn't able to get to run IronPython.2.7.9.zip on my Linux system, so I used the online version at https://tio.run/#python2-iron (after discovering that https://ironpython.net/try/ is dead). 2.7.9 (IronPython 2.7.9 (2.7.9.0) on Mono 4.0.30319.42000 (64-bit)) 26.2704391479 1.55628967285 So, it seems that rumors of Python2 being dead are somewhat exaggerated. Let's try a project which tries to provide "missing migration path" between Python2 and Python3 - https://github.com/naftaliharris/tauthon Tauthon 2.8.1+ (heads/master:7da5b76f5b, Mar 29 2020, 18:05:05) $ tauthon str_iadd-vs-StringIO_write.py 0.792158126831 0.0467159748077 Whoa, tauthon seems to be faithful to its promise of being half-way between CPython2 and CPython2. Anyway, let's get back to Python3. Fortunately, there's PyPy3, so let's try that: $ ./pypy3.6-v7.3.0-linux64/bin/pypy3 str_iadd-vs-StringIO_write.py 0.5423258490045555 0.01754526497097686 Let's not forget little Python brothers and continue with MicroPython 1.12 (https://github.com/micropython/micropython): $ micropython str_iadd-vs-StringIO_write.py 41.63419413566589 0.08073711395263672 Pycopy 3.0.6 (https://github.com/pfalcon/pycopy): $ pycopy str_iadd-vs-StringIO_write.py 25.03198313713074 0.0713810920715332 I also wanted to include TinyPy (http://tinypy.org/) and Snek (https://github.com/keith-packard/snek) in the shootout, but both (seem to) lack StringIO object. These results can be summarized as follows: of more than half-dozen Python implementations, CPython3 is the only implementation which optimizes for the dubious usage of an immutable string type as an accumulating character buffer. For all other implementations, unintended usage of str incurs overhead of about one order of magnitude, 2 order of magnitude for implementations optimized for particular usecases (this includes PyPy optimized for speed vs MicroPython/Pycopy optimized for small code size and memory usage). Consequently, other implementations have 2 choices: 1. Succumb to applying the same mis-optimization for string type as CPython3. (With the understanding that for speed-optimized projects, implementing mis-optimizations will eat into performance budget, and for memory-optimized projects, it likely will lead to noticeable memory bloat.) 2. Struggle against inefficient-by-concept usage, and promote usage of the correct object types for incremental construction of string content. This would require improving ergonomics of existing string buffer object, to make its usage less painful for both writing new code and refactoring existing. As you may imagine, the purpose of this RFC is to raise awareness and try to make headway with the choice 2. 3. Scope Creep, aka "Possible Future Work" ------------------------------------------ The purpose of this RFC is specifically to propose to apply *single* simple, obvious change. .__iadd__ is just an alias for .write, period. However, for completeness, it makes sense to consider both alternatives and where the path of adding "str-like functionality" may lead us. 1. One alternative to patching StringIO would be to introduce a completely different type, e.g. StringBuf. But that largely would be "creating more entities without necessity", given that StringIO already offers needed buffering functionality, and just needs a little touch of polish with interface. If 2 classes like StringIO and StringBuf existed, it would be extra quiz to explain difference between them and why they both exist. 2. On the other hand, this RFC fixates on the output buffering. But just image how much fun can be done re: input buffers! E.g., we can define "buf[0]" to have semantics of "tmp = buf.tell(); res = buf.read(1); buf.seek(tmp); return res". Ditto for .startswith(), etc., etc. So, well... This RFC is about making .__iadd__ to be an alias for .write, and cover with this output buffering usecase. Whoever may have interest in dealing with input buffer shortcuts would need to provide a separate RFC, with separate usecases and argumentation. 4. (Self-)Criticism and Risks. ------------------------------ 1. The biggest "criticism" I see is a response a-la "there's no problem with CPython3, so there's nothing to fix". This is related to a bigger questions "whether a life outside CPython exists", or put more formally, where's the border between Python-the-language and CPython-the-implementation. To address this point, I tried to collect performance stats for a pretty wide array of Python implementations. 2. Another potential criticism is that this may open a scope creep to add more str-like functionality to classes which classically expose a stream interface. Paragraph 3.2 is dedicated specifically to address this point, by invoking hopefully the best-practice approach: request to focus on the currently proposed feature, which requires very little changes for arguably noticeable improvements. (At the same time, for an abstract possibility of this change to be found positive, this may influence further proposals from interested parties). 3. Implementing the required functionality is pretty easy with a user subclass: class MyStringBuf(io.StringIO): def __iadd__(self, s): self.write(s) return self Voila. The problem is performance. Calling such .__iadd__() method implementing in Python is 3 times slower than calling .write() directly (with CPython3.6). But paradigmatic problem is even bigger: this RFC seeks to establish the best practice of using explicitly designed for the purpose type with ergonomic interface. Saying that "we lack such clearly designated type out of the box, but if you figure out that it's a problem (if you do figure that out), you can easily resolve that on your side, albeit with a performance hit when compared with it being provided out of the box" - that's not really welcoming or ergonomic. 5. Prior Art ------------ As many things related to Python, the idea is not new. I found thread from 2006 dedicated to it: https://mail.python.org/pipermail/python-list/2006-January/403480.html (strangely, there's some mixup in archives, and "thread view" shows another message as thread starter, though it's not: https://mail.python.org/pipermail/python-list/2006-January/357453.html) . The discussion there seemed to be without clear resolution and has been swamped into discussion of unicode handling complexities in Python2 and implementation details of "StringIO" vs "cStringIO" modules (both of which were deprecated in favor of "io"). -- Best regards, Paul mailto:pmiscml@gmail.com [View Less]

19 76

SerialExecutor for concurrent.futures + Convenience constructor
by Jonathan Crall April 10, 2020

April 10, 2020

I'd like to propose an improvement to `concurrent.futures`. The library's ThreadPoolExecutor and ProcessPoolExecutor are excellent tools, but there is currently no mechanism for configuring which type of executor you want. Also, there is no duck-typed class that behaves like an executor, but does its processing in serial. Often times a develop will want to run a task in parallel, but depending on the environment they may want to disable threading or process execution. To address this I use a … [View More]utility called a `SerialExecutor` which shares an API with ThreadPoolExecutor/ProcessPoolExecutor but executes processes sequentially in the same python thread: ```python import concurrent.futures class SerialFuture( concurrent.futures.Future): """ Non-threading / multiprocessing version of future for drop in compatibility with concurrent.futures. """ def __init__(self, func, *args, **kw): super(SerialFuture, self).__init__() self.func = func self.args = args self.kw = kw # self._condition = FakeCondition() self._run_count = 0 # fake being finished to cause __get_result to be called self._state = concurrent.futures._base.FINISHED def _run(self): result = self.func(*self.args, **self.kw) self.set_result(result) self._run_count += 1 def set_result(self, result): """ Overrides the implementation to revert to pre python3.8 behavior """ with self._condition: self._result = result self._state = concurrent.futures._base.FINISHED for waiter in self._waiters: waiter.add_result(self) self._condition.notify_all() self._invoke_callbacks() def _Future__get_result(self): # overrides private __getresult method if not self._run_count: self._run() return self._result class SerialExecutor(object): """ Implements the concurrent.futures API around a single-threaded backend Example: >>> with SerialExecutor() as executor: >>> futures = [] >>> for i in range(100): >>> f = executor.submit(lambda x: x + 1, i) >>> futures.append(f) >>> for f in concurrent.futures.as_completed(futures): >>> assert f.result() > 0 >>> for i, f in enumerate(futures): >>> assert i + 1 == f.result() """ def __enter__(self): return self def __exit__(self, ex_type, ex_value, tb): pass def submit(self, func, *args, **kw): return SerialFuture(func, *args, **kw) def shutdown(self): pass ``` In order to make it easy to choose the type of parallel (or serial) backend with minimal code changes I use the following "Executor" wrapper class (although if this was integrated into concurrent.futures the name would need to change to something better): ```python class Executor(object): """ Wrapper around a specific executor. Abstracts Serial, Thread, and Process Executor via arguments. Args: mode (str, default='thread'): either thread, serial, or process max_workers (int, default=0): number of workers. If 0, serial is forced. """ def __init__(self, mode='thread', max_workers=0): from concurrent import futures if mode == 'serial' or max_workers == 0: backend = SerialExecutor() elif mode == 'thread': backend = futures.ThreadPoolExecutor(max_workers=max_workers) elif mode == 'process': backend = futures.ProcessPoolExecutor(max_workers=max_workers) else: raise KeyError(mode) self.backend = backend def __enter__(self): return self.backend.__enter__() def __exit__(self, ex_type, ex_value, tb): return self.backend.__exit__(ex_type, ex_value, tb) def submit(self, func, *args, **kw): return self.backend.submit(func, *args, **kw) def shutdown(self): return self.backend.shutdown() ``` So in summary, I'm proposing to add a SerialExecutor and SerialFuture class as an alternative to the ThreadPool / ProcessPool executors, and I'm also advocating for some sort of "ParamatrizedExecutor", where the user can construct it in "thread", "process", or "serial" model. -- -Jon [View Less]

6 26

PEP 9999: Retire animal-unfriendly language
by Gerrit Holl April 1, 2020

April 1, 2020

(needs a sponsor) latest version at https://github.com/gerritholl/peps/blob/animal-friendly/pep-9999.rst PEP: 9999 Title: Retire animal-unfriendly language Author: Gerrit Holl <gerrit.holl(a)gmail.com> Discussions-To: python-ideas(a)python.org Status: Draft Type: Informational Content-Type: text/x-rst Created: 01-Apr-2020 Post-History: 01-Apr-2020 Sponsor: Abstract ======== Python has long used metasyntactic variables that are based on the consumption of meat and dairy products, such … [View More]as "spam", "ham", and "eggs". This language is not considerate to pigs or chicken and violates the spirit of the Code of Conduct. This PEP proposes to retire the use of those names in official Python documentation and source code and to recommend users of Python to do the same. Motivation and Rationale ======================== Estimates for the number of animals slaughtered for meat every year vary, but `worldindata`_ estimates around 80 billion individuals. Farmed animals are often kept in small cages with little to no access to daylight, suffer stress during life and slaughter, or are otherwise systematically mistreated. The `Python Code of Conduct`_ describes that community members are open, considerate, and respectful. The Python standard library and documentation contain numerous references to meat or dairy based food products that are not respectful to our fellow inhabitants of planet Earth. Examples include "spam", "bacon", and "eggs". To align the language use in the standard library and documentation with the Code of Conduct, use of such language should be retired. Current practice ================ There is a widespread tradition in the Python standard library, the documentation, and the wider community, to include references to Monty Pythons Flying Circus. The use of "spam", "bacon", "sausage", and "eggs" can be traced back to the `"Spam" sketch`_ originally broadcast by the British Broadcasting Corporation (BBC) on 8 September 1972. In this sketch, a couple are trying to order food in a diner where all items contain spam. The woman does not like spam and wants to order food without spam. A group of horned vikings then sing about the wonderful spam. To get an overview of the usage in the current standard library, the command ``cat $(find . -name '*.py') | grep -oi term | wc -l`` was used. This showed 2615 occurences for spam, 593 for ham (this include some false positives, among other reasons due to references to people whose name innociously contains the substring ham), 517 for eggs, 57 for bacon, and 10 for sausage. Searching ``*.rst`` in the documentation revealed 391 occurrences for spam, 82 for ham, 96 for eggs, 28 for bacon, and 10 for sausage. The source code for cpython revealed just 2 usages for spam and 1 for eggs. Proposed alternatives ===================== Keeping with the good practice of referencing sketches from Monty Python's Flying Circus, this PEP proposes to adopt the fruits mentioned in the `"Self-Defence Against Fresh Fruit" sketch`_: * raspberry (not currently in use) * banana (68 times in standard library) * apricot (not currently in use) * pineapple (8 times in standard library) * peach (once in standard library) * redcurrant (not currently in use) * damson (not currently in use) * prune (23 times in standard library) Other possible alternatives keeping with food items: * salad (occurs once in standard library) * aubergine (referred to in the spam sketch) * shallot (the same) * tofu (vegan protein alternative) Specification ============= For the reasons mentioned in the rationale, all references to meat or dairy products shall be removed from the Python standard library, the documentation, and the cpython source code. The wider Python community is recommended to follow this practice. In core Python: * Programmers SHALL NOT use the metasyntactic variables "spam", "ham", "bacon", or "sausage", neither as variable names, nor in example strings, nor in documentation. * Programmers SHALL NOT use the metasyntactic variable "eggs" in context with food items, but may still use it in context of other body parts. Prohibited: ``["salad", "eggs"]``. Allowed: ``["ovaries", "pouch", "eggs"]``. * Programmers SHALL NOT use any other metasyntactic variable that is unfriendly to animals. The wider Python community is encouraged to adopt these practices as well, but the continued use of animal-unfriendly metasyntactic variables will not be considered a violation of the code of conduct. Rejected ideas ============== The authors carefully considered the widespread use of the word "bug" in the meaning of a source code error. Insects including bugs play a crucial role in ecosystems around the world, and it is not fair to blame them for an error that can only be the programmer's. However, the use of the word "bug" for a source code error is too much ingrained into daily use, it far predates the Python community, is not limited to the Python community, and the word "bug" is less unfriendly than "spam", "ham", or "bacon". Therefore, the word "bug" may still be used. Reference Implementation ======================== The author promises to provide a reference implementation for Python 3.10, should this PEP be accepted. References ========== .. _worldindata: https://ourworldindata.org/meat-production .. _Python code of conduct: https://www.python.org/psf/conduct/ .. _"Spam" sketch: http://www.montypython.net/scripts/spam.php .. _"Self-Defence Against Fresh Fruit" sketch: http://www.montypython.net/scripts/fruit.php Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

17 24

returning a namedtuple on dict.items()
by Nadav Wexler March 31, 2020

March 31, 2020

Hi, That is my first post here, I hope it's the right place for it. so I was using some dicts had the following code: env = dict(t for t in os.environ.items() if 'SUBSTRING' not in t[0]) and I thought: It could have really been nice if i could do: env = dict(t for t in os.environ.items() if 'SUBSTRING' not in t.key) now I see that environ is not exactly a dict, but i think this is useful for all dictionary types. on the other hand, dicts are everywhere and i'm not sure about … [View More]

4 4

Re: PEP 9999: Retire animal-unfriendly language
by Michael Christensen March 31, 2020

March 31, 2020

We've got to be considerate of other languages: What if, for example, a German speaker is looking for example code in English documentation and sees the word "speck"? He/she will think it means bacon and will be offended! Sent via the Samsung Galaxy S® 6, an AT&T 4G LTE smartphone Get Outlook for Android<https://aka.ms/ghei36>

1 0

dunder methods for encoding & prettiness aware formal & informal representations
by Steve Jorgensen March 31, 2020

March 31, 2020

This is really an idea for an idea. I'm not sure what the ideal dunder method names or APIs should be. Encoding awareness: The informal (`str`) representations of `inf` and `-inf` are "inf" and "-inf", and that seems appropriate as a known-safe value, but if we're writing the representation to a stream, and the stream has a Unicode encoding, then those might prefer to represent themselves as "∞" and "-∞". If there were a dunder method for informal representation to which the destination … [View More]

20 54

pretty-printing in the terminal
by Per Bothner March 31, 2020

March 31, 2020

Inspired by the LWN article (https://lwn.net/Articles/815265/) about enhanced pretty-printing, I implemented a modified repr/displayhook that makes uses of the builtin pretty-printing of the DomTerm (https://domterm.org) terminal emulator. See this screenshot http://domterm.org/images/python-pretty1.png which shows the *same* REPL session displayed in two "panes" of different widths. More information here: http://domterm.org/Python-tips.html The idea is that Python handles *what* to print, … [View More]but the terminal emulator (or IDE like Jupyter or ipython) decides where to break the lines, add indentation, etc. The terminal knows the line-width and can dynamically re-format the output if the width changes - even for previously-printed output (and even for an exited python command). This is just like many terminals will automatically re-wrap over-long lines - but on steroids. This required a rather small amount of code to insert some extra escape sequences to delimit grouping, optional line-breaks, and indentation - see here: https://github.com/PerBothner/DomTerm/blob/master/tools/python/domterm/util… I'm not suggesting pprint should depend on the terminal being able to do pretty-printing, but I suggest that at the least type-specific pretty-printers should use overrideable methods to define grouping and optional newlines. That would allow an IDE or terminal-specific code to by-pass the Python pretty-printing and instead have the terminal/IDE do it. As an example you can see how Kawa does it - look for the writeList method in https://gitlab.com/kashell/Kawa/-/blob/master/gnu/kawa/functions/DisplayFor… This calls conceptually abstract methods like startLogicalBlock and endLogicalBlock (to delimit groups) and writeSpaceFill (to mark optional newline). Going beyond pretty-printing, I've been thinking about a "data explorer" where you click on buttons to hide/show more detail. I'm most familiar with this from the JavaScript consoles of Firefox and Chrome. I implemented a "finite" version of this idea for Kawa - see here: https://domterm.org/saved/dt-prettynested.html Note the triangles are clickable butoons that hide or show detail (and how this is integrated with pretty-printing). This prototype only handles the case where all the displayable data is printed, even though some may be initially hidden. To explore larger or circular data structures you'd want a protocol where the terminal/IDE can request more data. (I have a rough design for this, but that is a different discussion. Let me know if this would be interesting.) (I am not primarily a Python programmer, but it's been fun figuring out how to implement Python/DomTerm integration.) -- --Per Bothner per(a)bothner.com http://per.bothner.com/ [View Less]

2 1

Issue 34850: Syntax Warning in the real world (is 0 / is 1)
by Philip Kahn March 31, 2020

March 31, 2020

Yikes, this change is spitting out a lot of warnings on first run, and plenty in stuff that's not my code to boot. A huge number of the cases, though, seem to be "is 0" and "is 1" comparisons. Having read through the original issue I get where the SyntaxWarning is coming from, but it seems it'd be a genuine end-user experience improvement to not raise the warning on those two classic "truthy" comparisons. I understand it locks in that particular implementation detail, but it also seems … [View More]

6 5

Limit 'import as' syntax
by joperez＠hotmail.fr March 30, 2020

March 30, 2020

There should be one-- and preferably only one --obvious way to do it. (from The Zen of Python https://www.python.org/dev/peps/pep-0020/) However, in something as basic as import syntax, that's not the case. This example comes from PEP 221 (https://www.python.org/dev/peps/pep-0221/) : A slightly special case exists for importing sub-modules. The statement `import os.path` stores the module os locally as os, so that the imported submodule path is accessible as os.path. As a result, `import os.… [View More]

6 8

Limit 'import as' syntax
by joperez＠hotmail.fr March 30, 2020

March 30, 2020

There should be one-- and preferably only one --obvious way to do it. (from The Zen of Python https://www.python.org/dev/peps/pep-0020/) However, in something as basic as import syntax, that's not the case. This example comes from PEP 221 (https://www.python.org/dev/peps/pep-0221/) : A slightly special case exists for importing sub-modules. The statement `import os.path` stores the module os locally as os, so that the imported submodule path is accessible as os.path. As a result, `import os.… [View More]

1 0

About python3 on windows
by Frédéric De Jaeger March 26, 2020

March 26, 2020

Hi all, There is a recurring problem in my company where we use python in various places (python3). We do cross platform development windows/linux and our python scripts need to run everywhere. Some scripts are launched manually in a terminal. Others are launched via windows' gui interface. Others are launched through our build process (cmake based, but also manual Makefile) And others are launched via bash scripts. And possible several other scenario I've forgot. On windows, most users … [View More]

16 35

Re: About python3 on windows
by Eryk Sun March 26, 2020

March 26, 2020

On 3/25/20, Steve Barnes <GadgetSteve(a)live.co.uk> wrote: >> Except it's not necessarily what the original post wants. The OP wants the >> shebang "#!/usr/bin/env python3" to "work everywhere by >> default", for which I assume it's implied that it should work consistently >> everywhere. I'd prefer for the launcher's env search to also >> support versioned "pythonX[.Y][-32|-64]" commands such as "python3". > > The windows launcher already does support … [View More]

5 9

Instance method to test equivalence between set and iterable
by Steve Jorgensen March 25, 2020

March 25, 2020

Currently, the `issubset` and `issuperset` methods of set objects accept arbitrary iterables as arguments. An iterable that is both a subset and superset is, in a sense, "equal" to the set. It would be inappropriate for `==` to return `True` for such a comparison, however, since that would break the `Hashable` contract. Should sets have an additional method, something like `like(other)`, `issimilar(other)`, or `isequivalent(other)`, that returns `True` for any iterable that contains the all of … [View More]

13 29

Re: New explicit methods to trim strings
by Barry Scott March 24, 2020

March 24, 2020

> On 19 Mar 2020, at 22:12, Rob Cliffe <rob.cliffe(a)btinternet.com> wrote: > > > On 18/03/2020 20:16, Barry Scott wrote: >> >>> On 18 Mar 2020, at 18:03, Rob Cliffe via Python-ideas <python-ideas(a)python.org> wrote: >>> >>> Consider that the start or end of a string may contain repetitions of an affix. >>> >>> Should `-+-+-+Spam'.stripprefix('-+') remove just the first occurence? All of them? Does it need a '… [View More]

11 20

Developer messages on install (pip, poetry, etc.)
by Ronie Martinez March 23, 2020

March 23, 2020

Good day! I have been developing in Node for a few months now for non-Python projects (legacy project) and I found a NPM feature which could be helpful to developers of Python libraries. When you install a NPM package, some show a message from the author after installation (see example below). ------ Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library! The project needs your help! Please consider supporting of core-js on Open … [View More]

4 4

Syntax for loop invariants
by haael March 22, 2020

March 22, 2020

Python has more and more optional tools for formal correctness cheking. Why not loop invariants? Loop invariant is such a statement that if it was true before the loop iteration, it will also be true after the iteration. It can be implemented as an assertion of an implication. now_value = False while running_condition(...): prev_value = now_value now_value = invariant_condition(...) assert now_value if prev_value else True Here for ellipsis we can … [View More]

8 12

Add the imath module
by Serhiy Storchaka March 22, 2020

March 22, 2020

What are your thoughts about adding a new imath module for integer mathematics? It could contain the following functions: * factorial(n) Is just moved from the math module, but non-integer types are rejected. Currently math.factorial() accepts also integer floats like 3.0. It looks to me, the rationale was that at the time when math.factorial() was added, all function in the math module worked with floats. But now we can revise this decision. * gcd(n, m) Is just moved from the math … [View More]

14 38

New explicit methods to trim strings
by Alex Grigoryev March 21, 2020

March 21, 2020

Following the discussion here (https://link.getmailspring.com/link/7D84D131-65B6-4EF7-9C43-51957F9DFAA9@ge…) I propose to add 3 new string methods: str.trim, str.ltrim, str.rtrim Another option would be to change API for str.split method to work correctly with sequences. In [1]: def ltrim(s, seq): ...: return s[len(seq):] if s.startswith(seq) else s ...: In [2]: def rtrim(s, seq): ...: return s[:-len(seq)] if s.endswith(seq) else s ...: In [3]: def trim(s, seq): ...: return ltrim(rtrim(s, … [View More]

28 162

Adding a built-in data structure with binary search tree semantics
by 1njection1＠gmail.com March 20, 2020

March 20, 2020

In other languages, there are built-in data structures with binary search tree semantics, as an example, std::map in C++. By binary search tree semantics we mean the following: * We can insert elements into the structure in time faster than O(n), i.e. O(logn) for std::map * We can iterate over all of the elements of the structure in sorted order in linear time, i.e. O(n) for std::map Is there currently any data structure in Python with binary search tree semantics? To illustrate the need, … [View More]can the following program be written elegantly and efficiently with built-in Python data structures? Imagine that you are a fund manager at a bank. Bank accounts are added to your fund very frequently. A bank account has an ID and a value. A smaller ID means that the bank account was created earlier than one with a larger ID. Each time that an account is added to your fund, you would like to know the ID number of the 1000th oldest account, for your records. Here are implementations in Python 3 and C++, for comparison: ```python3 from random import randint dist = lambda: randint(0, 2147483647) def main(): my_map = {} # fills a map with 1000000 random mappings of type (int, int) for i in range(0, 1000000): my_map[dist()] = dist() # prints out the 1000th smallest key and its value target_key = nth_smallest_key(1000, my_map) print("({}: {})".format(target_key, my_map[target_key])) # writes a new random mapping to the map # then prints out the 1000th smallest key and its value if that key # has changed # 100000 times for i in range(100000): my_map[dist()] = dist() test_key = nth_smallest_key(1000, my_map) if target_key != test_key: target_key = test_key print("({}: {})".format(target_key, my_map[target_key])) # print an indicator every 100 iterations for comparison if i % 100 == 0: print("iteration: {}".format(i)) def nth_smallest_key(n, m): return sorted(m.keys())[n] if __name__ == "__main__": main() ``` ```c++ #include <cstdio> #include <map> #include <random> using namespace std; int nth_smallest_key(int n, map<int, int>& m); int main() { random_device rd; mt19937 psrng(rd()); uniform_int_distribution<> dist(0, 2147483647); map<int, int> my_map; // fills a map with 1000000 random mappings of type (<int>: <int>) for (int i = 0; i < 1000000; i++) { my_map[dist(psrng)] = dist(psrng); } // prints out the 1000th smallest key and its value int target_key = nth_smallest_key(1000, my_map); printf("(%d: %d)\n", target_key, my_map[target_key]); // writes a new random mapping to the map // then prints out the 1000th smallest key and its value if that key // has changed // 100000 times for (int i = 0; i <= 100000; i++) { my_map[dist(psrng)] = dist(psrng); int test_key = nth_smallest_key(1000, my_map); if (target_key != test_key) { target_key = test_key; printf("(%d: %d)\n", target_key, my_map[target_key]); } } return 0; } int nth_smallest_key(int n, map<int, int>& m) { map<int, int>::iterator curr = m.begin(); for (int i = 0; i < n; i++) { curr = next(curr); } return curr->first; } ``` Makefile: ```make runcpp: buildcpp ./main buildcpp: g++ -o main bst_semantics_cpp.cpp runpython: python3 bst_semantics_python.py ``` The C++ program runs on my machine in approximately 5 seconds ```bash $ time ./main (2211193: 2021141747) (2208771: 1079444337) (2208700: 1187119077) (2208378: 1447743503) ... (1996019: 1378401665) (1989217: 1457497754) (1989042: 1336229409) real 0m4.915s user 0m4.750s sys 0m0.094s $ ``` The Python program does not reach 1% of completion after 120 seconds ```bash $ time make runpython python3 bst_semantics_python.py (2158070: 1498305903) iteration: 0 iteration: 100 iteration: 200 ^CTraceback (most recent call last): File "bst_semantics_python.py", line 36, in <module> main() File "bst_semantics_python.py", line 23, in main test_key = nth_smallest_key(1000, my_map) File "bst_semantics_python.py", line 33, in nth_smallest_key return sorted(m.keys())[n] KeyboardInterrupt Makefile:8: recipe for target 'runpython' failed make: *** [runpython] Error 1 real 2m2.040s user 1m59.063s sys 0m0.375s $ ``` I would like to propose adding such a data structure to the Python standard library. It could be called a SortedDict. [View Less]

11 23

Support max heaps in heapq module
by Abhai Kollara March 20, 2020

March 20, 2020

Hi, I understand that the default implementation of heapq.heapify is to create a min-heap. The logic for max-heapify has already been implemented inside _heapify_max and along with that for corresponding push and pop operations. But they are not exposed to the user and hence not mentioned in any of the docs. I am unable to find any explanation as to why this is so. Current methods of implementing max-heaps (according to the SO question mentioned below) involves inverting the signs of numeric … [View More]

2 1

Formalized pretty & encoding-aware object representation (was dunder methods for...)
by Steve Jorgensen March 20, 2020

March 20, 2020

Based on the conversations stemming from my previous post, it is clear that the topic was too implementation-specific. It is not clear whether dunder methods are an appropriate component of the solution (they might or might not be). Also, it presumably makes sense to start by looking at prior art rather than inventing from scratch. Quotes from previous thread regarding prior art to look at: Jonathan Fine wrote: <snip> > Here's some comments on the state of the art. In addition to &… [View More]

2 3

Re: dunder methods for encoding & prettiness aware formal & informal representations
by Angus Hollands March 20, 2020

March 20, 2020

Has anyone mentioned the IPython pretty printer yet? I'm late to the conversation unfortunately, so apologies if someone else already raised it. https://ipython.readthedocs.io/en/stable/api/generated/IPython.lib.pretty.h… On Fri, 20 Mar 2020 at 16:02, <python-ideas-request(a)python.org> wrote: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/… [View More]mailman3/lists/python-ideas.python.org/ > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..."Today's Topics: > > 1. Re: dunder methods for encoding & prettiness aware formal & informal > representations > (Chris Angelico) > 2. Re: dunder methods for encoding & prettiness aware formal & informal > representations > (Steven D'Aprano) > 3. Re: dunder methods for encoding & prettiness aware formal & informal > representations > (Greg Ewing) > 4. Re: dunder methods for encoding & prettiness aware formal & informal > representations > (Steven D'Aprano) > > > > ---------- Forwarded message ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: "Steven D'Aprano" <steve(a)pearwood.info> > Cc: python-ideas <python-ideas(a)python.org> > Bcc: > Date: Fri, 20 Mar 2020 15:37:08 +1100 > Subject: [Python-ideas] Re: dunder methods for encoding & prettiness aware > formal & informal representations > On Fri, Mar 20, 2020 at 3:28 PM Steven D'Aprano <steve(a)pearwood.info> > wrote: > > > > On Fri, Mar 20, 2020 at 03:01:16PM +1100, Chris Angelico wrote: > > > On Fri, Mar 20, 2020 at 2:59 PM Steven D'Aprano <steve(a)pearwood.info> > wrote: > > > > > > > > On Thu, Mar 19, 2020 at 11:38:28PM +0000, Samuel Colvin wrote: > > > > > > > > > But it also looks for a "__pretty__" method on objects, and if > found uses > > > > > that to display the object. > > > > > > > > Are you aware that dunder names are reserved for Python's use? > > > > > > > > > > Really? > > > > Yes, really. > > > > > https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-… > > > > > > > Somebody better tell SQLAlchemy that they're breaking rules, then. > > > > Lots of people break the rules all the time. Doesn't mean they should. > > > > "Subject to breakage without warning" technically applies to a *lot* > of things that aren't guaranteed. Using __pretty__ as a protocol is no > different from any of those. > > IMO it's not exactly a serious crime, even if technically it's > something that could be broken. Also, since this is a proposal on > python-ideas, it'd have as much blessing as __copy__, which to my > knowledge has no meaning in the language itself, only in the standard > library; it'd be the same with __pretty__, defined by the pprint > module. > > ChrisA > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Fri, 20 Mar 2020 16:25:28 +1100 > Subject: [Python-ideas] Re: dunder methods for encoding & prettiness aware > formal & informal representations > On Fri, Mar 20, 2020 at 03:37:08PM +1100, Chris Angelico wrote: > > > "Subject to breakage without warning" technically applies to a *lot* > > of things that aren't guaranteed. > > Yes? > > > > Using __pretty__ as a protocol is no different from any of those. > > If we should choose to use a `__pretty__` dunder, we have no obligation > to follow Samuel's API, or make it a future-import, or give him any > warning, or make any allowances for the fact that he is already using > it. We can just break his code. > > Samuel may not have known that, but hopefully he will now. > > > > IMO it's not exactly a serious crime, > > Isn't it? Damn, I've already reported him to the federal police, the > SWAT team will be arriving in 5, 4, 3, 2, ... > > *wink* > > I didn't describe it as a crime at all, I just asked if he knew he was > using a reserved name. > > > > even if technically it's > > something that could be broken. Also, since this is a proposal on > > python-ideas, it'd have as much blessing as __copy__, which to my > > knowledge has no meaning in the language itself, only in the standard > > library; it'd be the same with __pretty__, defined by the pprint > > module. > > I don't think that's a distinction that means anything. Whether the > standard library or the interpreter itself breaks your code, it's still > broken. > > > > -- > Steven > > > ---------- Forwarded message ---------- > From: Greg Ewing <greg.ewing(a)canterbury.ac.nz> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Fri, 20 Mar 2020 18:30:24 +1300 > Subject: [Python-ideas] Re: dunder methods for encoding & prettiness aware > formal & informal representations > On 20/03/20 4:55 pm, Steven D'Aprano wrote: > > > Are you aware that dunder names are reserved for Python's use? > > Nobody is going to put you in jail if you use an unofficial dunder > name. You just run the risk that your use of it will conflict with > some official use in the future. > > -- > Greg > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Fri, 20 Mar 2020 17:20:25 +1100 > Subject: [Python-ideas] Re: dunder methods for encoding & prettiness aware > formal & informal representations > On Fri, Mar 20, 2020 at 06:30:24PM +1300, Greg Ewing wrote: > > On 20/03/20 4:55 pm, Steven D'Aprano wrote: > > > > >Are you aware that dunder names are reserved for Python's use? > > > > Nobody is going to put you in jail if you use an unofficial dunder > > name. You just run the risk that your use of it will conflict with > > some official use in the future. > > You're the second person mentioning crime or law. What did I say to > give people the impression that I think that using a dunder is a > criminal offence? > > I didn't say that it was illegal or breaking the law, or a felony or > even a misdemeaner. I didn't even make a value judgement about > whether it was a good thing or a bad thing to use dunder names. > > I said that dunders are reserved. Not everyone knows this. > > What should I have said that won't be misinterpreted as an accusation of > criminality? > > > > -- > Steven_______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > [View Less]

1 0

Formalized pretty & encoding-aware object representation (was dunder methods for...)
by Steve Jorgensen March 16, 2020

March 16, 2020

Based on the conversations stemming from my previous post, it is clear that the topic was too implementation-specific. It is not clear whether dunder methods are an appropriate component of the solution (they might or might not be). This suggestion is to try to solve 2 inter-related but different issues, possibly through the same mechanism, 2 unrelated mechanisms, or partially overlapping mechanisms. Although the current `__str__` and `__repr__` concepts seem perfectly appropriate to me, I … [View More]

1 1

Re: More appropriate behavior for the NotImplemented object
by Marco Sulla March 16, 2020

March 16, 2020

On Mon, 16 Mar 2020 at 04:51, Kyle Stanley <aeros167(a)gmail.com> wrote: > PEP 8 itself is very far from a definitive authority. Its scope is specifically intended to cover CPython stdlib development, not all uses of Python. Yes, I know. But it's a common convention, and Python relies often on conventions. For example, the underscore at the start of a variable. I mean, this breaks duck typing. I expect the object to quack, and instead it roars. And this is because bool(… [View More]

2 1

SQL string prefix idea
by minecraft2048＠gmail.com March 16, 2020

March 16, 2020

The idea is to add a new string prefix 's' for SQL string. This string doesn't do anything in Python, unlike b"" or f"" strings, but interactive Python shells like IPython or Jupyter can parse the following characters as SQL syntax instead of Python syntax and give SQL syntax highlighting and autocompletion, and if they are configured correctly, they can do column name autocompletion. Unfortunately when I try to type s"select * from table" it gave me syntax error instead, so I think this need … [View More]

15 22

Magnitude and ProtoMagnitude ABCs — primarily for argument validation
by Steve Jorgensen March 15, 2020

March 15, 2020

I have encountered cases in which I would like to validate that an argument can be properly compared with other instances of its type. This is true of numbers, strings, dates, … but not for `NoneClass`, `type`, …. One way that I have tried to handle this is to check whether the object can be compared to itself using `>`, `<`, `>=`, and `<=` and that it is neither `>` or `<` itself and is both `>=` and `<=` itself. The most glaring example of why this is insufficient is … [View More]

16 51

Allow using the or operator to denote unions in type annotations
by Noah Peter May March 14, 2020

March 14, 2020

Hello everyone, I'm not certain if this has been brought up before, but I felt like bringing it to the table anyway. As the title says, it would be convenient to represent a union type with the or operator. While typing extra characters is not the end of the world, having a more concise representation of unions would be helpful. It also saves a few moments of thought when giving a value a type, then realizing it should actually be a union of types. Before, you would have to arrow-key both … [View More]ways and surround them with "Union[" and "]" while with this new method you can simply append "or" and then the type. While not a world-changer, it might be useful. A few clarifications for those interested: This method of declaring unions will *NOT* replace "Union[...]". Instead, it will serve as syntactic sugar for representing the same thing. The next is actually a question: do we use "*bitwise* or" or "*boolean* or" (aka "|" vs "or")? I honestly don't know which would be preferable. The latter is more readable than the other but requires 3-4 characters (including whitespace) vs just 1-3 extra. Both are shorter than the 8-9 chars for the traditional union. I'll just use both in the examples for now. Finally, there are implementation details. There are some problems that should be addressed for both run-time and static analysis annotations. For run-time, these new annotations would cause an error or unwanted result in current versions. x: List | Tuple # TypeError: unsupported operand type(s) for |: ... x: List or Tuple # evaluates to "List" because of truthiness In either case, we could override the or-dunder method to return the "old" union representation of types List | Tuple == Union[List, Tuple] (Union[List, Tuple] or Union[int, str]) == Union[List, Tuple, int, str] This should allow any code relying on run-time annotations to work as expected. Note: I am uncertain if some versions of Python do not evaluate annotation expressions, but a quick test in 3.7 seems to show they do. >>> var: print('This expression runs') This expression runs I don't have much to say about how this relates to static analysis. Long story short, if this becomes adopted, static analysis tools like Mypy will simply have to adapt. It will complicate the analysis process since there are now two ways to represent unions, but that's the way it would be. Thoughts? Comments? Agreement of usefulness or blatant hatred for the suggestion? Let me know 😊 Cheers, Noah May [View Less]

5 9

More appropriate behavior for the NotImplemented object
by Steve Jorgensen March 13, 2020

March 13, 2020

I realize this is probably something that would be hard to change for compatibility reasons. Maybe someone can think of a way around that though? It seems to me that `not NotImplemented` should result in `NotImplemented` and attempting to convert it to `bool` should raise a `TypeError` exception. Take the following example: ``` def __lt__(self, other): return not self.__ge__(other): def __le__(self, other): return not self.__gt__(other): def __ge__(self, other): … [View More]

12 17

Re: python-ideas@python.org post from python-ideas@marco.sulla.e4ward.com requires approval
by Brett Cannon March 13, 2020

March 13, 2020

These are coming in fine because they are not handled by Mailman's filtering rules. And the reason for the holds is they are being flagged for " Message has implicit destination". On Fri, Mar 13, 2020 at 8:06 AM Marco Sulla < python-ideas(a)marco.sulla.e4ward.com> wrote: > Well, probably it depends by two possible errors: > > 1. sometimes I send directly to python-ideas(a)python.org. This way my > real address is used, and the system does not recognize it. I saved > the "… [View More]correct" address to my contact list, so now it's more simple to > not do this mistake > > 2. I noticed that I can't put the address of the list in the CC field. > Probably this is a problem of e4ward. Now I put it every time in the > To: field > > PS: this message, for example, has some problems? I put the "correct" > list address in the To field. > PPS: maybe it's more simple to create a marco_sulla_python(a)gmail.com > address.... > > On Fri, 13 Mar 2020 at 14:27, C. Titus Brown <ctbrown(a)ucdavis.edu> wrote: > > > > Hi Marco, > > > > all I can do is tell you what I’m seeing. Some of your posts go through > fine, and others haven’t. > > > > I’ll approve your posts when I have time (almost always within 24 hrs), > but some might be delayed. > > > > best, > > —t > > > > > On Mar 13, 2020, at 4:56 AM, Marco Sulla < > python-ideas(a)marco.sulla.e4ward.com> wrote: > > > > > > Hello. I'm simply using e4ward, a disposable email address service, to > > > avoid spam. > > > I always used e4ward in past, with every mailing list, and also with > > > Python mailing lists. Furthermore I'm using e4ward also with Python > > > List mailing list in the current time, and I have no problem. > > > > > > On Fri, 13 Mar 2020 at 02:33, C. Titus Brown <ctbrown(a)ucdavis.edu> > wrote: > > >> > > >> Hi Marco, > > >> > > >> your e-mailing/response system is causing me to have to approve many > of your messages manually. Please fix! > > >> > > >> thanks, > > >> —titus > > >> > > >> Begin forwarded message: > > >> > > >> From: python-ideas-owner(a)python.org > > >> Subject: python-ideas(a)python.org post from > python-ideas(a)marco.sulla.e4ward.com requires approval > > >> Date: March 12, 2020 at 5:47:03 PM PDT > > >> To: python-ideas-owner(a)python.org > > >> > > >> As list administrator, your authorization is requested for the > > >> following mailing list posting: > > >> > > >> List: python-ideas(a)python.org > > >> From: python-ideas(a)marco.sulla.e4ward.com > > >> Subject: Re: [Python-ideas] Re: Make `del x` an expression > evaluating to `x` > > >> > > >> The message is being held because: > > >> > > >> Message has implicit destination > > >> > > >> At your convenience, visit > > >> > https://mail.python.org/mailman3/lists/python-ideas.python.org/held_messages > > >> to approve or deny the request. > > >> > > >> From: Marco Sulla <python-ideas(a)marco.sulla.e4ward.com> > > >> Subject: Re: [Python-ideas] Re: Make `del x` an expression evaluating > to `x` > > >> Date: March 12, 2020 at 5:45:46 PM PDT > > >> To: Andrew Barnert <abarnert(a)yahoo.com>, Python Ideas < > python-ideas.python.org-mail.python.org.marco.sulla.e4ward.com(a)jr366bvb1z7.reply.e4ward.com > > > > >> Reply-To: python-ideas(a)marco.sulla.e4ward.com > > >> > > >> > > >> Well, so all this discussion is only for freeing _one_ memory > location earlier? > > >> Seriously... as the other users already said, if someone really need > > >> it, he can use normal loops instead of comprehensions and split > > >> complex expression so temporary objects are immediately free. > > >> Furthermore, 2x speedup observations are done by benchmarks. In the > > >> real world, how can you be sure that L2 cache is not already filled > > >> up? :-) > > >> > > >> > > >> From: python-ideas-request(a)python.org > > >> Subject: confirm fc443c1d3d0ff72f9c645014ec4124ca207f5288 > > >> Date: March 12, 2020 at 5:47:03 PM PDT > > >> > > >> > > >> If you reply to this message, keeping the Subject: header intact, > > >> Mailman will discard the held message. Do this if the message is > > >> spam. If you reply to this message and include an Approved: header > > >> with the list password in it, the message will be approved for posting > > >> to the list. The Approved: header can also appear in the first line > > >> of the body of the reply. > > >> > > >> > > > [View Less]

1 0

Re: Make `del x` an expression evaluating to `x`
by Marco Sulla March 12, 2020

March 12, 2020

??? We are saying the same thing. On Thu, Mar 12, 2020 at 11:18 AM Marco Sulla via Python-ideas <python-ideas(a)python.org> wrote: > This speedup is observed only for large objects On Fri, 13 Mar 2020 at 00:45, Ben Rudiak-Gould <benrudiak(a)gmail.com> wrote: > it's only beneficial to try it when the arrays are large.

1 0

Re: Make `del x` an expression evaluating to `x`
by Dominik Vilsmeier March 12, 2020

March 12, 2020

> On Thu, 12 Mar 2020 at 21:10, Dominik Vilsmeier > <dominik.vilsmeier(a)gmx.de> wrote: >> If I wanted to split the computation over multiple lines and yet have it >> optimized I would just reuse the same (target) name instead of creating >> a temporary one and then discarding it in the next step: >> >> a = b * c >> a += d > This is __exactly__ the point: the numpy patch author said that, with > that patch, this is done … [View More]

1 0

Non-blocking concurrent.futures ThreadPoolExecutor
by Remy NOEL March 12, 2020

March 12, 2020

I ran into a problem which, according to a google search is not that uncommon: concurrent.futures ThreadPoolExecutor, installs an exit handler preventing program exit if any of the jobs it is running is blocked. (Which might be a surprising behaviour to someone using those). It is possible to workaround this problem by either unregistering the exit handler concurrent.futures.thread._python_exit or by subclassing the threadpoolExecutor and overriding the _adjust_thread_count method with one … [View More]

4 7

Re: Exception for parameter errors
by Marco Sulla March 10, 2020

March 10, 2020

On Wed, 4 Mar 2020 at 12:33, Chris Angelico <rosuav(a)gmail.com> wrote: > Being able to easily distinguish "that isn't callable" from "the > parameters don't line up" from "the function, during execution, raised > TypeError" would be useful. I **completely** agree. And I add that the change can also be non-breaking, if the new ArgumentError is simply a subclass of TypeError. Yes, it's incorrect, but this way no old code will break. PS: if you read the official docs (https://… [View More]

1 0

Re: Fwd: Possible Addition to Python Language: Marked Sub-condition
by Shrinivas Kulkarni March 9, 2020

March 9, 2020

Hi Rajeev and others Thanks for your responses. I completely overlooked the simple fact that the conditions are evaluated to boolean values and as such can be stored in variables. In the hindsight there may perhaps be a couple of (far-fetched) merits to my proposal (like useful in conditions with dynamic function calls or sparing users the effort of maintaining rather ephemeral variables) but they are of course not worth the effort of making a change to the language itself. So I am sorry for … [View More]the spam. Will be careful the next time. Best regards Shrinivas Kulkarni On Mon, Mar 9, 2020 at 10:21 AM Rajeev J Sebastian <rajeev.sebastian(a)alokin.in> wrote: > > Hi Shrinivas, > > Wouldn’t these just be variables? > > Cond1 = color == Blue and count == 20 > Cond2 = color == Red > > if cond1 or cond2: > Do something > elif cond3: > Do something else > > Regards > Rajeev J Sebastian > On 8 Mar 2020, 20:42 +0530, Shrinivas Kulkarni <shrinivk(a)gmail.com>, wrote: > > Hello Everyone > > While writing python code, I frequently come across the need to do > certain tasks based on combined conditions. > > Much of the task for all the sub-conditions are common but some are > specific to one or more of these sub-conditions. > > A simplified example: > > ########################## Code ########################## > if (color == BLUE and count == 20) or (color == RED and count % 5 == 0): > rotate_the_wheel() # Common to the two sub-conditions > if(color == BLUE and count == 20): # First sub-condition > set_signal() > if(color == RED and count % 5 == 0): # Second sub-condition > clear_signal() > proc_post_rotate() # Common to the two sub-conditions > > I am not sure if there is a better way to do this. If not, maybe there > can be an extension to the language, which would allow marking a > sub-condition just like we mark a sub-expression in a regular > expression. > > Tentative syntax for this could be ({} marks the sub-condition and > \number refers back to it): > > ########################## Code ########################## > if {(color == BLUE and count == 20)} or {(color == RED and count % 5 == 0)}: > rotate_the_wheel() > if(\1): # First marked condition > set_signal() > if(\2): # Second marked condition > unset_signal() > proc_post_rotate() > > And like sub-expressions, the nesting of marked sub-condions should > also be possible: > > ########################## Code ########################## > if {{(color == BLUE and count == 20)} and {value == 20}} or {(color == > RED and count % 5 == 0)}: > if(\1):# Refers to the entire subcondition {{(color == BLUE and > count == 20)} and {value = 20}} > proc1() > if(\2):# Refers to sub-subcondition {value == 20} > > This will not only avoid the repetition of sub-conditions, but make > code readable since something like \1 will give an immediate > indication of a sub-condition that's defined earlier. > > Please let me know something similar is already implemented. > Even otherwise, all your thoughts, inputs and criticism are welcome. > > Thanks and best regards > Shrinivas Kulkarni > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/M5OUL… > Code of Conduct: http://python.org/psf/codeofconduct/ [View Less]

2 1

Mailman 3 monthly summaries broken for Python-ideas
by André Roberge March 9, 2020

March 9, 2020

I am currently attempting to go through the entire archives of Python-ideas to make a summary of the discussions about adding a syntax for decimal literals in Python. In doing so, I encountered a weird bug which I find especially annoying since it makes me look like a spammer. If one goes through the archives by month, from May 2010 [1] to June 2011 [2], the first post shown at the top for each and every one of these monthly treads is one I wrote on May 15, 2010 [3]. This problem is not … [View More]

4 3

Re: Possible Addition to Python Language: Marked Sub-condition
by Paul Moore March 9, 2020

March 9, 2020

On Sun, 8 Mar 2020 at 15:02, Shrinivas Kulkarni <shrinivk(a)gmail.com> wrote: > > Hello Everyone > > While writing python code, I frequently come across the need to do > certain tasks based on combined conditions. > > Much of the task for all the sub-conditions are common but some are > specific to one or more of these sub-conditions. > > A simplified example: > > ########################## Code ########################## > if (color == BLUE and … [View More]

3 2

When type hint is used, Python forbids checking variable types at run time
by 铁皮石斛 March 8, 2020

March 8, 2020

An immature idea. Python now has a type prompt, so you can provide a parameter for the Python interpreter to control whether to open the check type operation. For example, in the code, I explicitly use the type prompt, such as: # Test1.py def foo(a: str, b: str) -> str: c = None # type:str return 'hi' We can run the code with a parameter, such as python -T test1.py To prevent the Python interpreter from detecting the variable types in it, so as to improve the running … [View More]

2 1

Fwd: Possible Addition to Python Language: Marked Sub-condition
by Shrinivas Kulkarni March 8, 2020

March 8, 2020

Hello Everyone While writing python code, I frequently come across the need to do certain tasks based on combined conditions. Much of the task for all the sub-conditions are common but some are specific to one or more of these sub-conditions. A simplified example: ########################## Code ########################## if (color == BLUE and count == 20) or (color == RED and count % 5 == 0): rotate_the_wheel() # Common to the two sub-conditions if(color == BLUE and count == 20): # … [View More]

1 0

Re: [Python-ideas] Allow star unpacking within an slice expression
by Neil Girdhar March 6, 2020

March 6, 2020

I didn't think of this when we were discussing 448. I ran into this today, so I agree with you that it would be nice to have this. Best, Neil On Monday, December 4, 2017 at 1:02:09 AM UTC-5, Eric Wieser wrote: > > Hi, > > I've been thinking about the * unpacking operator while writing some numpy > code. PEP 448 allows the following: > > values = 1, *some_tuple, 2 > object[(1, *some_tuple, 2)] > > It seems reasonable to me that it should be extended to … [View More]

2 2

More descriptive error message than "too many values to unpack"
by Alex Hall March 6, 2020

March 6, 2020

Currently this code: d = {"key": "value"} for key, value in d: pass produces this error: ValueError: too many values to unpack (expected 2) I suggest that the error message should also have: 1. The name of the type of the unpacked value 2. The length of the unpacked value, if it exists and (to not execute arbitrary __len__ implementations) if the type belongs to a safe subset, e.g. only builtin types. Then the error message could be: ValueError: too many values to unpack (expected 2, … [View More]

8 11

CSV Dictwriter - Handling escape characters
by allu.matikainen＠hotmail.com March 6, 2020

March 6, 2020

Hi, I would like if the DictWriter would be able to write escape characters (\n and \r and their combinations such as \n\n\n and \r\n) as literal characters. At the moment, DictWriter writes \n:s as new lines (shown in notepad or Excel) and I would like it to just write "\n" characters in my csv file. I asked a question about this on Stack Overflow but have had no good answers yet. My question also has an example of the data and its form. https://stackoverflow.com/questions/60073809/python-how-to-write-literal-n-…

6 5

IDEA: Allow override of all assignments of a built in type to be a different type
by Steve Barnes March 5, 2020

March 5, 2020

One of the lovely things about Python is that we have the capability to avoid issues such as the vagaries of the floating point type with libraries such as decimal and fractions. This is wonderous to me but comes with an issue that I suspect is limiting its usage. That issue is that you have to litter your code with constructors of those types, e.g. ``` a = 0.1 b = 0.2 c = 0.3 a + b == c # This doesn't work but it is easy to forget why ``` vs: ``` from decimal import * # I know why this is … [View More]

11 16

Exception for parameter errors
by Steven D'Aprano March 5, 2020

March 5, 2020

At the moment, when you call a function, if there is any mismatch between the function parameters and the supplied arguments, TypeError is raised. For example: # Too few arguments len() # Too many arguments ord('a', 'b') # Invalid keywords min([], func=len) etc. (There are probably others.) This makes it harder to do feature detection in multi-version code than it need do. Here's an example of feature detection that does work: # math.gcd requires exactly two … [View More]arguments prior to 3.9 # but soon to take arbitrary number of arguments in 3.9 try: gcd() except TypeError: ... But sometimes its not absolutely clear whether the TypeError is because I've passed the wrong number of arguments or the wrong type of argument: try: foobar(spam='a', eggs=0) except TypeError: ... When I read that, I have to check the documentation to see why a TypeError could be generated. Sure, a comment will help, but comments are notoriously untrustworthy: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'." --Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Proposal: add a new exception, ParameterError, for parameter errors. For backwards compatibility it would have to be a subclass of TypeError. Where the interpreter now raises TypeError for invalid parameters, it will switch to ParameterError. SyntaxErrors will remain SyntaxErrors. min(key=func, values) SyntaxError: positional argument follows keyword argument This will allow developers to distinguish genuine type errors (argument is the wrong type) from other calling errors (argument is the correct type, but passed in the wrong way). Should there be a series of separate exceptions for TooFewArguments, TooManyArguments, InvalidKeywordArgument, etc? No, I don't think so. We don't need to bloat the builtins with three (or more) new exceptions if only one will do. Normally it will be enough for the developer to read the error message, which will distinguish between the various kinds of parameter error. Most people won't care about programmatically distinguishing the cases. Reading the message will be enough. But for stability reasons (what if the error messages are localised, or change in the future?), we could give the exception a "reason" attribute, with a documented stable ID, and make that the official way to programmatically distinguish between the different kinds of parameter errors: except ParameterError as err: if err.reason == 1: # too few arguments The signature could be: ParameterError(*args, *, reason) Thoughts? -- Steven [View Less]

10 18

None should raise a new exception, NoneError
by Soni L. March 3, 2020

March 3, 2020

All operations on None should raise a NoneError, which should be a TypeError for backwards compatibility. try: x = a[b][c][d][e][f] + g.foo(h) except NoneError: x = None we can then look into merging the proposals of None-aware operators and Exception-aware operators such that the Exception-aware operators fallback to NoneError if no exception type is provided. we should also look into making "except" default to using NoneError instead of BaseException, with a future flag.

8 12

Incremental step on road to improving situation around iterable strings
by Steve Jorgensen March 3, 2020

March 3, 2020

From reading many of the suggestions in this list (including some of my own, of course) there seem to be a lot of us in the Python coder community who find the current behavior of strings being iterable collections of single-character strings to be somewhat problematic. At the same time, trying to change that would obviously have vast consequences, so not something that can simply be changed, even with at a major version boundary. I propose that we start by making a small, safe step in the … [View More]

23 65

Re: Incremental step on road to improving situation around iterable strings
by Bruce Leban March 2, 2020

March 2, 2020

--- Bruce On Mon, Feb 24, 2020 at 1:19 PM Alex Hall <alex.mojaki(a)gmail.com> wrote: > I think maybe you forgot to reply to the list as well? > > I did mean to reply to the list, so adding it back in. > How many scripts do you have where your own code directly iterated over a > string? How hard do you think it would be to update that code? > Actually quite a bit. I write a lot of code that manipulates words to create puzzles. In every one of those I use strings … [View More]as iterables. For what it's worth, in one of these programs, the problem I encountered was that strings were not sufficiently performant to solve a very complex problem. I modeled the strings as integers, and built the subset of needed string-like operations on them -- including iteration. If the warning comes from a dependency that hasn't updated, you can simply > add the appropriate code to filter that category of warnings. I think you > have the option of two lines of Python code or one environment variable. > That's not a good answer IMHO. Warnings should matter. Telling me I can filter them out means it shouldn't be there in the first place. And I have to filter them out because warnings that don't matter obscure the ones that do. > > As for your sample, this has been requested twice by others, here was my > response: > > That represents a misunderstanding of my position. I think I'm an outlier >> among the advocates in this thread, but I do not believe that implementing >> any of the ideas in this proposal would significantly affect code that >> lives in the long term. Some code would become slightly better, some >> slightly worse. My concern surrounds the user experience when debugging >> code that accidentally iterates over a string. So it's impossible for me to >> show you code that becomes significantly better because that's not what I'm >> arguing about, and it's unfair to say that quoting people who have >> struggled with these bugs is not evidence for the problem. >> > The claim was made by someone that certain code is "problematic" because the strings are iterable. Asking to see before and after code is certainly relevant, so we can see exactly what the "problem" is and how it gets better. Maybe the code gets *worse* and debugging gets easier. That tradeoff is relevant. And more to the point, is there anything that can't be done by a better linter instead of mucking up the language? > Similarly for jdveiga, I cannot give you "real workable surprising code" >> because I'm talking about code that isn't workable as a result of being >> surprising. I have given examples of real non-working surprising code, and >> I can give more, and if that's not indicative of a real problem then I'm >> very confused and would appreciate more explanation. > > > Now my question is, what is the worst damage that this change could cause? > Because I haven't heard an argument that goes deeper than aesthetics. > The worst damage? The change generates spurious warnings from millions of lines of correct code. It annoys people to the extent they no longer think Python is a stable platform to use to build code. Python dies. Sure that worst case isn't likely but you asked. The bar for changing such a core feature of Python is much higher than "the worst case isn't going to kill the language." In my opinion, declaring code deprecated when there is zero likelihood that it will ever be removed is counter-productive at best. --- Bruce [View Less]

2 1

Means of comparing slices for intersection or containment containment and computing intersections or unions
by Steve Jorgensen Feb. 29, 2020

Feb. 29, 2020

I am purposefully proposing this for slices as opposed to ranges because it is about the bounds of the slices, not the items in synthetic sequences. Also, slices can refer to any type of value, not just integers. Presumably, these operations would raise exceptions when used with slices that have `step` values other than `None`. Alternatively, those could hypothetically be valid in restricted cases such as when all properties are either int or Fraction types. Probably better to have them be … [View More]

4 7