Mailman 3 January 2018 - Python-ideas

Format mini-language for lakh and crore
by David Mertz 02 Feb '18

02 Feb '18

In South Asia, a different style of digit delimiters for large numbers is used than in Europe, North America, Australia, etc. With some minor spelling differences, the term lakh is used for a hundred-thousand, and it is generally written as '1,00,000'. In turn, a crore is 100 lakh, and is written as '1,00,00,000'. Extending this pattern, larger numbers continue to use two digits in groups (other than the smallest grouping of three digits. So, e.g. 1e12 is written as 10,00,00,00,00,000. It's nice that we now have the optional underscore in numeric literals. So we could write a number as either `12_34_56_78_00_000` or `1_234_567_800_000` depending on what region of the world and which convention was more familiar. However, in *formatting* those numbers, the format mini-language only allows the European convention. So e.g. In [1]: x = 12_34_56_78_00_000 In [2]: "{:,d}".format(x) Out[2]: '1,234,567,800,000' In [3]: f"{x:,d}" Out[3]: '1,234,567,800,000' In order to get Indian number delimiters, you'd have to write a custom formatting function, notwithstanding that something like 1.5 billion people use the three-then-two delimiting convention. I propose that Python should have an additional grouping option, or some other way to specify this grouping convention. Oddly, the '_' grouping symbol is available, even though no one actually uses that grouper outside of programming languages like Python, e.g.: In [4]: f"{x:_d}" Out[4]: '1_234_567_800_000' I guess this is nice for something like round-tripping numbers used in code, but it's not a symbol anyone uses "natively" (I understand why comma or period cannot be used in numeric literals since they mean something else in Python already). I'm not sure what symbol or combination I would recommend, but finding something suitable shouldn't be so hard. Perhaps now that backtick no longer has any other meaning in Python, it could be used since it looks similar to a comma. E.g. in Python 3.8 we might have: >>> f"{x:`d}" '12,34,56,78,00,000' (actually, this probably isn't any parser issue even in Python 2 since it's already inside quotes; but the issue is moot). Or maybe a two character version like: >>> f"{x:2,d}" '12,34,56,78,00,000' Or: >>> f"{x:,,d}" '12,34,56,78,00,000' Even if `2,` was used, that wouldn't preclude giving an additional length descriptor after it. Now we can have: >>> f"{x:,.2f}" '1,234,567,800,000.00' Perhaps in the future this would work: >>> f"{x:2,.2f}" '12,34,56,78,00,000.00' -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

9 21

Adding str.isascii() ?
by INADA Naoki 31 Jan '18

31 Jan '18

Hi. Currently, int(), str.isdigit(), str.isalnum(), etc... accepts non-ASCII strings. >>> s = １２３" >>> s '１２３' >>> s.isdigit() True >>> print(ascii(s)) '\uff11\uff12\uff13' >>> int(s) 123 But sometimes, we want to accept only ascii string. For example, ipaddress module uses: _DECIMAL_DIGITS = frozenset('0123456789') ... if _DECIMAL_DIGITS.issuperset(str): ref: https://github.com/python/cpython/blob/e76daebc0c8afa3981a4c5a8b54537f756e8… If str has str.isascii() method, it can be simpler: `if s.isascii() and s.isdigit():` I want to add it in Python 3.7 if there are no opposite opinions. Regrads, -- INADA Naoki <songofacandy(a)gmail.com>

12 41

Suggest adding struct iter_unpack allowing mixed formats & add iter_pack
by Steve Barnes 30 Jan '18

30 Jan '18

In my work I make a lot of use of struct pack & unpack but sometimes find the fact that I either have to supply the whole format string or add position tracking mechanisms to my code explicitly so as to use pack_to or unpack_from if the structures that I am dealing with are not simple arrays of a single type. In my particular use case of de/encoding messages from a remote device that is sometimes forwarding on collected information to & from the devices that it controls the endianness is not always consistent between message elements in a single message. It would be very nice if: a) iter_unpack.next produced an iterator that optionally took an additional parameter of a format string, (with endianness permitted), to replace the current format string in use. b) there was a matching iter_pack function that the next method took either more data to pack into the buffer extending it or a new format string, either as a named parameter or with an exposed set_fmt method on the packer. This would allow my specific use case and the ones where a buffer or stream is constructed of blocks consisting of a byte or word that specifies the data type(s) to follow (and sometimes a count) followed by the actual data. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

2 1

Logging: a more perseverent version of the StreamHandler?
by liam marsh 30 Jan '18

30 Jan '18

Hello, Some time ago, I set up some logging using stdout in a program with the `stdout_redirected()` context manager, which had to close and reopen stdout to work. Unsurprisingly, the StreamHandler didn't take it well. So I made a Handler class which is able to reload the stream (AKA get the new sys.stdout) whenever the old one wasn't writable. But there might be some more legit use cases for stubborn StreamHandlers like that (that are not ugly-looking, hopefully temporary patches). The way I see it for now is a StreamHandler subclass which, instead of having a `stream` argument, would have |`getStream`|,|`reloadStream`| and |`location`|, and would be used this way: On initialisation, it would load the stream object, then act as a regular StreamHandler, but checking that the stream is writable at each `handler.emit()` call. If it is not, then reload it. If given, the |`getStream`| (a callable object which returns a ready-to-use stream) argument is used to load/reload the underlying stream, else it would be fetched at the location described by `location`, and, if it is still not writable, call |`reloadStream()` |(which should put a usable stream object at |`location`|) then try to fetch it again. Here is the current implementation I have: |||```| from .config import _resolve as resolve # will (uglily) be used later |class ReloadingHandler(StreamHandler):| | """| | A stream handler which reloads the stream object from one place if an error occurs| | """| | def __init__(self, getStream=None, reloadStream=None, location=None):| | """| | Initialize the handler.| || | If stream is not specified, sys.stderr is used.| | """ self.getStream = getStream self.stream = None # to be overwritten later if getStream is None:| || if location is None: self.location = 'sys.stderr' # note the lack of 'ext://' self.reloadStream = None else: | self.reloadStream = reloadStream self.location = location stream = self.reload() # gets the stream StreamHandler.__init__(self, stream||) |||| def reload(self): if self.getStream is not None: stream = self.getStream() else: try: stream = resolve(self.location) exc = None except Exception as err: exc = err # is this really needed? stream = None # just retry for now if stream is None or not stream.writable(): if self.reloadStream is None: if exc: raise exc else: raise ValueError("ReloadingHandler couldn't reload a valid stream") stream = resolve(self.location) # if it fails this time, do not catch exception here return stream ||| | def emit(self, record):| | """| | Emit a record.| || | If a formatter is specified, it is used to format the record.| | The record is then written to the stream with a trailing newline. If| | exception information is present, it is formatted using| | traceback.print_exception and appended to the stream. If the stream| | has an 'encoding' attribute, it is used to determine how to do the| | output to the stream.| | """| | if not self.stream.writable():| | self.reload()| | StreamHandler.emit(self, record)| |||``` | What do you think? (about the idea, the implementation, and the way I wrote this email)| | --- L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast. https://www.avast.com/antivirus

3 3

.then execution of actions following a future's completion
by Daniel Collins 26 Jan '18

26 Jan '18

Hello all, So, first time posting here. I’ve been bothered for a while about the lack of the ability to chain futures in python, such that the next future will execute upon the first’s completion. So I submitted a pr to do this. This would add the .then(self, fn) method to concurrent.futures.Future. Thoughts? -dancollins34 Github PR #5335 bugs.python.org <http://bugs.python.org/> issue #32672

3 10

Official site-packages/test directory
by Stefan Krah 26 Jan '18

26 Jan '18

Hello, I wonder if we could get an official site-packages/test directory. Currently it seems to be problematic to distribute tests if they are outside the package directory. Here is a nice overview of the two main layout possibilities: http://pytest.readthedocs.io/en/reorganize-docs/new-docs/user/directory_str… I like the outside-the-package approach, mostly for reasons described very eloquently here: http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_… CPython itself of course also uses Lib/foo.py and Lib/test/test_foo.py, so it would make sense to have site-packages/foo.py and site-packages/test/test_foo.py. For me, this is the natural layout. Stefan Krah

10 16

Non-intrusive debug logging
by Larry Yaeger 26 Jan '18

26 Jan '18

Everyone uses logging during code development to help in debugging. Whether using a logging module or plain old print statements, this usually requires introducing one or (many) more lines of code into the model being worked on, making the existing, functional code more difficult to read. It is also easy to leave logging code in place accidentally when cleaning up. I have a possibly odd suggestion for dramatically improving debug logging, in an almost automated fashion, without any of the usual problems. I'm willing to jump through PEP hoops, but won't waste anyone's time arguing for the idea if others don't embrace it. In brief, make an extremely succinct comment, such as "#l" or "#dbgl", semantically meaningful. If appended to a line of code, such as: x = 1 Y = 2 Z = 3 x = y + z #l When executed the line would be logged (presumably to stdout) as something like: x=1 -> y=2 + z=3 -> 5 In one log line you see the variables and their values, the operations being performed with them, and the result of the evaluated expression. Placing "#l" on a def Function() line would apply "#l" logging to every line in that function or method. That's it. Adding these log requests would be easy and not clutter the code. Removing them reliably would be a trivial global operation in any text editor. The resulting debug information would be fully diagnostic. Of course there are lots of details and edge cases to consider. There always are. For example, what if x had not been previously defined? A reasonable answer: x=. -> y=2 + z=3 -> 5 In every case the solution should minimize volume of output while maximizing logged information. One can debate the precise formatting of the output. The text used for the semantically meaningful comment string is open to debate. The output pipe could be made controllable. There are natural extensions to allow periodic and conditional logging. I have ideas for reducing (already unlikely) collisions with existing comments. But I want to keep this simple for now, and I think this captures the core of the idea. For your consideration. - larryy

9 8

Preemptive multitasking and asyncio
by Thomas Güttler 25 Jan '18

25 Jan '18

I found a question and answer at Stackoverflow[1] which says that asyncio/await is like cooperative multitasking. My whish is to have preemptive multitasking: The interpreter does the yielding. The software developer does not need to insert async/await keywords into its source code any more. AFAIK the erlang interpreter does something like this. I guess it is impossible to implement this, but it was somehow important for me to speak out my which. What do you think? Regards, Thomas Güttler [1] https://stackoverflow.com/questions/38865050/is-await-in-python3-cooperativ… -- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines

5 5

__vars__ special method
by Yahya Abou 'Imran 25 Jan '18

25 Jan '18

On top of this old proposition: https://bugs.python.org/issue13290 We could have a __vars__ method that would be called by vars() if defined. The use cases: 1. let vars() work with instances without __dict__; 2. hide some attributes from the public API. Example for 1: class C: __slots__ = 'eggs', 'spam' def __vars__(self): d = {} for attr in self.__slots__: if hasattr(self, attr): d[attr] = getattr(self, attr) return d Exemple for 2: class C: def __vars__(self): return {attr: value for attr, value in self.__dict__.items() if not attr.startswith('_')}

2 2

Re: [Python-ideas] Support WHATWG versions of legacy encodings
by Chris Barker 22 Jan '18

22 Jan '18

On Tue, Jan 16, 2018 at 9:30 PM, Stephen J. Turnbull < turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > In what context? WHAT-WG's encoding standard is *all about browsers*. > If a codec is feeding text into a process that renders them all as > glyphs for a human to look at, that's one thing. The codec doesn't > want to fatal there, and the likely fallback glyph is something from > the control glyphs block if even windows-125x doesn't have a glyph > there. I guess it sort of makes sense. > sure it does -- and python is not a browser, and python itself has nothigni visual -- but we sure want to be abel to write code that produces visual representations of maybe messy text... if you're feeding a program ... > the codec has no idea when or how that's > going to get interpreted. sure -- which is why others have suggested that if WATWG is supported, then it *should* only be used for encoding, not encoding. But we are supposed to be consenting adults here -- I see no reason to prevent encoding -- maybe it would be useful for testing??? (as with JSON data, which I believe is > "supposed" to be UTF-8, but many developers use the legacy charsets > they're used to and which are often embedded in the underlying > databases etc, ditto XML), OK -- if developers do the wrong thing, then they do the wrong thing -- we can't prevent that! And Python's lovely "text is unicode" model actually makes that hard to do wong. But we do need a way to decode messy text, and then send it off to JSON or whatever properly encoded. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

5 5

Python-ideas January 2018