Mailman 3 February 2018 - Python-ideas

List the methods of a metaclass in the help of its instances
by Yahya Abou 'Imran Feb. 11, 2018

Feb. 11, 2018

For example, in the help of ABCs, the register method witch is defined in ABCMeta is not listed. It's a little bit weird, since it's accessible from the class just like any classmethod or staticmethod, and it's a real feature of the class. I think it would be great if there was sections in the help that look like: | ---------------------------------------------------------------------- | Metaclass methods defined in <TheMetaclass>: | ---------------------------------------------------------------------- | Metaclass methods inherited from <TheMetaclassParent>: The same rules of visibility would apply of course.

1 0

Easing set-up of of console python applications
by Mario Corchero Feb. 10, 2018

Feb. 10, 2018

Hello All! I got asked how to configure the logging stack to be able to output directly to console using both stdout and stderr and I could not really find a great answer as adding both as StreamHandlers will result in error and above messages going to stdout. The usecase is having a cli or app that wants to log to console as other tools. Errors and bove to stderr and normal information to stdout. I know the recommended way in Python is to just use print on simple scripts, but it can happen that you import a library you want to see the logs of, and therefore you need to set the logging stack. I have drafted two implementations but I am open to suggestions: 1) A "Console Handler" that uses multiple streams and chooses based on the level. 2) An inverted filter that can be used to filter everything above info for the stdout Stream handler. What do you people think? If people like it I'll send an issue + PR. Regards, Mario Corchero

2 4

Consider generalizing Decimal to support arbitrary radix
by Neil Girdhar Feb. 8, 2018

Feb. 8, 2018

Arbitrary radix comes up every now and then and Decimal already has a radix() method. It would be nice when initializing a Decimal object to be able to specify an arbitrary radix>=2.

7 14

Re: [Python-ideas] Complicate str methods
by Franklin? Lee Feb. 8, 2018

Feb. 8, 2018

On Feb 8, 2018 13:06, "Serhiy Storchaka" <storchaka(a)gmail.com> wrote: 08.02.18 12:45, Franklin? Lee пише: > Could it be that re uses an optimization that can also be used in str? > CPython uses a modified Boyer-Moore for str.find: > https://github.com/python/cpython/blob/master/Objects/string > lib/fastsearch.h > http://effbot.org/zone/stringlib.htm > Maybe there's a minimum length after which it's better to precompute a > table. > Yes, there is a special optimization in re here. It isn't free, you need to spend some time for preparing it. You need a special object that keeps an optimized representation for faster search. This makes it very unlikely be used in str, because you need either spend the time for compilation on every search, or use some kind of caching, which is not free too, adds complexity and increases memory consumption. Note also in case of re the compiler is implemented in Python. This reduces the complexity. The performance of the one-needle case isn't really relevant, though, is it? This idea is for the multi-needle case, and my tests showed that re performs even worse than a loop of `.find`s. How do re and .find scale with both number and lengths of needles on your machine?

1 0

Re: [Python-ideas] Complicate str methods
by Franklin? Lee Feb. 8, 2018

Feb. 8, 2018

On Thu, Feb 8, 2018 at 11:24 AM, Steve Dower <steve.dower(a)python.org> wrote: > Easily fixed by installing one of the alternate regex libraries. MRAB's regex library, the most prominent alternative, does not use the linear-time search algorithm. The only libraries I know that do are the ones with re2, though I haven't looked deeply. Let it be known that I tried to install pyre2 (re2 on PyPI) on Ubuntu For Windows for the tests, and after hours of no success, I decided that the alternative libraries were not the point. I eventually got it working (https://github.com/axiak/pyre2/issues/51), and here are the results: # Same setup as before, with findsub_re2 using `find=re2.search, esc=re2.escape`. pattern2 = re2.compile('|'.join(map(re.escape, needles))) %timeit findsub(haystack, needles) #=> 1.26 ms ± 15.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) %timeit findsub_re(haystack, needles) #=> 745 ms ± 19.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit findsub_re_cached(haystack, pattern) #=> 733 ms ± 16.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit findsub_regex(haystack, needles) #=> 639 ms ± 12.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit findsub_re2(haystack, needles) #=> 34.1 ms ± 1.13 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) %timeit findsub_re_cached(haystack, pattern2) #=> 9.56 µs ± 268 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In any case, installing re2 is even more of an advanced solution for a basic problem than using re. > re performance and its edge cases have been discussed endlessly. Please look > it up before restarting that discussion. I'm not trying to restart the discussion. I'm trying to say that the assumptions being made about its superior performance are unfounded. Two members have suggested that re is the performant option, and that's just not true.

1 0

Consider making Decimal's context use PEP 567
by Neil Girdhar Feb. 7, 2018

Feb. 7, 2018

Decimal could just pull its Context object from a context variable rather than having to pass it in to all functions. This would be similar to how numpy works.

3 3

Re: [Python-ideas] Possible Enhancement to py Launcher - set default
by Paul Moore Feb. 7, 2018

Feb. 7, 2018

On 7 February 2018 at 19:35, Steve Dower <steve.dower(a)python.org> wrote: > Checking the Version (!=SysVersion) property should be enough (and perhaps > we need to set it properly on install). The launcher currently only works > with PythonCore entries anyway, so no need to worry about other distros. Fair enough. But there was a separate proposal to make the launcher handle non-PythonCore cases - there's a risk of conflicting feature requests here :-) > PEP 514 allows for other keys to be added as well (it specifies a minimum > set), so we could just set one for this. “NoDefaultLaunch” or similar. Sure - it could be purely a launcher convention, rather than having to be specifically noted in PEP 514. Paul

1 0

Possible Enhancement to py Launcher - set default
by Steve Barnes Feb. 7, 2018

Feb. 7, 2018

When a new version of python is in alpha/beta it is often desirable to have it installed for tests but remain on a previous version for day to day use. However, currently the Windows py launcher defaults to the highest version that it finds, which means that unless you are very careful you will end up having to explicitly specify your older version every time that you start python with it once you have installed the newer version. I an thinking that it would be relatively simple to expand the current launcher functionality to allow the user to set the default version to be used. One possible syntax, echoing the way that versions are displayed with the -0 option would be to allow py -n.m* to set and store, either in the registry, environment variable or a configuration file, the desired default to be invoked by py or pyw. Personally I thing that this would encourage more people to undertake testing of new candidate releases of python. I would be interested in any feedback on the value that this might add. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

8 18

Support WHATWG versions of legacy encodings
by Rob Speer Feb. 6, 2018

Feb. 6, 2018

Hi! I joined this list because I'm interested in filling a gap in Python's standard library, relating to text encodings. There is an encoding with no name of its own. It's supported by every current web browser and standardized by WHATWG. It's so prevalent that if you ask a Web browser to decode "iso-8859-1" or "windows-1252", you will get this encoding _instead_. It is probably the second or third most common text encoding in the world. And Python doesn't quite support it. You can see the character table for this encoding at: https://encoding.spec.whatwg.org/index-windows-1252.txt For the sake of discussion, let's call this encoding "web-1252". WHATWG calls it "windows-1252", but notice that it's subtly different from Python's "windows-1252" encoding. Python's windows-1252 has bytes that are undefined: >>> b'\x90'.decode('windows-1252') UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 0: character maps to <undefined> In web-1252, the bytes that are undefined according to windows-1252 map to the control characters in those positions in iso-8859-1 -- that is, the Unicode codepoints with the same number as the byte. In web-1252, b'\x90' would decode as '\u0090'. This may seem like a silly encoding that encourages doing horrible things with text. That's pretty much the case. But there's a reason every Web browser implements it: - It's compatible with windows-1252 - Any sequence of bytes can be round-tripped through it without losing information It's not just this one encoding. WHATWG's encoding standard ( https://encoding.spec.whatwg.org/) contains modified versions of windows-1250 through windows-1258 and windows-874. Support for these encodings matters to me, in part, because I maintain a Unicode data-cleaning library, "ftfy". One thing it does is to detect and undo encoding/decoding errors that cause mojibake, as long as they're detectible and reversible. Looking at real-world examples of text that has been damaged by mojibake, it's clear that lots of text is transferred through what I'm calling the "web-1252" encoding, in a way that's incompatible with Python's "windows-1252". In order to be able to work with and fix this kind of text, ftfy registers new codecs -- and I implemented this even before I knew that they were standardized in Web browsers. When ftfy is imported, you can decode text as "sloppy-windows-1252" (the name I chose for this encoding), for example. ftfy can tell people a sequence of steps that they can use in the future to fix text that's like the text they provided. Very often, these steps require the sloppy-windows-1252 or sloppy-windows-1251 encoding, which means the steps only work with ftfy imported, even for people who are not using the features of ftfy. Support for these encodings also seems highly relevant to people who use Python for web scraping, as it would be desirable to maximize compatibility with what a Web browser would do. This really seems like it belongs in the standard library instead of being an incidental feature of my library. I know that code in the standard library has "one foot in the grave". I _want_ these legacy encodings to have one foot in the grave. But some of them are extremely common, and Python code should be able to deal with them. Adding these encodings to Python would be straightforward to implement. Does this require a PEP, a pull request, or further discussion?

21 84

Format mini-language for lakh and crore
by David Mertz Feb. 1, 2018

Feb. 1, 2018

In South Asia, a different style of digit delimiters for large numbers is used than in Europe, North America, Australia, etc. With some minor spelling differences, the term lakh is used for a hundred-thousand, and it is generally written as '1,00,000'. In turn, a crore is 100 lakh, and is written as '1,00,00,000'. Extending this pattern, larger numbers continue to use two digits in groups (other than the smallest grouping of three digits. So, e.g. 1e12 is written as 10,00,00,00,00,000. It's nice that we now have the optional underscore in numeric literals. So we could write a number as either `12_34_56_78_00_000` or `1_234_567_800_000` depending on what region of the world and which convention was more familiar. However, in *formatting* those numbers, the format mini-language only allows the European convention. So e.g. In [1]: x = 12_34_56_78_00_000 In [2]: "{:,d}".format(x) Out[2]: '1,234,567,800,000' In [3]: f"{x:,d}" Out[3]: '1,234,567,800,000' In order to get Indian number delimiters, you'd have to write a custom formatting function, notwithstanding that something like 1.5 billion people use the three-then-two delimiting convention. I propose that Python should have an additional grouping option, or some other way to specify this grouping convention. Oddly, the '_' grouping symbol is available, even though no one actually uses that grouper outside of programming languages like Python, e.g.: In [4]: f"{x:_d}" Out[4]: '1_234_567_800_000' I guess this is nice for something like round-tripping numbers used in code, but it's not a symbol anyone uses "natively" (I understand why comma or period cannot be used in numeric literals since they mean something else in Python already). I'm not sure what symbol or combination I would recommend, but finding something suitable shouldn't be so hard. Perhaps now that backtick no longer has any other meaning in Python, it could be used since it looks similar to a comma. E.g. in Python 3.8 we might have: >>> f"{x:`d}" '12,34,56,78,00,000' (actually, this probably isn't any parser issue even in Python 2 since it's already inside quotes; but the issue is moot). Or maybe a two character version like: >>> f"{x:2,d}" '12,34,56,78,00,000' Or: >>> f"{x:,,d}" '12,34,56,78,00,000' Even if `2,` was used, that wouldn't preclude giving an additional length descriptor after it. Now we can have: >>> f"{x:,.2f}" '1,234,567,800,000.00' Perhaps in the future this would work: >>> f"{x:2,.2f}" '12,34,56,78,00,000.00' -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

9 21

JSON encoding protocol with __json__ dunder method
by Kiss, György Feb. 1, 2018

Feb. 1, 2018

Hi! Most of the classes (even if very simple like datetime.datetime) cannot be serialized to JSON by default. Would it be a good idea for the default json.JSONEncoder to call the __json__ dunder method automatically if the object has one? I can't find anything about why this protocol or PEP doesn't exists yet. Currently almost everyone implements it like this, there are thousands of results on GitHub to this: https://github.com/search?l=Python&q=def+__json__&type=Code&utf8=%E2%9C%93 but there is no canonical way (Python/standard way) to do this. It would be very nice, because a custom JSONEncoder would not be needed and everyone could implement JSON serialization in One True Way (yay for code reuse!) The implementation could be very simple, would look something like this: class JSONEncoder: def default(self, obj): if hasattr(obj, '__json__'): return obj.__json__() return current_implementation Or to make it easier even for decoding, it could be __to_json__ and __from_json__ or __json_encode__ and __json_decode__ or something like these. Was there already a PEP/discussion about this? György

2 1