Mailman 3 November 2018 - Python-ideas

Enhancing range object string displays
by Steven D'Aprano 20 Nov '18

20 Nov '18

On the bug tracker, there is a proposal to enhance range objects so that printing them will display a snapshot of the values included, including the end points. For example: print(range(10)) currently displays "range(10)". The proposal is for the __str__ method to instead return "<range object [0, 1, ..., 8, 9]>". https://bugs.python.org/issue35200 print(range(2, 200, 3)) would display <range object [2, 5, ..., 194, 197]> Note that the original proposal was for range objects' __repr__ to display this behaviour. But given the loss of eval(repr(obj)) round tripping, and the risk of breaking backwards compatibility, it was decided that isn't acceptable but using the same display for __str__ (and hence produced by print) would be nearly as useful but without the downsides. The developer who proposed the feature, Julien, now wants to reject the feature request. I think it is still a useful feature for range objects. What do others think? Is this worth re-opening? -- Steve

6 6

Re: [Python-ideas] f-string "debug" conversion
by Rene Nejsum 17 Nov '18

17 Nov '18

+1 for this, I would use it all the time for debugging and tracing programs breakpoints and IDE’s can be nice, but my code is filled with lines like: logger.debug(f“transaction_id={transaction_id}, state={state}, amount={amount}, etc={etc}”) So yeah, well +10 actually :-) /rene

1 0

Proposing additions to the standard library
by Jonathan Crall 15 Nov '18

15 Nov '18

I'm interested in proposing several additions to the Python standard library, and I would like more information on the procedure for doing so. Are all additions done via a PEP? If not what is the procedure. If so, I've read that the first step was to email this board and get feedback. I have a library called `ubelt` that contains several tools that I think might be worthy of adding to the standard library. Here's my bullet point pitch: - Python is batteries included. Ubelt contains extra batteries. function are extra batteries. - Most function in ubelt are fast. All 222 tests takes 7.33 seconds. - Ubelt has 100% test coverage (sans `# nocover` locations). - I'm only championing a subset of the functions in ubelt. There are certainly functions in there that do not belong in the standard library. - I have a Jupyter notebook that give a demo of some select functions (not necessarily the same as the ones proposed here): https://github.com/Erotemic/ubelt/blob/master/docs/notebooks/Ubelt%20Demo.i… - I do have documentation (mostly in docstrings) and in the docs folder, but I've been having trouble auto-updating read-the-docs. Here is the link anyway: https://ubelt.readthedocs.io/en/latest/ Here is a tentative list of interesting functions. Hopefully the names are descriptive (if not, see docstrings: https://github.com/Erotemic/ubelt) ub.cmd ub.compressuser ub.group_items ub.dict_hist ub.find_duplicates ub.AutoDict ub.import_module_from_path ub.import_module_from_name ub.modname_to_modpath, ub.modpath_to_modname ub.ProgIter ub.ensuredir ub.expandpath almost everything in util_list: allsame, argmax, argmin, argsort, argunique, chunks, flatten, iter_window, take, unique These functions might be worth modifying into dictionary methods: ub.dict_subset ub.dict_take ub.map_vals ub.map_keys ub.Timerit ub.Timer Because I built the library, I tend to like all the functions. Its difficult to decide if they are stdlib worthy, so there might be some false positives / negatives. I'm on the fence about: CacheStamp, Cacher, NoParam, argflag, argval, dzip, delete, hash_data, hash_file, memoize, memoize_method, NiceRepr, augpath, userhome, ensure_app_cache_dir, ensure_app_resource_dir, find_exe, find_path, get_app_cache_dir, get_app_resource_dir, platform_cache_dir, platform_resource_dir, CaptureStdout, codeblock, ensure_unicode, hzcat, indent, OrderedSet Its my hope that some of these are actually useful. Let me know any of the following: what you think, if there are any questions, if something else needs to be done, or what the next steps are. -- -Jon

6 7

Relative Imports
by danish bluecheese 14 Nov '18

14 Nov '18

Hi all, Im tired of not being able to make relative imports freely. Now trying to develop a module which enable any project to use relative imports once it is loaded. Anybody interested in? Best,

6 12

Range and slice syntax
by Nicholas Harrison 13 Nov '18

13 Nov '18

I'm aware that syntax for ranges and slices has been discussed a good amount over the years, but I wanted to float an idea out there to see if it hasn't been considered before. It's not really original. Rather, it's a combination of a couple parts of Python, and I find it fascinatingly-consistent with the rest of the language. This will look similar to PEP 204, but there are some important differences and clarifications. (start:stop:step) Meet a range/slice object. Parentheses are required. (Its syntax in this regard follows exactly the same rules as a generator expression.) I say both range and slice because it can be used in either role. On the one hand, it is iterable and functions exactly like range(start, stop, step) in those contexts. On the other, it can also be passed into list indexing exactly like slice(start, stop, step). This is a proposal that range and slice are really the same thing, just in different contexts. Why is it useful? I at least find its syntax to be simple, intuitive, and concise -- more so than the range(...) or slice(...) alternatives. It's quite obvious for an experienced Python user and just as simple to pick up as slice notation for a beginner (since it *is* slice notation). It condenses and clears up sometimes-cumbersome range expressions. A couple examples: sum(1:6) # instead of sum(range(1, 6)) list(1:6) for i in (1:6): print(i**2) (i**2 for i in (1:6)) It also makes forming reusable slices clearer and easier: my_slice = (:6:2) # instead of slice(None, 6, 2) my_list[my_slice] It has a couple of siblings that should be obvious (think list or set comprehension): [start:stop:step] # gives a list {start:stop:step} # gives a set This is similar to passing a range/slice object into the respective constructor: [1:6] # list(1:6) or [1, 2, 3, 4, 5] {1:6} # set(1:6) or {1, 2, 3, 4, 5} Note that the parentheses aren't needed when it is the only argument of a function call or is the only element within brackets or braces. It takes on its respective roles for these bracket and brace cases, just like comprehensions. This also gives rise to the normal slice syntax: my_list[1:6:2] # What is inside the brackets is a slice object. my_list[(1:6:2)] # Equivalent. The parentheses are valid but unnecessary. So here's the part that requires a little more thought. Any of the values may be omitted and in the slice context the behavior has no changes from what it already does: start and stop default to the beginning or end of the list depending on direction and the step defaults to 1. In the range context, we simply repeat these semantics, but noting that there is no longer a beginning or end of a list. Step defaults to 1 (just like range or slice). Start defaults to 0 when counting up and -1 when counting down (just like slice). If stop is omitted, the object will act like an itertools.count object, counting indefinitely. I have found infinite iteration to be a natural and oft-desired extension to a range object, but I can understand that some may want it to remain separate and pure within itertools. I also admit that the ability to form an infinite list with only three characters can be a scary thought (though we are all adults here, right? ;). Normally you have to take a couple extra keystrokes: from itertools import count list(count()) # rather than just [:] If that is the case, then raising an error when iter() is called on a range/slice object with no stop value could be another acceptable course of action. The syntax will still be left valid. And that's mainly it. Slice is iterable or range is "indexable" and the syntax can be used anywhere successive values are desired. If you want to know what it does or how to use it in some case, just think, "what would a slice object do?" or "what would a range object do?" or "how would I write a generator expression/list comprehension here?". Here are a few more examples: for i in (:5): # 5 elements 0 to 4, i.e. range(5) print(i**2) for i in (1:): # counts up from one for as long as you want, i.e. count(1) print(i**2) if i == 5: break it = iter(:) # a convenient usage for an infinite counter next(it) ' '.join(map(str, (:5:2))) # gives '0 2 4' [(:5), (5:10)] # list of range/slice objects [[:5], [5:10]] # list of lists [*(:5), *(5:10)] # uses unpacking to get flat list [*[:5], *[5:10]] # same unpacking to get flat list Otherwise you'd have to do: [list(range(5)), list(range(5, 10))] # list of lists [*range(5), *range(5, 10)] # flat list Tuples: tuple(1:6:2) # (1, 3, 5) *(1:6:2), # same I don't actually have experience developing the interpreter and underlying workings of Python, so I don't know how much of a change this requires. I thought it might be possible since the constructs already exist in the language. They just haven't been unified yet. I also realize that there are a few other use-cases that need to be ironed out. The syntax might also be too minimal in some cases to be obvious. One of the trickiest things may be what it will be called, since the current language has the two different terms. In the end it's just another range/slice idea, and the idea has probably already been proposed sometime in the past few decades, but what thoughts are there? - Nicholas

8 16

Python octal escape character encoding "wats"
by Joao S. O. Bueno 10 Nov '18

10 Nov '18

I just saw some document which reminded me that strings with a backslash followed by 3 octal digits. When a backslash is followed by 3 octal digits, that means a character with the corresponding codepoint and all is well. The "valid scenaario": In [42]: "\777" Out[42]: 'ǿ' The problem is when you have just two valid octal digits In [40]: "\778" Out[40]: '?8' Which is ambiguous at least -- why is this not "\x07" "77" for example? (0ct(77) actually corresponds to the "?" (63) character) Or...when the first digit is not valid as octal - that is: In [41]: "\877" Out[41]: '\\877' And then when the second digit is not valid octal: In [43]: "\797" Out[43]: '\x0797' WAT? So, between the possibly ambiguous scenario with two octal digits followed by a no-octal digit, and the complety unexpected expansion to a 4-hexadecimal digit codepoint in the last case, what do you say of deprecating any r"\[0-9]{1,3}" sequence that don't match full 3 octal digits, and yield a syntax error for that from Python 3.9 (or 3.10) on? Best regards, js -><-

4 5

Re: [Python-ideas] Python-ideas Digest, Vol 144, Issue 24
by Alex Shafer 10 Nov '18

10 Nov '18

I think this about the limitation to . and .. possibly? > > Message: 6 > Date: Sat, 10 Nov 2018 11:00:16 +1100 > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] Relative Imports > Message-ID: > < > CAPTjJmrtApAXib4wM6FEh-XkpU_cf5RNX2TR0VzCAGgxPPgf5A(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > On Sat, Nov 10, 2018 at 10:52 AM danish bluecheese > <danish.bluecheese(a)gmail.com> wrote: > > > > ??? src > > ??? __init__.py > > ??? main.py > > ??? test > > ??? __init__.py > > ??? test_main.py > > > > assume the structure above. To be able to use relative imports with such > fundamental structure either i can go for sys.path hacks or could run as a > module from one further level parent. > > I do not like this :D want to be able to use relational imports freely > as soon as I provide correct relational path. > > > > Please let me know, if it is not clear on any aspect. > > The main thing that's not clear here is what you're proposing. What is > the idea under discussion? > > ChrisA > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > > > ------------------------------ > > End of Python-ideas Digest, Vol 144, Issue 24 > ********************************************* >

1 0

Add "default" kwarg to list.pop()
by Giampaolo Rodola' 09 Nov '18

09 Nov '18

Sorry in advance if this has been proposed in the past but I couldn't find anything on python-ideas: >>> l = [] >>> l.pop(default=1) 1 FWIW my use case consists in reading entries from /proc/diskstats where lines can have a variable number of fields depending on the kernel version: https://github.com/giampaolo/psutil/issues/1354#issuecomment-434495870 https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8ca… As such it would be convenient to set missing fields to 0 as "reads = fields.pop(default=0)" instead of catching IndexError every time. Extra: for consistency set.pop() should probably have the same. Thoughts? -- Giampaolo - http://grodola.blogspot.com

18 41

Re: [Python-ideas] Serialization of CSV vs. JSON
by David Shawley 07 Nov '18

07 Nov '18

On Nov 4, 2018, at 12:43 PM, Michael Selik <michael.selik(a)gmail.com> wrote: > If you're making a module >> > > On Sun, Nov 4, 2018, 5:49 AM David Shawley <daveshawley(a)gmail.com wrote: > > Personally, I would place this sort of serialization logic outside of the > > Standard Library -- maybe following the pattern that the rust community > > adopted on this very issue. In short, they separated serialization & > > de-serialization into a free-standing library. > > You don't need a bug on the tracker or discussion on -dev to share a module > on PyPI or GitHub. When you've got something started, share a link in this > thread. I modified a branch of python/cpython to implement what I had in mind. [1] The idea is to introduce a new protocol with a single method: self.jsonformat() -> object If this method exists, then json.encoder.JSONEncoder will call it to generate a JSON representation *instead* of calling *default*. This method must return a value that json.encoder.JSONEncoder can encoder or fail in the same manner as the *default* hook. The implementation wasn't too difficult once I learned a little more about how Standard Library classes are implemented when C speedups are included. There are a few things that I haven't done: 1. I didn't guard the functionality with a flag to the JSONEncoder initializer. This was oversight but I would add one before doing a PR against python/cpython. 2. As discussed before this is an asymmetric proposal since there is no support for detecting and de-serializing in JSONDecoder. That is what I had in mind. I'm not sure how we want to spell extension methods like this one. I chose to not use a double-underscore method since I view them as ``for use by the interpreter/language'' more so than for Library-recognised methods. The name is the least of my worries. Let me know if there is any reason that I shouldn't move forward with a bpo and PR against python/cpython. - cheers, dave. [1]: https://github.com/dave-shawley/cpython/pull/2 <https://github.com/dave-shawley/cpython/pull/2> -- "State and behavior. State and behavior. If it doesn’t bundle state and behavior in a sensible way, it should not be an object, and there should not be a class that produces it." eevee

2 1

Serialization of CSV vs. JSON
by Philip Martin 06 Nov '18

06 Nov '18

Is there any reason why date, datetime, and UUID objects automatically serialize to default strings using the csv module, but json.dumps throws an error as a default? i.e. import csv import json import io from datetime import date stream = io.StringIO() writer = csv.writer(stream) writer.writerow([date(2018, 11, 2)]) # versus json.dumps(date(2018, 11, 2))

10 17