Mailman 3 May 2018 - Python-ideas

Let try-except check the exception instance
by Danilo J. S. Bellini 01 Jun '18

01 Jun '18

Hi! I was working on handling some exceptions from external software (e.g. database constraint triggers) switching the handler based on the messages that had been sent. Today we can do something like (running on Python 3.6.5): >>> try: ... # [...] ... session.commit() # Here it raises! ... # [...] ... except DatabaseError as exc: ... msg = get_db_error_msg_from_exception(exc) ... if msg == "beyond_limit": ... # [...] ... elif msg == "no_funds": ... # [...] ... else: ... raise That works, but I'd like to do something like: >>> try: ... # [...] ... except BeyondLimit: ... # [...] ... except NoFunds: ... # [...] Creating classes to "match" the exception in their __instancecheck__. Well, I tried to do so. A simplified example would be: >>> class MsgCheckerMeta(type): ... def __instancecheck__(cls, instance): ... return str(instance) == cls.internal ... >>> class ExceptionHasMessage(Exception, metaclass=MsgCheckerMeta): ... internal = "message" Using these new classes, we would get this: >>> try: ... raise Exception("message") ... except ExceptionHasMessage: ... print("Yeah!") ... Traceback (most recent call last): File "<stdin>", line 2, in <module> Exception: message Yet, >>> isinstance(Exception("message"), ExceptionHasMessage) True >>> try: ... raise Exception("message") ... except Exception as exc: ... print(isinstance(exc, ExceptionHasMessage)) ... True The idea is to allow catching exceptions beyond checking their MRO, using a class that checks the exception instance by implementing a custom __instancecheck__. -- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap)

14 16

Re: [Python-ideas] Let try-except check the exception instance
by Ken Hilton 31 May '18

31 May '18

In one of my own packages, I use the following to catch dynamic error messages: def handle_error(exc): # do stuff with exc with catch('error_code', handle_error): # do stuff that can raise an error The definition of "catch" is as follows, assuming the root exception actually raised is called "CodedError" whose "code" attribute is the error code: from contextlib import contextmanager @contextmanager def catch(code=None, caught=None, always=None): try: yield except CodedError as exc: if isinstance(code, str): is_caught = exc.code == code else: is_caught = exc.code in code #assume it's a container if (code is not None) and not is_caught: raise if caught is not None: caught(exc) finally: if always is not None: always() Perhaps you could make use of this? Suggesting, Ken Hilton;

1 0

Proposal: A Reduce-Map Comprehension and a "last" builtin
by Peter O'Connor 31 May '18

31 May '18

Dear all, In Python, I often find myself building lists where each element depends on the last. This generally means making a for-loop, create an initial list, and appending to it in the loop, or creating a generator-function. Both of these feel more verbose than necessary. I was thinking it would be nice to be able to encapsulate this common type of operation into a more compact comprehension. I propose a new "Reduce-Map" comprehension that allows us to write: signal = [math.sin(i*0.01) + random.normalvariate(0, 0.1) for i in range(1000)] smooth_signal = [average = (1-decay)*average + decay*x for x in signal from average=0.] Instead of: def exponential_moving_average(signal: Iterable[float], decay: float, initial_value: float=0.): average = initial_value for xt in signal: average = (1-decay)*average + decay*xt yield average signal = [math.sin(i*0.01) + random.normalvariate(0, 0.1) for i in range(1000)] smooth_signal = list(exponential_moving_average(signal, decay=0.05)) I've created a complete proposal at: https://github.com/petered/peps/blob/master/pep-9999.rst , (and a pull-request <https://github.com/python/peps/pull/609>) and I'd be interested to hear what people think of this idea. Combined with the new "last" builtin discussed in the proposal, this would allow u to replace "reduce" with a more Pythonic comprehension-style syntax. - Peter

22 65

Re: [Python-ideas] A real life example of "given"
by Adam Bartoš 30 May '18

30 May '18

A side comment: > potential_updates = { > y: potential_update > for x in need_initialization_nodes > for y in [x, *x.synthetic_inputs()] > given potential_update = command.create_potential_update(y) > if potential_update is not None} Probably, I would write @make_dict def potential_updates(): for x in need_initialization_nodes: for y in [x, *x.synthetic_inputs()]: potential_update = command.create_potential_update(y) if potential_update is not None: yield y, potential_update If such pattern covered a lot of examples, even some syntax sugar could be added. For example: foo[0].potential_updates = dict(_) from: for x in need_initialization_nodes: for y in [x, *x.synthetic_inputs()]: potential_update = command.create_potential_update(y) if potential_update is not None: yield y, potential_update where <statement> from: <suite> would be equivalent to def _(): <suite> _ = _() <statement> Best regards, Adam Bartoš

1 0

Add shutil.chown(..., recursive=False)
by Giampaolo Rodola' 30 May '18

30 May '18

...as in (not tested): def _rchown(dir, user, group): for root, dirs, files in os.walk(dir, topdown=False): for name in files: chown(os.path.join(root, name), user, group) def chown(path, user=None, group=None, recursive=False): if recursive and os.path.isdir(path): _rchown(dir, user, group) ... It appears like a common enough use case to me ("chown -R path"). Thoughts? -- Giampaolo - http://grodola.blogspot.com

8 16

Was `os.errno` undocumented?
by Petr Viktorin 30 May '18

30 May '18

Hello, Python 3.7 removes the undocumented internal import `os.errno`. We consider that a an implementation detail, which can be changed *without notice* even in a bugfix release. Projects that depend on it are incorrect and should be fixed. On bpo-33666, there's a debate on whether the removal should be mentioned in release notes, on the grounds that it broke some projects, is used in quire a few tutorials/books/examples, and it's been working since Python 2.5 or so. But here's the thing: the more I think about this, the less I consider `os.errno` as "undocumented". Here's what I consider a reasonable path a programmer might go through: # Where do I find errno values? # Maybe it's in `os`, like all other basic platform bindings? >>> import os >>> os.err<tab> os.errno os.error( >>> help(os.errno) Help on built-in module errno: ... # Yup, There it is! Even more serious: # Hmm, this old example on some website tells me to use `os.errno`. # It's the first time I'm hear about that, so I'll be extra careful; >>> import os >>> os.errno <module 'errno' (built-in)> # Yup, it's there! Let's double-check the docs. >>> help(os.errno) Help on built-in module errno: ... # Yup, looks quite alright! Let's use it! As you can see, the built-in documentation does not contain *any* warnings against using `os.errno`. You might think the fact that it's called "errno" and not "os.errno" is a red flag, but it's not, really -- "os.path" is (on my system) named "posixpath", yet "os.path" is the name to use. While most people might prefer searching docs.python org to `help()`, editors are getting better and better to presenting introspection and the built-in docs, so more and more people are preferring `pydoc`-ish docs to going online. I don't think we can reasonably expect people who used built-in help as above to go back and check that Python's official docs *do not* contain `os.errno`. Effectively, while `os.errno` is not very *discoverable* using official docs alone, I don't think calling it *undocumented* is fair. So, removing it without notice is not very friendly to our users. Is that reasoning sound? Should our policy on removing internal imports take that into account?

10 16

Mashup the existing statement grammars to capture predicates
by Carl Smith 29 May '18

29 May '18

This is another suggestion for new syntax for assigning a name to the value of the predicate in an if, elif or while statement. It still uses `as` for its keyword, but with (more flexible) params instead of a direct assignment. It mashes up the if/while, def/class and for-in grammars, so it still looks like Python, and boils down to this: if|elif|while <predicate> as (<params>): <suite> If the params contain one simple name (the required minimum), the value of the predicate is assigned to that name. In any other case, the value must be a sequence, which gets unpacked: while input('$ ').split() as (command, *args): if run(command, parse(args)) as (result): render(result) else: sys.exit() -- Carl Smith carl.input(a)gmail.com

2 3

shutil zero-copy and exotic filesystems
by Giampaolo Rodola' 29 May '18

29 May '18

Hello, I've been working on a patch which speeds up shutil.copy* operations for all 3 major platforms (Linux, Windows, OSX): https://bugs.python.org/issue33671 Since the speedup is quite consistent I'd love to see this merged in, but considering shutil.copy* is quite crucial I wanted to hear other folk's opinion first. Attached patch attempts to use platform-specific zero-copy syscalls [1] by default and fallbacks on using plain read() / write() variant in case of immediate failure. In theory this should work fine, in practice I haven't tested it on exotic (e.g. network) filesystems. In order to diminish risks of breakage I think if it would make sense to: - add a global shutil.NO_ZEROCOPY variable defaulting to False - add a "no_zerocopy" argument to all functions involving a copy (copyfile(). copy(), copy2(), copytree(), move()) Thoughts? [1] sendfile() (Linux), fcopyfile() (OSX), CopyFileW (Windows) since the matter is a bit sensitive in terms of potential breakage on exotic / untested (e.g. network) filesystems I want to raise some attention about: https://bugs.python.org/issue33671 Attached patch attempts to use platform-specific zero-copy syscalls by default and fallbacks on using plain read() / write() copy in case of immediate failure. In order to diminish risks I think it would make sense to: - add a global shutil.NO_ZEROCOPY variable defaulting to False - add a "no_zerocopy" argument to all functions involving a copy (copyfile(). copy(), copy2(), copytree(), move()) Thoughts? -- Giampaolo - http://grodola.blogspot.com

3 4

Re: [Python-ideas] Keyword for direct pass through of kwargs to super
by Michael Lohmann 28 May '18

28 May '18

[Steven D'Aprano] > obj = Aardvark(27, spam=3, eggs=5, cheese=True) > > So you look up help(Aardvark), and it tells you that the signature is > > Aardvark.__init__(self, foo) > > What the hell? If Aardvark.__init__ only takes a single argument This is wrong! This would at some point down the line throw an error TypeError: __init__() got an unexpected keyword argument 'eggs‘ (or at some point **kwargs are being accepted and not passed on to super which would be terrible on its own). The whole point I was trying to make is: If it doesn’t make any sense to init a class with **kwargs: why write down that it would (or even **could**) accept them? Shouldn’t the init tell you something like 'If you instantiate this class then this is everything you can give me'? Well, right now in addition it says 'Just give me anything with keywords‘. I would think that something like 'Oh, and if I am in the middle of an MRO: I will pass everything down the line‘ would be a much better description of it’s true intentions instead. [Carl Smith] > By using **kargs in the constructor and the call > to `super`, you are indicating that the signature passes through But can you be certain? Couldn’t someone have just used a `kargs.pop('eggs')`? Okay. You could argue that they probably wouldn’t have done so. But for making automated documentation it probably would be useful to make sure it didn’t happen. I think that (as Raymond Hettinger once said) 'super is super', but can’t you make it a bit smarter with telling it: 'Hey - If you don’t expect 'eggs', keep calm, it probably (or rather certainly) wasn’t meant for you so just pass it on to your super'. Best, Michael

6 17

multiprocessing, freeze support by default for Pool
by Bradley Phillips 28 May '18

28 May '18

Hi All There are about 347 results for the following query on the google search engine: "python multiprocessing pyinstaller crash site:stackoverflow.com" The windows fork bomb is a known frequent occurrence. I know that freeze_support is supposed to be called but this issue still catches me out most times (and many others). I propose adding to line 117 of multiprocessing/context.py if (sys.platform == 'win32'): self.freeze_support() Thanks Regards Brad

2 1