Mailman 3 May 2020 - Python-ideas

Aside RE: Re: macaddress or networkaddress module
by Steve Barnes 13 Aug '20

13 Aug '20

As an aside I have a perfect example to back up what Paul is saying below. I work for a large corporation where developers are permitted to install python modules on their development machines, (subject to some licence restrictions), but the proxy settings required to get to PyPi vary from user to user, site to site, day to day (sometimes hour to hour) and connection type to connection type, e.g. If you are connected via WiFi you need to use a different proxy to that needed if you are on a wired connection and on a VPN (we have more than one in use) different again. To help with this I have put together a script that will try pip, if it fails will check the current location for pac.pac, get and parse it for proxies, correct a common incompatibility and then try each discovered proxy and if sill none work fall back to a list of common proxies. Once it finds a working setting it advises the user what to use in the current situation. Obviously, since the script is intended to work out how to get to PyPi, I needed to stick 100% to the standard libraries. -----Original Message----- From: Paul Moore <p.f.moore(a)gmail.com> Sent: 06 April 2020 09:11 To: Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp> Cc: Pete Wicken <petewicken(a)gmail.com>; Python-Ideas <python-ideas(a)python.org> Subject: [Python-ideas] Re: macaddress or networkaddress module On Mon, 6 Apr 2020 at 04:14, Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > That's OK for developers on their "own" machines, but I think there > are still a lot of folks who need approval from corporate (though I > haven't heard from Paul Moore on that theme in quite a while, maybe > it's gotten better?) Howdy :-) It's actually got a bit worse, to the point where I don't bother trying to fight the system as much any more, so things have gone quieter because of that :-) But yes, I still believe that there are important use cases for code that only uses the stdlib, and "not being able to get to the internet" is a valid use case that we shouldn't just dismiss. On the other hand, with regard to the comment you were replying to: > Up to around a decade ago, installing third-party libraries > was a huge mess, but nowadays, PyPI works, Python comes with pip > pre-installed, and telling developers they need internet access > when they first start a new project isn’t considered onerous. I think all of that is true. What I think is more important, though, is around things that aren't so much "projects" as "scripts", and that *distributing* such scripts that depend on 3rd party libraries is still problematic. Here, I'm not talking about making full-fledged packages, or py2exe style fully bundled apps, but "here's this monitoring script I wrote, let's ship it across 10 production systems and why don't you try it out on your dev boxes?" There's a *significant* step change in complexity between doing that if the script depends on just the stdlib, versus if it depends on a 3rd party module. And we don't have a really good answer to that yet (zipapps, and tools like shiv that wrap zipapps, are pretty good in practice, but they are not well known or commonly used). Paul _______________________________________________ Python-ideas mailing list -- python-ideas(a)python.org To unsubscribe send an email to python-ideas-leave(a)python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5BN5D… Code of Conduct: http://python.org/psf/codeofconduct/

3 3

PEP 472 -- Support for indexing with keyword arguments
by Andras Tantos 11 Aug '20

11 Aug '20

Hello! I'm not sure I'm addressing the right audience here, so please direct me to the appropriate channel if that's the case... My name is Andras Tantos and I'm working on a Python library desscribing HW designs. I came across this problem of __getitem__ and co. not supporting kwargs. Apparently this extension was proposed and rejected as PEP 472. Apart from my use-case, which is arguably a corner-case and not worth modifying the language for, I believe there are two important use-cases that are worth considering with the latest improvements in the language: 1. With the recent type-hint support, the feature could be made way more descriptive if this PEP got implemented. For example, instead of doing the following: def func(in: Dict[str, int]) one could write: def func(in: Dict[key=str, value=int]) 2. It would also make 'generic classes' much cleaner to implement, similar to the way type-hints look. Consider the following code: class_Generic(object): Specializations = [] @classmethod def__getitem__(cls, *args): name = f"Generic_{len(cls.Specializations)}" Specialized = type(name, (cls,), {"specials": tuple(args)}) cls.Specializations.append(Specialized) returnSpecialized def__init__(self, value= None): self.value = value def__str__(self): ifhasattr(self, "specials"): return(f"[{type(self)}- "+ ",".join(str(special) forspecial inself.specials) + f"] - {self.value}") else: return(f"[{type(self)}- GENERIC"+ f"] - {self.value}") Generic = _Generic() #g = Generic() - fails because of no specialization is given s1 = Generic[12]() s2 = Generic[42]("Hi!") print(s1) print(s2) Running this simple example results in: python3 -i python_test.py [<class '__main__.Generic_0'> - 12] - None [<class '__main__.Generic_1'> - 42] - Hi! You can see how the specialized parameters got passed as well as the ones to '__init__'. Obviously, in real code the idea would be to filter generic parameters and set up 'Specialized' with the right set of methods and arguments. Now, without kwargs support for __getitem__, it's impossible to pass named arguments to the specialization list, which greatly limits the usability of this notation. I don't know how convincing these arguments and use-cases are for you, but could you advise me about how to start the 'ball rolling' to drum-up support for re-activating this PEP? Thanks again, Andras Tantos

24 74

Improve handling of Unicode quotes and hyphens
by Steve Barnes 06 Aug '20

06 Aug '20

Hi All, Apologies if this has already been discussed to death. Python 3 allows Unicode characters in strings and identifiers but the actual quotation marks are only accepted in plain ASCII, i.e. the following all successfully initialise strings: ``` S1 = "Double Quoted" # Opened and closed with chr(34)0x22 S2 = 'Single Quoted' # Opened and closed with chr(39)0x27 ``` But the following all result in an error - "SyntaxError: invalid character in identifier": ``` S1 = "Double Quoted" # Opened with \u201c and closed with \u201d S2 = 'Single Quoted' # Opened with \u2018 and closed with \u2019 ``` To the experienced eye, and depending on the character font used, it is "obvious" what the problem is! The wrong quotation marks were used. The big problem, especially for beginners, is that the same keys were typed, just in the "wrong" editor or even the wrong editor mode or context I have found that in Outlook if the font is FixSys or I am replying to a plain text email it is fine but otherwise it is "helpful" - unfortunately, especially on Windows, "wrong" editors abound and include, but are not limited to, MS-Outlook, MS-Word, some online editing environments such as Quora. On top of that is the helpful substitution of a m-hyphen for minus when you press space a word later so: A = 3 - 2 # With a space syntax error due to \u2013 A = 3 - 2 # No Space or CR after I last typed it is OK as 0x2d Use cases that catch people out: 1. Sending a code snipped by email using Outlook 2. User manuals written in MS-Word - (many peoples work environment) 3. Articles published on Quora - people expect to be able to copy and paste the code for some reason. I am sure that many us have encountered these issues or similar. What can be done? 1. Persuade Microsoft, and others, to stop being so helpful by default - good luck with that! 2. Tell all users that they need to use a "proper" editor or IDE - This seems like adding an additional barrier to new & casual users. 3. Better yet tell them to use a "proper" OS like .... - At the very least many of us have to use Windows at work. 4. Start accepting hyphens as minus & Unicode quotation marks - this would be the ideal answer for pasted code but has a lot of possible things to iron out such as do we require that the quotes match and are in the typographically correct order. It is also quite a big & complex change to the python interpreter. 5. Normalise the input to the python interpreter (at least for these characters and possibly a few others) so that entering or reading from a file S1 = "Double Quoted" becomes S1 = "Double Quoted", etc. - this should be a easier change to the interpreter but, from a purist point of view, could be said to make us as bad as the others because we are not honouring what the user entered. 6. Change the error message "SyntaxError: invalid character in identifier" to include which character and it's Unicode value so that it becomes "SyntaxError: invalid character 0x201c " in identifier" - this is almost certainly the easiest change and fits well with explicit is better than implicit but still leaves it to the user to correct the erroneous input (which could be argued is both good and bad). I would like to suggest that an incremental approach might be the best - clarifying the existing error message being the thing that should not break anything and either substituting for problem characters or processing them "properly" as a later enhancement. Steve Barnes

16 46

python interpreter docker builder
by Antonio Cavallo 15 Jul '20

15 Jul '20

Hi is there any interest (or anyone has done it before), building the python interpreter using docker? The basic idea is building the toolchain (gcc) and on top of that the python interpreter. On mac/linux it can build natively, but it can use docker to target linux from mac/windows. Thanks

5 15

Python __main__ function
by redradist＠gmail.com 29 Jun '20

29 Jun '20

Hi all, In Python we often use the following syntax to call the main logic of script when it was ran: ```python def main(): pass # whatever should be done for `python ./script.py` if __name__ == '__main__': main() ``` Maybe it is a time to introduce the new module level function like __main__ ? Consider the following code: ```python def __main__(): pass # whatever should be done for `python ./script.py` ``` It is much easy an less code to write ;) Under-hood `Python` just will generate the following code: ```python def __main__(): pass # whatever should be done for `python ./script.py` # Below generated code if __name__ == '__main__': __main__() ``` If there are two `if __name__ == '__main__':` it is also not an issue: ```python def __main__(): pass # whatever should be done for `python ./script.py` def main(): pass # whatever should be done for `python ./script.py` if __name__ == '__main__': main() # Below generated code if __name__ == '__main__': __main__() ``` Or we could require user to have only one `if __name__ == '__main__':` ... What do you think, please share your opinion ...

14 20

Make `del x` an expression evaluating to `x`
by Eric Wieser 12 Jun '20

12 Jun '20

TL;DR: should we make `del x` an expression that returns the value of `x`. ## Motivation I noticed yesterday that `itertools.combinations` has an optimization for when the returned tuple has no remaining ref-counts, and reuses it - namely, the following code: >>> for v in itertools.combinations([1, 2, 3], 1): ... print(id(v)) ... del v # without this, the optimization can't take place 2500926199840 2500926199840 2500926199840 will print the same id three times. However, when used as a list comprehension, the optimization can't step in, and I have no way of using the `del` keyword >>> [id(v) for v in itertools.combinations([1, 2, 3], 1)] [2500926200992, 2500926199072, 2500926200992] `itertools.combinations` is not the only place to make this optimization - parts of numpy use it too, allowing a = (b * c) + d to elide the temporary `b*c`. This elision can't happen with the spelling bc = b * c a = bc + d My suggestion would be to make `del x` an expression, with semantics "unbind the name `x`, and evaluate to its value". This would allow: >>> [id(del v) for v in itertools.combinations([1, 2, 3], 1)] [2500926200992, 2500926200992, 2500926200992] and bc = b * c a = (del bc) + d # in C++, this would be `std::move(bc) + d` ## Why a keyword Unbinding a name is not something that a user would expect a function to do. Functions operate on values, not names, so `move(bc)` would be surprising. ## Why `del` `del` already has "unbind this name" semantics, and allow it to be used as an expression would break no existing code. `move x` might be more understandable, but adding new keywords is expensive ## Optional extension For consistency, `x = (del foo.attr)` and `x = (del foo[i])` could also become legal expressions, and `__delete__`, `__delattr__`, and `__delitem__` would now have return values. Existing types would be free to continue to return `None`.

15 50

PYTHONLOGGING env variable
by Bar Harel 06 Jun '20

06 Jun '20

Another idea I've had that may be of use: PYTHONLOGGING environment variable. Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level. Another option would be that -x dev or a different -x logging will initialize basic config. Will be useful mostly for debugging purposes instead of temporarily modifying the code. Kinda surprised it doesn't exist tbh. Bar Harel

9 22

Advanced Debugger Support C-API is incomplete?
by Andrew Pashkin 06 Jun '20

06 Jun '20

I want to implement my own version of _PyThread_CurrentFrames() <https://github.com/python/cpython/blob/e838a9324c1719bb917ca81ede8d766b5cb5…>, but it seems like it's not possible using public C-API. The comment <https://github.com/python/cpython/blob/e838a9324c1719bb917ca81ede8d766b5cb5…> in the source code of _PyThread_CurrentFrames() says that lists of interpreters and threads can be mutated even when GIL is held and so HEAD_LOCK() should be used. At the same time Advanced Debugger Support <https://docs.python.org/3/c-api/init.html#advanced-debugger-support> API exposes access to these lists but there is no way (or I don't see how) to achieve the same thing that HEAD_LOCK() does. Am I missing something or Advanced Debugger Support is incomplete? -- With kind regards, Andrew Pashkin. cell phone - +375 (44) 492-16-85 Skype - waves_in_fluids e-mail - andrew.pashkin(a)gmx.co.uk

1 1

Optional keyword arguments
by James Lu 02 Jun '20

02 Jun '20

Many a python programmer have tired to see code written like: def bar(a1, a2, options=None): if options is None: options = {} ... # rest of function syntax if argument is not passed, evaluate {} and store to options def foo(options:={}): pass syntax if argument is not passed or is None, evaluate {} and store to options* def foo(options?={}): pass The Zen of Python states "there shouldn't be two ways to do the same thing." Thus, only one of ":=" or "?=" should be adopted. They should be evalued on: - Which encourages writing better code? - Which catches mistakes easier? Do we want to encourage callers to pass None to indicate default arguments? spam = { data: True } if arg else None bar(a1, a2, param=spam) versus bar(a1, a2, { data: True }) if arg else bar(a1, a2) versus _ = foo.curry(a1, a2) _({data: True}) if arg else _(a1, a2) Since Python is a strongly typed language, it seems more consistent to me that this code should throw an error: def getoptions(): ... # code to get options # whoops! missing return statement #return os.environ foo(a1, a2, param=getoptions()) := should be adopted because it catches mistakes more quickly. On the other hand, ?= replaces the "if kwarg is not None: kwarg = ..." idiom. (I propose adopting only ":=". I show "?=" as a strawman.)

23 151

How to propose a change with tests where the failing test case (current behaviour) is bad or dangerous
by Steve Barnes 01 Jun '20

01 Jun '20

There is a long standing issue, (or at least I consider it a poor feature some others consider it compliance with other existing tools), with a couple of python standard libraries that I would like to raise an issue for & PR(s) to address but when it comes to putting tests in place the failing test case can potentially crash the host machine, sometimes in an unrecoverable manner, due to the potential to generate an infinite size file. Is there a way of proposing a change with tests that refuse to run in some situations or can be terminated based on time or resource limits being exceeded? Steve Barnes

9 21