Mailman 3 October 2021 - Python-ideas

os.workdir() context manager
by Marc-Andre Lemburg 22 Oct '21

22 Oct '21

Hello all, I sometimes write Python scripts which need to work in specific work directories. When putting such code into functions, the outer function typically does not expect the current work dir (CWD) to be changed, so wrap the code which need the (possibly) modified CWD using a simply context manager along the lines of: class workdir: def __init__(self, dir): self.dir = dir def __enter__(self): self.curdir = os.getcwd() os.chdir(self.dir) def __exit__(self, *exc): os.chdir(self.curdir) return False Would there be interest in adding something like this to the os module as os.workdir() ? Example: def backup_home_dir(account): with os.workdir(os.path.join('/home', account)): # Create a backup of the account dir, rooted at the account's # home dir restic.backup(repo, '.') Notes: - The context manager is not thread safe. There's no thread safe model for the current work dir. OTOH, scripts usually don't use threads, so not a big deal. - The context manager could be made more elaborate, e.g. adding optional logging for debugging purposes. - The same could be added to pathlib's objects as .workdir() method returning a context manager. Thoughts ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Sep 14 2021) >>> Python Projects, Coaching and Support ... https://www.egenix.com/ >>> Python Product Development ... https://consulting.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/

17 37

Fwd: [Python-Dev] Re: Semi-proposal: Tagged None
by David Mertz, Ph.D. 21 Oct '21

21 Oct '21

I actually thought this thread was on pyhon-ideas where it is now appropriate. Chris quotes everything I wrote, and his comments are all useful, so let me move starting with that. ---------- Forwarded message --------- From: Chris Angelico <rosuav(a)gmail.com> Date: Thu, Oct 21, 2021, 8:42 PM Subject: [Python-Dev] Re: Semi-proposal: Tagged None To: Python-Dev <python-dev(a)python.org> On Fri, Oct 22, 2021 at 3:23 AM David Mertz, Ph.D. <david.mertz(a)gmail.com> wrote: > > On Thu, Oct 21, 2021 at 2:52 AM Steven D'Aprano <steve(a)pearwood.info> wrote: >> >> On Tue, Oct 19, 2021 at 05:09:42PM -0700, Michael Selik wrote: >> > None and its ilk often conflate too many qualities. For example, is it >> > missing because it doesn't exist, it never existed, or because we never >> > received a value, despite knowing it must exist? > > >> >> 30+ years later, and we cannot easily, reliably or portably use NAN >> payloads. Most people don't care. If we offerred them a dozen or a >> thousand distinct sentinels for all the various kinds of missing data, >> how many people would use them and how many would just stick to plain >> old None? > > > In data science, I have been frustrated by the sparsity of ways of spelling "missing value." Might be worth redirecting this to -ideas. > Besides the distinction Michael points out, and that Steven did in relation to NaNs with payloads, I encounter missingness of various other sorts as well. Crucially, an important kind of missing data is data where the value I received seems unreliable and I have decided to *impute* missingness rather than accept a value I believe is unreliable. > > But there is also something akin to what Michael points out (maybe it's just an example). For example, "middle name" is something that some people simply do not have, other people choose not to provide on a survey, and others still we just don't know anything beyond "it's not there." > And some people have more than one (I have a brother with two of them). Not the best example to use, since names have WAY more complexities than different types of absence, but there are other cases where that sort of thing comes up. For instance, if someone says on a survey that s/he is in Australia, and then you ask for a postcode, then leaving it blank should be recorded as "chose not to provide"; but if the country is listed as Timor-Leste / East Timor, then "not applicable" would be appropriate, since the country doesn't use postal codes. > Of course, when I impute missingness, I can do so at various stages of data cleaning, and for various different reasons or confidences. None (or NaN) are sort of OK, but carrying metadata as to the nature of missingness would be nice. > Right. Using postcodes as an example again, for someone in Australia, a postcode of "E3B 0H8" doesn't make sense, as that isn't the format we use. So you could wipe that out and replace it with "No postal code, malformed data entered". > So my strawman suggestion is tagging None's. I suppose spellings like `None[reason]` or `None(reason)` are appealing. > > An obvious problem that I recognize is that it's not obvious this can "play nice" with the common idiom `if mydata is not None: ...`. None really is a singleton, and a "tagged singleton" or "annotated singleton" probably doesn't work well with Python's object model. > > My goal, of course, would be to have TaggedNone be a kind of subclass of None, in the same way that bool is a subclass of int, and hence True is a kind of 1. However, I'd want a large number of custom None's, with some sort of accessible string or numeric code or something to inspect which one it was. > But this is where I start to disagree. None should remain a singleton, but "no data available" could be its own thing, tied in with the way that you do your data storage and stats. As such, you wouldn't be checking it with 'is', so you wouldn't have that problem (the Python 'is' operator will only ever test for actual object identity). Keep None simple and dependable, and then "Missing Data" can be an entire class of values if you so desire. ChrisA _______________________________________________ Python-Dev mailing list -- python-dev(a)python.org To unsubscribe send an email to python-dev-leave(a)python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6NY5NQC… Code of Conduct: http://python.org/psf/codeofconduct/

3 4

Making imports callable
by Abdur-Rahmaan Janhangeer 21 Oct '21

21 Oct '21

Greetings list, Let's say i import module1 import module1 can we do module1() straight out of the box? If not i propose something like if __name__ == '__main__': __file__.callable = True Kind Regards, Abdur-Rahmaan Janhangeer about <https://compileralchemy.github.io/> | blog <https://www.pythonkitchen.com> github <https://github.com/Abdur-RahmaanJ> Mauritius

2 3

PEP able? : Request new 'fn' format specifier to specify relationship between positional arguments
by Shash Srinivasa 21 Oct '21

21 Oct '21

Hi, Draft PEP attached: Proposal: def is_between(n1,n2,n3): fn"is number {n1} between {n2} and {n3}" fn"{n2}<{n1}<{n3}?" # the token "fn" immediately following the def statement, speficy the descriptive text # multipe "fn" specifiers describes multiple ways the funtion can be invoked # the "fn" statement is at the same indentation level as the def statement return (n1 > n2 and n1 < n3) . . . # if "fn" statements are found indented and not immediately following def statement, indicate function invocation. # function can be invoked using descriptive text pecified in "fn" statement. if fn"is number {value} between {valid} and {low_threshold}": print("underflow") if is_between(value,low_threshold,high_threshold): print("in range") if fn"{low_threshold}>{value}>{high_threshold} ?" print("overflow") . . Please comment Thanks Shash

1 0

Implementing string unary operators
by Jeremiah Vivian 20 Oct '21

20 Oct '21

I posted a previous thread about overloading the unary `+` operator in strings with `ord`, and that expanded to more than just the unary `+` operator. So I'm saying now, there should be these implementations: > +string - `int(string, 10)` (or just `int(string)`) > -string - `int(string, 8)` > ~string - `int(string, 16)` Or: > +string - `string.lstrip()` > -string - `string.rstrip()` > ~string - `string.strip()` If anyone has better ideas, they can post it here.

15 41

Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?
by Alex Waygood 19 Oct '21

19 Oct '21

Indeed — we essentially lie to mypy about the method resolution order for list, dict, etc (mypy thinks that list directly inherits from collections.abc.MutableSequence — see the typeshed stub here: https://github.com/python/typeshed/blob/32bc2161a107db20c2ebc85aad31c29730d…), but numeric ABCs are not special-cased by typeshed in the same way as collections ABCs. Fundamentally, mypy has no knowledge about virtual subclassing. Again, I covered this very comprehensively on StackOverflow ;) https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-… Best, Alex > On 16 Oct 2021, at 13:54, Steven D'Aprano <steve(a)pearwood.info> wrote: > > On Sat, Oct 16, 2021 at 09:54:13AM +0100, Alex Waygood wrote: >>>> On 16 Oct 2021, at 06:13, Steven D'Aprano <steve(a)pearwood.info> wrote: >>> Be careful about believing what you are told. >> Indeed, MyPy will correctly raise errors if you assign {None: []} to a >> variable annotated with dict[str, Number]. However, you'll find that >> MyPy also raises an error if you assign {'foo': 4} to a variable >> annotated with dict[str, Number]: > > Hah, serves me right for not testing it for both positive and negative > cases. > > In my very limited testing now, I see that the problem appears to be > with the Number type. mypy correctly accepts this: > > Data = Dict[str, int] > a: Data = {'spam': 42} > > (no errors), but if you change the annotation to use Number instead of > int, it wrongly flags that as a type error. > > Possibly mypy doesn't know that ints and floats are instances of Number? > > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FJY3Q… > Code of Conduct: http://python.org/psf/codeofconduct/

4 4

Real Positional Arguments or OO Currying
by Mathew Elman 19 Oct '21

19 Oct '21

I don't know if this has been suggested before, or if this is outlandishly impossible (though I would be surprised if it was), so apologies in advance if so. I have on occasion come across a situation where I use/write a signature like this: def insert_x_into_y(x, y): ... or worse def insert_into(item, container): ... where, despite a driving idea of python syntax being readability in english, the function signature is distinctly not english. "I'll just go and insert into this item that container", is not only never said but is actually ambiguous in english. What would be really cool, is if python let you write function signatures like this: def insert_(item)_into_(container): ... where the arguments dispersed between the function name are positional only argument, and any key word arguments would have to go at the end. It would create a function that could be called as: insert_(1)_into_(my_list) or insert__into_(1, my_list) The purpose of allowing both should be obvious - so that the function can be referenced and called in other places. (Rather than just skipping the brackets the function call with only the end parentheses could have a special stand in character e.g. ., ?, !, _ or other if that was more preferred.) This sort of signature is particularly annoying for boolean checks like `isinstance` (N.B. I am _not_ suggesting changing any builtins), which one could wrap with: def is_(obj)_an_instance_of_(type): return isinstance(obj, type) For precedence in other languages, this is similar to curried functions in functional languages e.g Haskell, especially if each part of a function were to be callable, which would be up for debate. Allowing each part to be called would make sense if each "next" partial function were an attribute on the previous and what it returned, making it a sort of object oriented currying. Then the syntax could be with a `.`: def is_(obj)._an_instance_of_(type): ... is_(1)._an_instance_of_(int) is_._an_instance_of_(1,int)

9 27

Accessing target name at runtime
by Jeremiah Paige 19 Oct '21

19 Oct '21

Hello Python community, Earlier this year I put forward a proposal for decorators on variables which I felt was well received in concept, but fell flat on the syntax. One of the responses to that thread was rather than abusing the decorator, to use a new symbol to get the target of an assignment statement as a string on the right hand side. I would call this compile-time reflection of assignment targets, or simply reflection. Although initially continuing on with my original syntax, the idea of using such a token in my own code came up on more than one occasion this year and I wanted to see what it might really look like. From that, this proposal is to add a new token to Python, currently spelled <<<, which, at compile time, is replaced with the string representation of the nearest assignment target, either assignment statement or assignment expression. It does not bind with either keyword arguments or default parameters. Neither does it work in any kind of non-assignment binding. When encountered anywhere other than the right hand side of some assignment, it is an unconditional SyntaxError. Having access to the target at runtime is most often useful in meta programming and the standard library has several factory functions that require the user to repeat the variable name in the function call to get expected behavior. Bellow are some examples of where I believe the reflection token would be used if adopted. >>> Point = namedtuple(<<<, 'x, y, z') >>> Point <class '__main__.Point'> >>> UUIDType = NewType(<<<, str) >>> UUIDType __main__.UUIDType >>> class Colors(Enum): ... Black = <<< ... GRAY = <<< ... WHITE = <<< ... >>> Colors.GRAY.value 'GRAY' >>> HOME = '$' + <<< >>> HOME '$HOME' >>> if error := response.get(<<<): ... if errorcode := error.get(<<<): ... print(f"Can't continue, got {errorcode=}") ... Can't continue, got errorcode=13 To get a feel for using this new token I have created a fork of the 3.11 alpha that implements a *very* incomplete version of this new grammar, just enough to actually produce all of the examples above. It also passes a small new test suite with further examples https://github.com/ucodery/cpython/blob/reflection/Lib/test/test_reflection… .

7 17

Except * formatting
by Rob Cliffe 17 Oct '21

17 Oct '21

I'm sending this to python-list because my emails to python-dev keep getting bounced back (after a few days delay). I've no idea why. Instead of `except group ...`, what about `except for ...`? No new keywords. Reads naturally in English. Hints that there is more than one kind of exception coming. Best wishes Rob Cliffe

2 1

Re: Structure Pattern for annotation
by Alex Waygood 17 Oct '21

17 Oct '21

I agree with Steven. I very much like Abdulla's proposed syntax for dicts, TypedDicts and sets. But I'm not sure that the idea for `Annotated` is workable, and the proposal for lists seems too prone to ambiguity, given how extensively square brackets are already used in typing syntax. One question about the proposals, however: how would we represent other mappings apart from dicts? Would `collections.abc.Mapping` and `types.MappingProxyType` still be spelled as `Mapping[str, int]` and `MappingProxyType[str, int]`? If so, that would be a slightly annoying inconsistency. I do also quite like the idea of an improved syntax for tuples specifically. Tuples are already very different types to lists/strings/etc in the context of typing, essentially representing heterogeneous structs of fixed length rather than homogenous sequences of unspecified length. As such, I think it "makes sense" to special-case tuples without special-casing other sequences such as list, `collections.abc.Sequence` or `collections.deque`. def foo() -> tuple[list[list[str]], list[list[str]]] would become def foo() -> (list[list[str]], list[list[str]]) That feels quite readable and natural, to me. Best, Alex > On 14 Oct 2021, at 09:25, Steven D'Aprano <steve(a)pearwood.info> wrote: > On Thu, Oct 14, 2021 at 12:32:57AM +0400, Abdulla Al Kathiri wrote: > >> Today I found myself write a function that returns a tuple of list of >> list of strings (tuple[list[list[str]], list[list[str]]]). Wouldn’t it >> easier to read to write it like the following: >> ([[str]], [[str]])? > > Not really. Your first example is explicit and I can get the meaning by > just reading it out loud: > > tuple[list[list[str]], list[list[str]]] > > "tuple (of) list (of) list (of) str, list (of) list (of) str > > Your abbreviated version: > > ([[str]], [[str]]) > > is too terse. I have to stop and think about what it means, not just > read it out loud. Without the hint of named types (tuple and list), my > first reaction to seeing [str] is "is this an optional string?". > > And then I wonder why it's not written: > > ([[""]], [[""]]) > > Why abbreviate list and tuple but not string? > > Code is read more than it is written, and can be too terse as well as > too verbose. > > On the other hand: > >> Similarly for TypedDict, replace the following.. >> class Movie(TypedDict): >> name: str >> year: int >> with >> {‘name’: str, ‘year’: int} > > To my eye, that one does work. As far as I know, curly brackets {} > aren't otherwise used in annotations (unlike square brackets), and they > don't look like "optional" to me. They look like a dict. > > So on first glance at least, I think that: > > {'name': str, 'year': int} > > is better than the class syntax we already have. > > > Likewise: > >> dict[str, int] will be {str: int} >> set[int] will be {int}. > > work for me too. > > >> Also, Annotated[float, “seconds”] can be replaced with something like >> float #seconds indicating whatever comes after the hashtag is just >> extra information similar to a normal comment in the code. > > No, because the # indicates that the rest of the line is a comment. This > is already legal: > > def func(d: {str # this is an actual comment > : int}) -> Any: ... > > so this would be ambiguous between a real comment and an annotation. > > Even if we agreed to change the behaviour of comments, you suggested: > > func(d: {str # product label: [float] # prices from 2000 to 2015}) > > How is the interpreter to know that the first annotation is just > > "product label" > > rather than this? > > "product label: [float] # prices from 2000 to 2015" > > So I don't think this works. > > > -- > Steve > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/T7E54… > Code of Conduct: http://python.org/psf/codeofconduct/

7 8