Python-ideas

python-ideas@python.org

67 participants
5256 discussions

Fwd: Simple curl/wget-like download functionality in urllib (like http offers server)

by Tom P

In the context of building Docker images, it is often required to download stuff. If curl/wget is available, great, but often slim images don't include that. The urllib could provide a very simple download functionality (like http offers a simple server): from urllib.request import urlopen data = urlopen('https://.../install-poetry.py').read() # print or save data

1 day, 2 hours

os.workdir() context manager

by Marc-Andre Lemburg

Hello all, I sometimes write Python scripts which need to work in specific work directories. When putting such code into functions, the outer function typically does not expect the current work dir (CWD) to be changed, so wrap the code which need the (possibly) modified CWD using a simply context manager along the lines of: class workdir: def __init__(self, dir): self.dir = dir def __enter__(self): self.curdir = os.getcwd() os.chdir(self.dir) def __exit__(self, *exc): os.chdir(self.curdir) return False Would there be interest in adding something like this to the os module as os.workdir() ? Example: def backup_home_dir(account): with os.workdir(os.path.join('/home', account)): # Create a backup of the account dir, rooted at the account's # home dir restic.backup(repo, '.') Notes: - The context manager is not thread safe. There's no thread safe model for the current work dir. OTOH, scripts usually don't use threads, so not a big deal. - The context manager could be made more elaborate, e.g. adding optional logging for debugging purposes. - The same could be added to pathlib's objects as .workdir() method returning a context manager. Thoughts ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Sep 14 2021) >>> Python Projects, Coaching and Support ... https://www.egenix.com/ >>> Python Product Development ... https://consulting.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/

1 day, 5 hours

Fwd: [Python-Dev] Re: Semi-proposal: Tagged None

by David Mertz, Ph.D.

I actually thought this thread was on pyhon-ideas where it is now appropriate. Chris quotes everything I wrote, and his comments are all useful, so let me move starting with that. ---------- Forwarded message --------- From: Chris Angelico <rosuav(a)gmail.com> Date: Thu, Oct 21, 2021, 8:42 PM Subject: [Python-Dev] Re: Semi-proposal: Tagged None To: Python-Dev <python-dev(a)python.org> On Fri, Oct 22, 2021 at 3:23 AM David Mertz, Ph.D. <david.mertz(a)gmail.com> wrote: > > On Thu, Oct 21, 2021 at 2:52 AM Steven D'Aprano <steve(a)pearwood.info> wrote: >> >> On Tue, Oct 19, 2021 at 05:09:42PM -0700, Michael Selik wrote: >> > None and its ilk often conflate too many qualities. For example, is it >> > missing because it doesn't exist, it never existed, or because we never >> > received a value, despite knowing it must exist? > > >> >> 30+ years later, and we cannot easily, reliably or portably use NAN >> payloads. Most people don't care. If we offerred them a dozen or a >> thousand distinct sentinels for all the various kinds of missing data, >> how many people would use them and how many would just stick to plain >> old None? > > > In data science, I have been frustrated by the sparsity of ways of spelling "missing value." Might be worth redirecting this to -ideas. > Besides the distinction Michael points out, and that Steven did in relation to NaNs with payloads, I encounter missingness of various other sorts as well. Crucially, an important kind of missing data is data where the value I received seems unreliable and I have decided to *impute* missingness rather than accept a value I believe is unreliable. > > But there is also something akin to what Michael points out (maybe it's just an example). For example, "middle name" is something that some people simply do not have, other people choose not to provide on a survey, and others still we just don't know anything beyond "it's not there." > And some people have more than one (I have a brother with two of them). Not the best example to use, since names have WAY more complexities than different types of absence, but there are other cases where that sort of thing comes up. For instance, if someone says on a survey that s/he is in Australia, and then you ask for a postcode, then leaving it blank should be recorded as "chose not to provide"; but if the country is listed as Timor-Leste / East Timor, then "not applicable" would be appropriate, since the country doesn't use postal codes. > Of course, when I impute missingness, I can do so at various stages of data cleaning, and for various different reasons or confidences. None (or NaN) are sort of OK, but carrying metadata as to the nature of missingness would be nice. > Right. Using postcodes as an example again, for someone in Australia, a postcode of "E3B 0H8" doesn't make sense, as that isn't the format we use. So you could wipe that out and replace it with "No postal code, malformed data entered". > So my strawman suggestion is tagging None's. I suppose spellings like `None[reason]` or `None(reason)` are appealing. > > An obvious problem that I recognize is that it's not obvious this can "play nice" with the common idiom `if mydata is not None: ...`. None really is a singleton, and a "tagged singleton" or "annotated singleton" probably doesn't work well with Python's object model. > > My goal, of course, would be to have TaggedNone be a kind of subclass of None, in the same way that bool is a subclass of int, and hence True is a kind of 1. However, I'd want a large number of custom None's, with some sort of accessible string or numeric code or something to inspect which one it was. > But this is where I start to disagree. None should remain a singleton, but "no data available" could be its own thing, tied in with the way that you do your data storage and stats. As such, you wouldn't be checking it with 'is', so you wouldn't have that problem (the Python 'is' operator will only ever test for actual object identity). Keep None simple and dependable, and then "Missing Data" can be an entire class of values if you so desire. ChrisA _______________________________________________ Python-Dev mailing list -- python-dev(a)python.org To unsubscribe send an email to python-dev-leave(a)python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6NY5NQC… Code of Conduct: http://python.org/psf/codeofconduct/

1 day, 20 hours

Making imports callable

by Abdur-Rahmaan Janhangeer

Greetings list, Let's say i import module1 import module1 can we do module1() straight out of the box? If not i propose something like if __name__ == '__main__': __file__.callable = True Kind Regards, Abdur-Rahmaan Janhangeer about <https://compileralchemy.github.io/> | blog <https://www.pythonkitchen.com> github <https://github.com/Abdur-RahmaanJ> Mauritius

1 day, 21 hours

Unpacking in tuple/list/set/dict comprehensions

by Erik Demaine

Extended unpacking notation (* and **) from PEP 448 gives us great ways to concatenate a few iterables or dicts: ``` (*it1, *it2, *it3) # tuple with the concatenation of three iterables [*it1, *it2, *it3] # list with the concatenation of three iterables {*it1, *it2, *it3} # set with the union of three iterables {**dict1, **dict2, **dict3} # dict with the combination of three dicts # roughly equivalent to dict1 | dict2 | dict3 thanks to PEP 584 ``` I propose (not for the first time) that similarly concatenating an unknown number of iterables or dicts should be possible via comprehensions: ``` (*it for it in its) # tuple with the concatenation of iterables in 'its' [*it for it in its] # list with the concatenation of iterables in 'its' {*it for it in its} # set with the union of iterables in 'its' {**d for d in dicts} # dict with the combination of dicts in 'dicts' ``` The above is my attempt to argue that the proposed notation is natural: `[*it for it in its]` is exactly analogous to `[*its[0], *its[1], ..., *its[len(its)-1]]`. There are other ways to do this, of course: ``` [x for it in its for x in it] itertools.chain(*its) sum(it for it in its, []) functools.reduce(operator.concat, its, []) ``` But none are as concise and (to me, and hopefully others who understand * notation) as intuitive. For example, I recently wanted to write a recursion like so, which accumulated a set of results from within a tree structure: ``` def recurse(node): # base case omitted return {*recurse(child) for child in node.children} ``` In fact, I am teaching a class and just asked a question on a written exam for which several students wrote this exact code in their solution (which inspired writing this message). So I do think it's quite intuitive, even to those relatively new to Python. Now, on to previous proposals. I found this thread from 2016 (!); please let me know if there are others. https://mail.python.org/archives/list/python-ideas@python.org/thread/SBM3LY… There are several arguments for and against this feature in that thread. I'll try to summarize: Arguments for: * Natural extension to PEP 448 (it's mentioned as a variant within PEP 448) * Easy to implement: all that's needed in CPython is to *remove* some code blocking this. Arguments against: * Counterintuitive (to some) * Hard to teach * `[...x... for x in y]` is no longer morally equivalent to `answer = []; for x in y: answer.append(...x...)` (unless `list1.append(a, b)` were equivalent to `list1.extend([a, b])`) Above I've tried to counter the first two "against" arguments. Some counters to the third "against" argument: 1. `[*...x... for x in y]` is equivalent to `answer = []; for x in y: answer.extend(...x...)` (about as easy to teach, I'd say) 2. Maybe `list1.append(a, b)` should be equivalent to `list1.extend([a, b])`? It is in JavaScript (`Array.push`). And I don't see why one would expect it to append a tuple `(a, b)`; that's what `list1.append((a, b))` is for. I think the main argument against this is to avoid programming errors, which is fine, but I don't see why it should hold back an advanced feature involving both unpacking and comprehensions. Erik -- Erik Demaine | edemaine(a)mit.edu | http://erikdemaine.org/

2 days, 4 hours

PEP able? : Request new 'fn' format specifier to specify relationship between positional arguments

by Shash Srinivasa

Hi, Draft PEP attached: Proposal: def is_between(n1,n2,n3): fn"is number {n1} between {n2} and {n3}" fn"{n2}<{n1}<{n3}?" # the token "fn" immediately following the def statement, speficy the descriptive text # multipe "fn" specifiers describes multiple ways the funtion can be invoked # the "fn" statement is at the same indentation level as the def statement return (n1 > n2 and n1 < n3) . . . # if "fn" statements are found indented and not immediately following def statement, indicate function invocation. # function can be invoked using descriptive text pecified in "fn" statement. if fn"is number {value} between {valid} and {low_threshold}": print("underflow") if is_between(value,low_threshold,high_threshold): print("in range") if fn"{low_threshold}>{value}>{high_threshold} ?" print("overflow") . . Please comment Thanks Shash

2 days, 8 hours

Implementing string unary operators

by Jeremiah Vivian

I posted a previous thread about overloading the unary `+` operator in strings with `ord`, and that expanded to more than just the unary `+` operator. So I'm saying now, there should be these implementations: > +string - `int(string, 10)` (or just `int(string)`) > -string - `int(string, 8)` > ~string - `int(string, 16)` Or: > +string - `string.lstrip()` > -string - `string.rstrip()` > ~string - `string.strip()` If anyone has better ideas, they can post it here.

2 days, 16 hours

Re: Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?

by Alex Waygood

Indeed — we essentially lie to mypy about the method resolution order for list, dict, etc (mypy thinks that list directly inherits from collections.abc.MutableSequence — see the typeshed stub here: https://github.com/python/typeshed/blob/32bc2161a107db20c2ebc85aad31c29730d…), but numeric ABCs are not special-cased by typeshed in the same way as collections ABCs. Fundamentally, mypy has no knowledge about virtual subclassing. Again, I covered this very comprehensively on StackOverflow ;) https://stackoverflow.com/questions/69334475/how-to-hint-at-number-types-i-… Best, Alex > On 16 Oct 2021, at 13:54, Steven D'Aprano <steve(a)pearwood.info> wrote: > > On Sat, Oct 16, 2021 at 09:54:13AM +0100, Alex Waygood wrote: >>>> On 16 Oct 2021, at 06:13, Steven D'Aprano <steve(a)pearwood.info> wrote: >>> Be careful about believing what you are told. >> Indeed, MyPy will correctly raise errors if you assign {None: []} to a >> variable annotated with dict[str, Number]. However, you'll find that >> MyPy also raises an error if you assign {'foo': 4} to a variable >> annotated with dict[str, Number]: > > Hah, serves me right for not testing it for both positive and negative > cases. > > In my very limited testing now, I see that the problem appears to be > with the Number type. mypy correctly accepts this: > > Data = Dict[str, int] > a: Data = {'spam': 42} > > (no errors), but if you change the annotation to use Number instead of > int, it wrongly flags that as a type error. > > Possibly mypy doesn't know that ints and floats are instances of Number? > > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FJY3Q… > Code of Conduct: http://python.org/psf/codeofconduct/

3 days, 19 hours

Real Positional Arguments or OO Currying

by Mathew Elman

I don't know if this has been suggested before, or if this is outlandishly impossible (though I would be surprised if it was), so apologies in advance if so. I have on occasion come across a situation where I use/write a signature like this: def insert_x_into_y(x, y): ... or worse def insert_into(item, container): ... where, despite a driving idea of python syntax being readability in english, the function signature is distinctly not english. "I'll just go and insert into this item that container", is not only never said but is actually ambiguous in english. What would be really cool, is if python let you write function signatures like this: def insert_(item)_into_(container): ... where the arguments dispersed between the function name are positional only argument, and any key word arguments would have to go at the end. It would create a function that could be called as: insert_(1)_into_(my_list) or insert__into_(1, my_list) The purpose of allowing both should be obvious - so that the function can be referenced and called in other places. (Rather than just skipping the brackets the function call with only the end parentheses could have a special stand in character e.g. ., ?, !, _ or other if that was more preferred.) This sort of signature is particularly annoying for boolean checks like `isinstance` (N.B. I am _not_ suggesting changing any builtins), which one could wrap with: def is_(obj)_an_instance_of_(type): return isinstance(obj, type) For precedence in other languages, this is similar to curried functions in functional languages e.g Haskell, especially if each part of a function were to be callable, which would be up for debate. Allowing each part to be called would make sense if each "next" partial function were an attribute on the previous and what it returned, making it a sort of object oriented currying. Then the syntax could be with a `.`: def is_(obj)._an_instance_of_(type): ... is_(1)._an_instance_of_(int) is_._an_instance_of_(1,int)

4 days, 6 hours

Accessing target name at runtime

by Jeremiah Paige

Hello Python community, Earlier this year I put forward a proposal for decorators on variables which I felt was well received in concept, but fell flat on the syntax. One of the responses to that thread was rather than abusing the decorator, to use a new symbol to get the target of an assignment statement as a string on the right hand side. I would call this compile-time reflection of assignment targets, or simply reflection. Although initially continuing on with my original syntax, the idea of using such a token in my own code came up on more than one occasion this year and I wanted to see what it might really look like. From that, this proposal is to add a new token to Python, currently spelled <<<, which, at compile time, is replaced with the string representation of the nearest assignment target, either assignment statement or assignment expression. It does not bind with either keyword arguments or default parameters. Neither does it work in any kind of non-assignment binding. When encountered anywhere other than the right hand side of some assignment, it is an unconditional SyntaxError. Having access to the target at runtime is most often useful in meta programming and the standard library has several factory functions that require the user to repeat the variable name in the function call to get expected behavior. Bellow are some examples of where I believe the reflection token would be used if adopted. >>> Point = namedtuple(<<<, 'x, y, z') >>> Point <class '__main__.Point'> >>> UUIDType = NewType(<<<, str) >>> UUIDType __main__.UUIDType >>> class Colors(Enum): ... Black = <<< ... GRAY = <<< ... WHITE = <<< ... >>> Colors.GRAY.value 'GRAY' >>> HOME = '$' + <<< >>> HOME '$HOME' >>> if error := response.get(<<<): ... if errorcode := error.get(<<<): ... print(f"Can't continue, got {errorcode=}") ... Can't continue, got errorcode=13 To get a feel for using this new token I have created a fork of the 3.11 alpha that implements a *very* incomplete version of this new grammar, just enough to actually produce all of the examples above. It also passes a small new test suite with further examples https://github.com/ucodery/cpython/blob/reflection/Lib/test/test_reflection… .

4 days, 13 hours

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas